• Introduction
• About this series
• Why hybrid? — A single search won't reach your business-related questions.
  • Vector search based on meaning (Milvus)
  • Full-text search by keyword (Elasticsearch)
  • Run them in parallel and integrate them — and the prerequisites for them to really work.
• Rerank selects higher ranks — a two-stage sieve design
  • Collect a larger quantity in the first round.
  • Re-scoring of relevance in the second stage
  • Fail-safe mechanism that prevents response interruption even if the external reranker experiences delays.
• The overall flow — the main stream of KnowledgeNetworkRetriever
• Searching isn't a one-way street — the existence of auxiliary retrievers
• Connecting extraction and search — This is where the discussion from Part 4 comes into play.
• [Things to take home and consider] A sheet for categorizing your company's queries into "meaning-focused/word-focused/mixed"
• Next episode preview
• summary

Introduction

In the previous, fourth installment, we introduced how Super RAG carefully preserves the structure at the entry point of document extraction—its design that incorporates merged table cells and figure captions without disrupting them for subsequent searches. At the end of the article,Extraction plus chunking while preserving structure determines the ceiling of subsequent searches."I summarized it as follows.

The main character this time isHow much of the ceiling space can actually be used?Responsible forsearchThis is the content. No matter how carefully you incorporate the material, you won't get an answer unless the search engine recognizes it as relevant. Conversely, the value of the structure you painstakingly maintained in the fourth round can be multiplied many times over depending on how you design the search section.

This article will explain how Super RAG constructs this search step.Hybrid search that runs both meaning and word axes in parallel.and,Reranking top candidates based on their relevance to the question.--We will break it down in a way that is easy for non-engineers to understand. At the end of the article, we will provide some points to take home and consider:An exercise sheet for categorizing your company's questions into meaning-focused, word-focused, and mixed categories.We have prepared the following for you. Please use it as a starting point to understand the trends in your company's search queries after you have finished reading it.

About this series

This is the fifth article in the series. The series is structured into five chapters: (I) Why Super RAG, (II) How to implement it, (III) Visualizing the contents, (IV) What is happening in the field, and (V) Decision on implementation. Chapter III, "Visualizing the contents," covers extraction (Part 4) → Search (Part 5) → In the order of Answer Strategy (Part 6), we will cover the three pillars that support the accuracy of Super RAG.

The fourth installment isDesigned to prevent any items from being missed at the entrance.If that is the case, then this article isDesigned to extract all captured material without any loss.The 6th installment is "Design to convert extracted information into answers.If you read all three parts together, you will get a clearer picture of how Super RAG builds up its precision.

Why hybrid? — A single search won't reach your business-related questions.

Questions asked in the workplace, even if they look similar,From a computer's perspective, they are completely different things.This is often the case. For example, here are two examples.

A: "What precautions should I take when using a safety harness on site?"
B: "What is the maintenance schedule for model number XYZ-123?"

A wrote somewhere in the document:Precautions when using a safety harnessIt may not be written exactly as "fall protection equipment." It is more likely to be written using other words such as "fall protection equipment," "harness," or "full harness type."Paraphrasing that humans understand to mean "the same thing"If you can't search for it, you won't receive it.

On the other hand, B said,Even a single character difference in the model number makes it a completely different product.That's right. "XYZ-123" and "XYZ-132" are different models, and it should be able to pick up even if it says "XYZ123". If you search based on semantic similarity, similar model numbers will get mixed in as noise.A search engine that strictly tracks the exact match of the written characters.This is necessary. These two questionsSupported by a single type of searchThat is difficult in principle. Super RAG has not given up on this,Search by meaning (vector search)and,Search by word (full-text search)of,Run them in parallel and integrate the results.—This is the design we've adopted. This is a mechanism called "hybrid search."

Hybrid search with 2 lanes

Vector search based on meaning (Milvus)

Vector search finds the "meaning" of a sentence.vectorThis method converts the data into a sequence of numbers called TP1T and measures similarity based on how close the sequences are.MilvusThis uses a vector database.

Both the question text and the chunks in the document are converted to vectors using the same procedure."Sentences with similar meanings tend to cluster together in a vector space."This property allows you to retrieve relevant text. For example, in question A at the beginning, even if the text mentions "harness" or "fall arrest equipment," it will still be considered as a candidate because its meaning is similar to "safety belt."Search engine strong in paraphrasingis.

However, vector search also has its weaknesses. Because it judges based on semantic similarity,Words that require a strict match of characters, such as model numbers, proper nouns, and serial numbers, are prone to being mixed up with similar but incorrect terms.A typical example is in question B, where "XYZ-123" and "XYZ-132" are placed in the same semantic group.

Full-text search by keyword (Elasticsearch)

Full-text search breaks down the text into words."This word is included here."This method uses an index to find matches based on word correspondence.ElasticsearchIt employs a system that has been used in the industry for many years.BM25This scoring method measures the strength of the hit.

Because it's word-based, it excels at handling model numbers and proper nouns like B. It can accurately extract only the parts that say "XYZ-123," and it also takes into account the frequency and rarity of its appearance in the document to bring the most likely relevant part to the top.Search engine strong in precise terminologyis.

Conversely, they are weak at questions like A, which have many possible paraphrases.If the words written and the words in the question are different, it won't be a match, no matter how similar their meanings are.—This is a typical example of getting stuck when relying solely on full-text search.

Run them in parallel and integrate them — and the prerequisites for them to really work.

Super RAG uses these two methodsInstead of running them sequentially, run them in parallel and merge the results from both.It works in parallel, so there's no delay in selecting one and then switching to the other. Both results are combined into a single scored candidate list and passed to the next stage. If the same location is hit by both, they are grouped together into a single candidate.

There is one design point I would like to convey through this article. The concept of hybrid search itself is widely known in the industry. However,It's not as simple as saying, "If you run both in parallel, you'll automatically get good results."—That's the reality we're seeing in the implementation field. For both methods to perform at their best,① Quality of the materialsand,② Optimizing the merging methodThese two elements must be present, and Super RAG incorporates the know-how accumulated through business use into this process.

① Quality of the materialsThis is exactly what we introduced in the fourth installment. It's thanks to the high-precision extraction centered around DocReader and the chunking tailored to the nature of the source material that both vector and full-text searches are possible.You can go and read the same high-quality material.If table cell merges are broken, figure captions are separated from the text, or semantically separate chunks are mixed together, both methods will pick up noise. Conversely,Because the materials are prepared during the extraction stage, both lanes of the hybrid search can leverage their respective strengths.—That's the relationship. In many cases where hybrid search is introduced on a typical RAG platform but the expected accuracy is not achieved, it's not the search method itself,Material quality in the preceding sectionThat is the cause.

②Optimizing the merging methodThis is the culmination of the tuning that Super RAG has accumulated through business use. Vector search and full-text search use fundamentally different scoring methods, and if you simply list the scores from both and rank them, the results will be biased towards one method. How to integrate candidates that point to the same location, how to equalize the scores from both, and what combination to use in the final ranking—this know-how in merging directly impacts the quality of the results.

The design implications are simple."Business-related questions are a mix of questions focusing on meaning and questions focusing on specific words."In line with this reality, the search engine will handle both. Users will not need to sort their questions themselves into meaning-oriented or word-oriented.If you simply write a question, both lanes will accept it.—That is the biggest operational advantage.

Rerank selects higher ranks — a two-stage sieve design

Once hybrid search has gathered candidates, the process doesn't end there. Super RAG hasRerankThis includes an additional selection process. The concept of Rerank is similar to human document screening. The first stage is reviewing application documents.Put in a generous amountThen, they carefully compare and re-rank them in the second interview—this"Gather them widely, then rearrange them later."That's the idea.

Hybrid search + rerank funnel

Collect a larger quantity in the first round.

In the hybrid search section, Super RAG isGather more candidates than the final number needed.We have a setting for this. For example, even if we ultimately want to pass 5 items to the LLM, we initially select a wider range, such as 20 or 30 items.

The reason why people intentionally pick up more than they need is...Vector search scores and full-text search scores use "different measuring sticks."That's because a score of 0.85 for vector search and a score of 0.85 for full-text search are not the same number. If you select the top N results based solely on the score, you'll end up with a bias towards one method or miss out on truly relevant candidates.First, leave plenty of room to spare, and leave the final sorting to the second stage.That's the pragmatic approach I've taken.

Re-scoring of relevance in the second stage

The main character of the second act isRerankerThat's it. The reranker will ask each of the candidates gathered in the first round questions to the users.Read in pairs"How well-suited is this candidate to answer this question?"Direct scoring in relation to the questionTo do.

Vector search and full-text search encode and compare the question and chunks independently. In contrast, a reranker encodes the question and candidates...Place them side by side and compare them.Therefore, the computational complexity will increase.The accuracy of the judgment will improve dramatically.Differences in the primary measurement criteria can be standardized to the same evaluation standard in the second stage.

Inside Super RAG, this rerankerInterchangeable modulesIt is designed as follows: Specifically,A method that directly calls the Rerank API provided by Cohere.,A method of calling the same Cohere reranking model via hosting on Azure.,A method that uses the Cohere reranker or LLM itself as a reranker via the LlamaIndex framework.—We have these options internally.

However, these are not designed to be switched between on a per-customer basis.During operation, the Super RAG development team selects and provides the single best option available at that time.From the customer's perspective, this means that the development team is always guaranteeing that the highest quality re-ranking process is in effect.

The intention behind modularization lies elsewhere. The tools and libraries surrounding RAG, including the relauncher, are evolving rapidly.More accurate models and more latency- and cost-effective options may emerge in the future.Super RAG will continue to investigate these trends.The flexibility to replace the internal components when a better reranker appears.We continue to maintain this as part of our design. The advantages for our customers are,By using Super RAG, you can enjoy the best reranker available at that time.—That's the point.

Fail-safe mechanism that prevents response interruption even if the external reranker experiences delays.

Reranker is from inside Super RAGCall it as an external API service.It's a component. Cohere and Azure hosting usually operate stably,Global peak usage times and temporary load conditions on the service sideDepending on the situation, there will inevitably be moments when the response is significantly slower than usual. This is an unavoidable characteristic when relying on external APIs.

If we were to design it so that we "dutifully wait for the reranker who never returns," the delay on the reranker's side would remain unchanged. Overall response delay for Super RAG This directly leads to, in the worst caseNo response was given.This could lead to serious problems. Super RAG avoids this,A limit will be set on the waiting time for reranking. If the limit is exceeded, reranking will be skipped and the main process will proceed.--FailsafeIt incorporates this feature.

As a result, even if delays or failures occur on the reranker side,We will continue to respond to our customers.The accuracy of the rankings will be somewhat rough, but stillI will definitely give you an answer.—That is the intention behind the design. This is essential for systems that have components that depend on external APIs.Ensuring redundancyTherefore, Super RAG is one of the important features that makes it suitable for business use.

The overall flow — the main stream of KnowledgeNetworkRetriever

Up to this point, we have looked at hybrid search and reranking separately. In actual Super RAG, these two, plus several refining processes, are bundled into a single pipeline.KnowledgeNetworkRetrieverA search engine called the main stream is running. In this section, we will trace how this main stream flows from beginning to end, from the user's perspective.

KnowledgeNetworkRetriever Mainstream Flow

1. Apply a filterWhen a question arrives, firstSearch rangeThis narrows the search. Specifications such as "only within this folder," "only this group of files," or "only chunks that satisfy this metadata" are passed from the upstream query management (an extension of authentication and user permissions, as introduced in Part 3). By narrowing the scope, subsequent searches avoid looking at unnecessary areas.

2. Hybrid search and mergeFor the narrowed range, we run semantic search and full-text search in parallel, as discussed in this article, and then integrate the results. Candidates that refer to the same section are combined into one, and the internally held data...Chunk window (a block cut to a wider width)However, to show to the userText for displayIt can be neatly mapped to this.

3. RerankAs seen in this article, the top items are reordered by relevance. If the external reranker is delayed, the failsafe will kick in and the reranking will be skipped, and the process will proceed to the next step with the main stream's ranking.

4. Keep it within a readable length (fit context length).There is a limit to the number of characters (context length) that LLM can read at once. If you pass all the collected candidates, it will overflow,OpenAI's publicly available token quantification libraryWhile measuring the length, pack them in from the top down, stopping just before they fit.

5. Returns the scored node.The candidates who have made it this far are,List with score, source, and metadataThis is then passed on to the answer generation (the topic for the 6th session).

From the user's perspective, it's simple.When you ask a question, you get a list of candidates organized in order of relevance, with a consistent length."—The KnowledgeNetworkRetriever's design is such that the five processes described above are folded into this structure."

Searching isn't a one-way street — the existence of auxiliary retrievers

Up to this pointSearching the main streamRegarding this matter, Super RAGDedicated entrance for each material's propertiesAn auxiliary search tool with the following features is also available. The design philosophy is consistent with the 4th edition (where the chunkers differ depending on the material)."Even within the same search process, the optimal method of searching differs depending on the nature of the subject."That's the idea.

I'll introduce some representative examples without going into too much detail.

• DictionaryErrorThis is a search engine that retrieves only chunks of terms from glossaries and dictionaries via a separate path. By using a different route than the main text search, it retrieves definitions of terms.Without being lost in the document body hitsYou can retrieve it. This is the starting point for the "Dictionary Strategy" that we will cover in the sixth lesson.
• Attachment RetrieverThis is a search engine that targets files attached to each chat (attachments used only within that conversation, not permanently registered in a folder).Attachment operationsIn conjunction with categories,"Regarding the attached application form"It supports limited references like this.
• DynamicFewshotRetrieval: Close to the questionExample Q&A pairsThis is a search engine that retrieves information. By providing LLM with several example answers like "this is how you answer this type of question," it helps to stabilize the answer style.

Having these auxiliary retrievers available means"Process all questions using only mainstream hybrid search."Going beyond that simple idea,Choosing the right approach based on the nature of the question and the subject matter.The ability to do this is a characteristic of the Super RAG search layer. When each auxiliary retriever is called and how it contributes to the answer is...Answer strategies for each question type in the 6th sessionThen I step in.

Connecting extraction and search — This is where the discussion from Part 4 comes into play.

Here, I will summarize, just once, how the fourth installment (extraction) and this article (search) are connected.

At the end of the fourth session, Super RAGChoose chunkers according to the properties of the material.We introduced designs such as FAQChunker, which is dedicated to FAQs, and DictionaryChunker, which is dedicated to dictionaries.

These are the main hybrid search methods discussed in this article, as well as auxiliary retrievers such as dictionaries.How to use the search barThis is the premise.Because the glossary is identified as "term-definition pairs," DictionaryRetriever can retrieve entries in meaningful units..Because FAQs are identified and chunked as FAQs, hybrid search can evaluate them without confusing them with the main text..

In other words,Extraction builds the foundation / Search is a one-time use / Answer strategy is selectionThis is a three-stage structure, and the precision design of Super RAG is that each stage properly utilizes the work of the previous stage. The fourth stage of structure-preserving extraction is,This is the first time it is retrieved in this search stage.—The methods of retrieval are the hybrid search, rerank, and auxiliary retriever that we've looked at in this article.

[Things to take home and consider] A sheet for categorizing your company's queries into "meaning-focused/word-focused/mixed"

Up to this point, hybrid search hasThe two axes of meaning and wordWith that in mind, RerankGather information broadly and sort by relevance.We've introduced the mechanism. Thank you for reading.Which axis would be more effective in our line of work?You might be thinking, "This..."To get an ideaWe have prepared a simple sheet for this purpose.

Practice sheet for sorting your company's queries

From five perspectives, each is:Tasks/Deliverables/TipsWe will focus on this point.

Perspective 1: Collect recent queries

Questions actually asked to AI and search engines in the past month,10-20 itemsWe collect a fair amount of data, including internal assistants, help desk tickets, and support inquiry logs."The exact words that were actually used in practice."The key is to collect them.
Tips: Not a virtual query thought up in your head,Actual logTaking data from this source significantly improves your understanding of the trends.

Perspective 2: Sorting into 3 lanes

The collected questions are divided into three lanes: "meaning-focused," "mixed," and "word-focused." The criteria for judgment are simple.

• MeaningThe question asks about the concept, situation, and procedure. The answer is alsoIt can be rephrased.Examples: "What precautions should I take when using a safety harness on site?" "Please explain the expense reimbursement process."
• Word-relatedThe question isThe specific word itselfThe answer must include (model number, clause number, proper noun, abbreviation) and the word must also appear in the answer. Examples: "Maintenance cycle for model number XYZ-123", "Infrastructure requirements for Super RAG".
• mixture: Possesses both properties. Example: "Does XYZ-123 conform to safety harness standards?" (The model number is word-based, while the conformity judgment is semantic-based.)

TipsIf you're unsure:Add to the mixtureThat's fine. The more mixed data types you have, the greater the benefits of hybrid search become.

Perspective 3: Understanding the effective points from the distribution.

We'll look at the distribution of the sorted results. This is less about the customer setting something up,The most effective approach among the Super RAG methods will vary depending on the customer's business query patterns.Please consider it from that perspective.

• Many words are close to the original.Tasks (primarily model numbers, serial numbers, and proper nouns):Full-text search and rerankingThis is where it shines. A two-stage filtering process—precise keyword searches that avoid mis-hits due to incorrect model numbers, and a process that brings truly relevant results to the top—supports the accuracy of the answers.
• Mostly meaning-basedTasks (focusing on procedures, concepts, and situations):Vector search and preceding chunk designThis is where it really shines. The chunking method, tailored to the properties of the material, which we introduced in the fourth installment, underpins the accuracy of semantic searches.
• Many are mixedWork (This is where most of the actual work is done):Both lanes of hybrid search are available.That's the key to its effectiveness. Whether the question requires meaning or a specific word, the user can ask it without having to consciously categorize it—that operational simplicity is what makes it so effective.

TipsInstead of assuming that "our approach is word-centric" or "meaning-centric,"The decision is based on the distribution of actual queries.That's important. There's often a discrepancy between assumptions and reality.

Perspective 4: Extract "hits that were different from what was expected" into a separate table.

Of the sorted questions, in the existing system"The expected answers weren't provided," "Irrelevant points came up at the top of the results."We will extract only the relevant items into a separate table. This will be material directly related to the theme of the 6th session (answer strategies for each question type).
TipsIf you write down what should have been output and what was output instead, you can use them directly as validation queries in PoCs and trials.

Perspective 5: Consultation on verification during the trial.

Once the sorting is complete, the most reliable way to verify the results is to actually run a Super RAG search on real data. The specifics such as scope, period, and number of target files will vary depending on the business requirements.Please use the contact form at the end of the article.Please feel free to contact us. Our sales team will provide suggestions tailored to your specific situation.
TipsIf you could summarize and share the results from perspectives 1-4, it would greatly streamline the process of designing what to test in the initial trial.

Next episode preview

Next time (Episode 6),Answer strategies for different question types We will deliver this to you. In this article,How to sort search resultsWe covered "[this topic]" but next time we will discuss the assembled candidates.How to translate this into an answerThe main point is the judgment made by the answer generator.

Specifically, we will delve into the philosophy behind Super RAG, which automatically categorizes questions and selects the appropriate response strategy for each.DictionaryErrororDynamicFewshotRetrievalIts applications will be revealed in the sixth installment.

By reading the 4th (extraction) → 5th (search) → 6th (answer strategy) as a set, you will understand Super RAGConsistent quality design from entry to exitThe outline becomes visible.

summary

The search uses the materials imported in the fourth session.The hurdles to actually getting the answer generatedThe search layer of Super RAG isThe two axes of meaning and wordRun them in parallel,The broadly collected candidates are re-ranked and sorted by relevance.—This two-step filtering process allows us to address the diversity of business-related questions. Furthermore, the mainstreamKnowledgeNetworkRetrieverIn addition to hybrid search and reranking, it integrates refinement processes such as filtering the reference range and adjusting the context length into a single process, and addresses the latency of external rerankers.Do not stop the response with fail-safe mode.By incorporating redundancy, it demonstrates to users the simplicity of "just ask a question and you'll get well-organized answers." If you're wondering "how effective are each of these axes in our specific business queries?", please first consider the three-lane categorization outlined in this article. If you then wish to verify this with actual data,Please feel free to contact us using the inquiry form at the end of the article.You can compare the results by running your own data through free or paid trials.

<Articles in this series>

• [Part 1] Achieving a Quality Beyond Standard RAG — Technical Innovations Supporting Super RAG
• [Part 2] A Thorough Explanation of Three Embedded Patterns — Analyzing API Call Flows and Dify Expense Review
• [Part 3] API Function Catalog — What can be done, and what should be handled in-house?
• [Part 4] The Contents of Document Extraction — How is the Engine Selected? (Scheduled for release in June 2026)
• [Part 5] Hybrid Search and Reranking — Making the Most of Extracted Content by Balancing "Semantic" and "Word" (Scheduled for release in June 2026)
• [Part 6] Super RAG Answer Strategy — Design Decisions for Mastering Single-hop and Three Techniques to Support Answer Quality (Scheduled for release in June 2026)