Why One Retrieval Method Is Not Enough
Keyword search remains the right tool for product identifiers, error codes, project names, people, and regulatory clauses. It is explainable, operationally mature, and straightforward to tune with field boosts, synonyms, and query rules. Its weakness appears when users describe a problem in language that differs from the source document. A query such as “customer cannot finish signing in” may need to retrieve a document written around “authentication failure,” even though the terms barely overlap.
Semantic retrieval addresses that gap by representing queries and content as vectors. It can connect paraphrases, cross-department terminology, and mixed Chinese-English language, but it may underweight a critical model number, region, or software version. Reranking solves a different problem: after candidates have been retrieved, a model reads the query together with each candidate and makes a finer relevance judgment. These methods should not compete for ownership of the whole search experience. Each belongs at a different stage of the pipeline.
Retrieve Broadly, Fuse Carefully, Then Rerank
A practical design runs keyword and vector retrieval in parallel, merges their candidates, removes duplicates, and sends a limited set to a reranker. Avoid comparing raw BM25 scores directly with vector similarity scores because they do not share a calibrated scale. Rank-based methods such as reciprocal rank fusion are a dependable starting point. Learned weights or query-specific routing can come later, once the team has representative relevance labels and understands where the simpler approach fails.
The number of candidates passed to reranking is an engineering trade-off. Too few candidates can exclude the correct document before the strongest model sees it. Too many increase latency, inference cost, and timeout risk. The retrieval stage should optimize for not missing plausible answers, while the reranker should improve precision near the top. Chunking and deduplication are equally important: without document-level grouping, several neighboring passages from one document can crowd out useful alternatives.
- Analyze the query: detect identifiers, dates, departments, document types, and natural-language intent before choosing filters.
- Retrieve in parallel: preserve exact evidence in the keyword index while vectors capture paraphrases and conceptual similarity.
- Fuse rankings: combine candidates using rank positions rather than uncalibrated scores, then remove duplicate documents and overlapping chunks.
- Rerank selectively: apply a cross-encoder or another relevance model only to the most promising candidates.
- Assemble results: group passages by document and show useful snippets, sources, versions, and applied filters.
Design for Enterprise Permissions, Data, and Models
Access control must be part of retrieval, not a visual mask applied just before results are displayed. Whenever possible, filter by user, group, department, or classification while retrieving candidates. Unauthorized content should never enter reranking, summarization, or diagnostic logs. If the vector service cannot express the required permissions, compensate through index partitioning, security metadata, or a controlled query layer. Also test how quickly permission changes propagate to the index; correct rules with stale membership data are still unsafe.
Model selection should not be based only on a public benchmark. Chinese tokenization, traditional and simplified variants, English abbreviations, internal codes, and tables can materially change retrieval behavior. Evaluate embedding models on the organization’s own query-document pairs. For rerankers, consider supported input length, batching, deployment location, and tail latency. A general-purpose language model can make nuanced judgments, but applying it to every candidate often makes cost and behavior harder to control. A smaller specialized reranker is frequently a better fit for high-volume interactive search.
- Chunk with structure: split around headings and semantic boundaries while retaining document IDs, pages, versions, and parent relationships.
- Test bilingual behavior: include mixed-language queries, abbreviations, character-width variants, synonyms, and internal product names.
- Weight fields deliberately: titles, document types, valid versions, and body text need distinct signals, but recency should not automatically defeat relevance.
- Provide graceful fallback: if vector retrieval or reranking times out, return permission-safe keyword results instead of failing the entire request.
Evaluate the Pipeline So Failures Are Diagnosable
Search quality should not be judged from a few queries chosen by the implementation team. Build a repeatable evaluation set from real work patterns, including exact identifiers, vague questions, cross-language phrasing, multiple intents, and queries with no valid answer. Domain reviewers should label relevance. Measure whether retrieval found the correct material before measuring whether reranking placed it near the top. Recall, MRR, and nDCG can support comparisons, but qualitative review remains necessary because a metric may not expose outdated or misleading content.
Production telemetry should reveal the stage at which a useful result disappeared. With appropriate privacy controls, record the query, filters, candidate sources, fused ranks, reranker outputs, final interactions, and zero-result events. Logging only the final list makes it difficult to distinguish an indexing gap from an access-control filter, weak query interpretation, failed vector recall, or a poor reranking decision.
Roll out through shadow traffic or a controlled audience, then inspect latency, cost, ranking stability, and permission correctness before expanding. The strongest enterprise search architecture is not the one with the most models; it is the one in which every stage can be measured, explained, degraded safely, and replaced. When the sources span ERP, CRM, file repositories, and knowledge platforms, an experienced integration team can also help establish consistent identity, metadata, and testing boundaries across those systems.