Searchplex | When Reranking Becomes a System Boundary<br>Searchplex is presenting at Berlin Buzzwords 2026. Meet us there.

Search Stack Audit<br>Search Stack AuditBrowse services, solutions, company pages, and the full Vespa consulting tree.

Services<br>Solutions<br>Industries<br>Case StudiesBlogCompany

Production Search<br>When Reranking Becomes a System Boundary<br>A reranker only sees what retrieval allows to survive. That changes evaluation, system boundaries, and ownership of relevance.

Ravindra HarigeFounder at Searchplex

May 25, 2026Production Search

The reranker's quality ceiling is defined by the candidate set it receives from retrieval. This becomes most visible in external rerankers, but it applies to any multi-stage ranking system where retrieval and ranking are separated. Once this ceiling is reached, further gains rarely come from model tuning alone. They require changes to retrieval, candidate generation, or the structure of the pipeline itself.

Retrieval defines eligibility, reranking defines order

Before reranking, the system has already decided what is eligible to rank: query interpretation, retrieval strategy, lexical and semantic fusion, filtering, permissions, and latency constraints all shape the candidate set.

If a document is not retrieved, it never participates in ranking, and no downstream stage can reintroduce it.

This creates a hard constraint: reranking can only reorder what retrieval surfaces. It cannot recover missing documents. As a result, retrieval quality and reranking quality are different properties:

retrieval controls what enters the system (recall)

reranking controls how that subset is ordered

A strong reranker can produce good top-k results even when retrieval is incomplete. But the system may still miss relevant documents entirely, even if ranking quality looks strong.

The external reranker tradeoff

External reranking typically follows a simple pattern: retrieve candidates, compute features, and reorder results with a stronger model. This works well when relevance can be inferred from document-level signals like text, metadata, or embeddings.

The limitation appears when ranking depends on query–document interaction structure built during retrieval. The retrieval stage can compute rich signals: term matches and coverage, field-level contributions, proximity and alignment, query-specific scoring components that describe how the document matched the query, not just what it contains.

External rerankers typically do not receive this full interaction structure. They operate on a reduced representation: document text, metadata, embeddings, and a limited set of precomputed features. This produces a structural tradeoff:

more candidates → less per-candidate retrieval context

more context → fewer candidates

Reranking becomes compensatory under constraint

Two-stage retrieval is a standard production pattern because full-corpus scoring is too expensive at query time. The shift happens when reranking stops refining a good candidate set and starts compensating for weak retrieval. In this regime, performance gains often come from increasing the rerank window rather than improving retrieval itself. The transition is observable: a system where ranking quality improves by widening the rerank window, rather than by improving retrieval, is a system where reranking has become compensatory. The window size is no longer a latency knob; it is load-bearing. Retrieval is no longer the quality driver; it is the constraint.

A query like red waterproof hiking shoes size 11 illustrates the ceiling. Lexical retrieval may preserve exact constraints but miss semantically relevant products. Semantic retrieval may capture relevant footwear but lose attribute precision. The reranker cannot recover what was never retrieved. In production, this often shows up as stable top-3 relevance scores while long-tail recall silently degrades across query variants.

Ranking is a projection, not a recomputation

Retrieval does not just filter documents: it computes a rich query-time representation: term matches, field contributions, proximity scores, BM25 components, semantic embeddings. That computation happens once, against the full index, under the constraints of query time.

Reranking does not redo that computation. It operates on whatever survives into the candidate set: document text, metadata, a reduced feature vector, maybe a few exported signals. The retrieval-time computation is not rerun: it is partially preserved, partially discarded, and the reranker works with what remains.

Ranking in a two-stage system is therefore not a single scoring function applied to documents. It is a projection: retrieval-time signals compressed into a candidate representation, then re-scored under a different model with different inputs. That projection is lossy by construction: not a failure of implementation, but the structural consequence of separating retrieval from ranking. Every gain the reranker makes operates within...