What is reranking? How AI picks which sources actually get used

Reranking is the second stage in how an AI decides which sources actually get used in an answer. After a fast first stage has retrieved a broad batch of candidate sources, a dedicated, more precise model gives each of them a new relevance score and re-orders them. In short: retrieval finds the candidates, reranking decides the order — and only the top makes it into the answer.

Why retrieval is split into two stages

This is about a trade-off between speed and precision. The first search uses embeddings and a vector database to find candidates lightning-fast among millions of documents — but the goal there is to not miss anything relevant, not to rank perfectly. So it typically retrieves 50–100 candidates that sit roughly in the right area. Reranking then takes just this small batch and runs a heavier model over each one to determine real relevance. Running the heavy model over the whole database would be far too slow; running it over 100 candidates is manageable. This pattern is called two-stage retrieval.

How reranking works

Rerankers are usually cross-encoders. Where the fast vector search embeds query and document separately and compares them afterwards, a cross-encoder reads the query and candidate text together and outputs a single relevance score for the pair. Because it sees both texts at once, it catches nuances in context — which of two apparently similar chunks actually answers what was asked. That makes it slow but precise, which is exactly why it is used last, over a small batch. Commercial rerankers such as Cohere Rerank are common choices in production.

Why it matters for AI visibility

Here is the key lesson: being retrieved is not enough — you have to survive reranking. Your content may well land in the first batch of 100 candidates, but if a competitor answers the actual question more precisely and directly, you get scored down in reranking and never reach the answer. Two chunks can lie equally close to the query in embedding space, but the reranker chooses the one that answers the question most precisely. So being "near" the topic is not enough — you have to hit the question.

How you can use this

Write so that each passage answers one concrete question directly and completely, without the reader having to hunt. Put the answer first (BLUF), use the language the question is actually asked in, and avoid burying the key point in long, general paragraphs. The more precisely a chunk answers the question, the better it survives reranking. CitationLab Monitor with Chunkalyzer helps you see which passages are relevant enough to make it through — and which get retrieved but filtered out. Want to know whether your content survives reranking today? Start with a review of your AI visibility.