# Rerank your RAG results

Reranking is a second pass over your retrieved chunks: a query-aware model
re-scores the top candidates and reorders them, so the passage that
answers the question sits first instead of fourth or tenth. You add it between
retrieval and the answer, and it lifts answer quality for a small added latency.

This guide explains what reranking fixes, how a cross-encoder
scores a passage, and how to wire reranking into a Claude Code workflow through
an MCP retrieval server. It assumes you already have a retrieval pipeline. If you
do not, read [RAG fundamentals](/guides/rag-fundamentals) first.

## What problem does reranking fix?

Vector search ranks chunks by [embedding similarity](/guides/embeddings-for-beginners). That score is a coarse proxy
for relevance, computed by comparing two vectors that were each encoded without
seeing the other. It is good enough to pull a broad set of candidates, and weak
at deciding which of those candidates is the single best answer.

Three problems follow from that:

- **The right chunk is present but not first.** Models attend best to the start
  and end of a long context and worst to the middle. If the answer chunk lands at
  position seven, the model reads past it. (See
  [feed Claude the right context](/guides/feed-claude-the-right-context).)
- **Near-misses crowd the top.** Passages that share vocabulary with the query
  but do not answer it score high on similarity. They push real answers down.
- **You cannot just send more chunks.** Sending the top fifty fills the window
  with noise, raises cost, and makes the answer chunk harder to find, not easier.

Reranking addresses all three by reordering the candidates with a model that
reads the query and each passage together.

## Two-stage retrieval

Reranking turns retrieval into two stages.

**Stage one casts a wide net.** Use vector search, or
hybrid search,
and fetch a generous candidate set, around 50 passages. This stage optimises for
recall:
you want the answer to be somewhere in the set.

**Stage two reorders for precision.** Pass those candidates to a reranker, which
scores each one against the query and sorts them. Keep the top handful, around
five, and send only those to Claude.

The split matters because the two stages have different jobs. Stage one is fast
and approximate so it can score thousands of chunks. Stage two is slower and more
accurate, and it only runs on the few dozen survivors from stage one. You get the
breadth of cheap retrieval and the judgement of an expensive model without paying
the expensive cost on every chunk.

## How does a cross-encoder score a passage?

The reranker model used here is a cross-encoder. The name describes the
mechanism, and the contrast with embeddings is the whole point.

A bi-encoder
(what your embedding model is) encodes the query and the passage separately, into
two vectors, then measures the angle between them. Because each text is encoded
alone, the model never sees how a specific word in the query relates to a
specific word in the passage.

A cross-encoder feeds the query and the passage into the model together, as one
input. Every token in the query can attend to every token in the passage. The
model reads them as a pair and outputs one number: how relevant this passage is
to this query. That joint reading is why it judges relevance more precisely.Cross-encoders have a fixed input limit, often around 512 tokens for query plus passage combined. A chunk longer than that gets silently truncated before scoring, so an answer in the tail of a big chunk is never read and ranks low for no visible reason. Keep chunks within the reranker's window, or check its max-token spec before you trust the scores.

| Property | Bi-encoder (embeddings) | Cross-encoder (reranker) |
| --- | --- | --- |
| Input | Query and passage encoded separately | Query and passage encoded together |
| Output | A vector per text | One relevance score per pair |
| Precomputable | Yes, index passages ahead of time | No, depends on the query |
| Relative cost | Low | Higher per pair |
| Best role | Stage one, score thousands fast | Stage two, score the top candidates |

The trade-off is built in. You cannot precompute cross-encoder scores, because the
score depends on the query you do not have yet. So a cross-encoder is too slow to
rank a whole corpus, and well suited to rank a short candidate list. That is
exactly the two-stage split above.

## How many candidates should I rerank?

Two numbers control reranking: how many candidates stage one fetches, and how
many survivors you send to Claude.

- **Candidate count (stage one output).** Larger sets give the reranker more
  chances to find a buried answer, at the cost of more pairs to score. Start at 50
  and adjust against your evaluation set.
- **Final count (what reaches Claude).** Smaller is usually better. Once the
  reranker has put the answer first, sending three to five strong passages beats
  sending fifteen mixed ones. Fewer passages means less noise and lower cost.

A reasonable starting point: fetch 50 candidates, rerank them, send the top 5.
Then move the numbers and watch your metrics, rather than guessing.The quiet failure is fetching only 5 candidates and reranking those 5: the reranker can reorder them but cannot pull in the answer chunk that ranked 23rd, so it changes almost nothing and looks broken. The candidate count must be much larger than the final count (50 in, 5 out) for stage two to have buried answers to rescue. If reranking gives you no measurable lift, check this gap before you blame the reranker model.

## How do I wire reranking into Claude Code?

You do not write model-calling code for this. You expose a retrieval tool to
Claude Code through an [MCP server](/guides/mcp-fundamentals), and reranking lives
inside that tool. When Claude needs to search your data, it calls the tool, the
tool retrieves and reranks, and Claude receives passages that are already in the
right order.

Several community MCP servers already implement this two-stage shape. They expose
a single search tool and run hybrid retrieval plus a reranker behind it, so the
agent never sees the internals. Examples include code-search servers that index a
repository and return reranked passages, and document servers that do the same for
a markdown vault.

Register one the same way as any MCP server. In your project config:

```json
{
  "mcpServers": {
    "docs-search": {
      "command": "npx",
      "args": ["-y", "your-rerank-search-server", "./docs"]
    }
  }
}
```

Then confirm Claude Code sees the tool:

```bash
claude mcp list
```

Once it is connected, you do not invoke reranking by hand. You ask a question and
let Claude call the search tool. The reranking happens inside the tool call.

```text
Search the project docs for how the checkout flow handles a declined card, then summarise the steps and cite the file each step comes from.
```

If the server reranks correctly, the passages Claude cites are the ones that
directly describe the declined-card path, not passages that merely mention
"checkout" or "card" somewhere in the text.

## Reranking and hybrid search work together

Reranking and hybrid search solve different parts of the same problem, and pair
well.

- **Hybrid search** widens stage one. It runs [keyword search alongside vector
  search](/guides/hybrid-search-for-rag), so exact tokens like `ERR_CARD_DECLINED` or an order ID get caught even
  when the embedding misses them. This is about getting the right chunk into the
  candidate set.
- **Reranking** sharpens stage two. It reorders whatever stage one produced. This
  is about getting the right chunk to the top of the set.

A common arrangement: hybrid search returns 50 candidates, a reranker reorders
them, and the top 5 go to Claude. Hybrid search owns recall; the reranker owns
the final order.

## Measure the gain, do not assume it

Reranking adds a model call to every search, so confirm it earns its latency
before you keep it.

Evaluate the retrieval stage on its own as well as the final answer. A good model
handed badly ordered context produces a wrong answer that looks like a model
failure, so you have to measure ordering directly.

- **Hit rate at the final count.** Of your test questions, for how many is a
  relevant passage inside the top 5 you send to Claude? Run it with
  reranking off, then on, and compare.
- **Position of the answer chunk.** Track where the known-good passage lands
  before and after reranking. If reranking moves answers from position eight to
  position one, that is the win, made visible.
- **Latency budget.** Time a search with and without the rerank step. If your
  candidate set is large and the gain is small, lower the candidate count or
  rerank fewer items.

You can ask Claude Code to build this harness against a small labelled set:

```text
I have a list of test questions and the file each one should be answered from, in eval/questions.jsonl. Write a script that runs each question through the search tool twice, once with reranking disabled and once enabled, and reports the hit rate in the top 5 and the median position of the correct passage for each mode.
```

## When is reranking not the bottleneck?

Reranking reorders the candidate set. It cannot conjure a passage that stage one
never fetched. If your evaluation shows the answer chunk is missing from the
candidates entirely, reranking will not save you, and the fix lives upstream:

- **The answer is not in the candidate set.** Raise the candidate count, or add
  hybrid search so exact terms are not lost.
- **Chunk boundaries split the fact.** If the answer spans two chunks and neither
  is self-contained, fix chunking before touching the reranker. See
  [RAG fundamentals](/guides/rag-fundamentals) for chunking.
- **The index is stale.** A reranker cannot reorder content that was never
  indexed. Re-index as the source changes.

Reranking is the last 20 percent: it makes a candidate set that already contains
the answer reliably surface it. Get retrieval and chunking right first, then add
reranking to sharpen the order. Next, read
[RAG for coding agents](/guides/rag-for-coding-agents) for how the standard recipe
shifts when the corpus is a codebase.