# RAG for Coding Agents

Apply textbook RAG to a codebase and it quietly underperforms. Code is not prose,
and the retrieval that works for documents works against you here. This guide is
about why, and how real coding agents find the context they need. If the
pipeline is new to you, read [RAG fundamentals](/guides/rag-fundamentals) first.

## Why is code retrieval different from text RAG?

Three things make code retrieval its own problem:

- **Structure is meaning.** A function, class, or method is a unit. Fixed-size
  chunking splits a function down the middle or merges unrelated code, and either
  one destroys the chunk's usefulness.
- **Exact symbols beat paraphrase.** Code questions are often "where is
  `refreshToken` called?" The exact name matters more than semantic similarity,
  which is the opposite of prose, where paraphrase is the point.
- **The answer is spread out.** Meaning lives across call graphs, imports, and
  type definitions in different files. One chunk rarely holds the whole story.

## How do coding agents retrieve code?

Two design philosophies have emerged, and the contrast is instructive.

**Search the live code, no index.** Claude Code deliberately does not build a
vector index. It searches the way a developer does: ripgrep and glob to find
things, then read the files. Its team has been public that an earlier indexed
approach was beaten by agentic search, for four reasons. Precision: exact matches
with no fuzzy false positives. Simplicity: no index to build or maintain.
Freshness: a prebuilt index drifts the moment you start editing. Privacy: nothing
leaves the machine to be embedded.

**Index with structure.** Cursor takes the other path: it hashes files to detect
changes, chunks with syntax-aware (tree-sitter) splitters, stores vectors, and
combines semantic search *with* grep. Cursor reports this beats grep alone on
codebase questions, with the gap widening on very large repositories.

Both are real positions. The honest summary is that exact, structural search is
the backbone, and embeddings help most at scale.

## How should I chunk code for retrieval?

If you do index code, [do not chunk by line count](/guides/chunk-your-documents). Use the syntax tree: split at
function, class, and method boundaries so each chunk is a coherent unit. Research
on AST-aware chunking (recursively splitting large nodes and merging small
siblings under a size budget) shows real gains on code retrieval and on
end-to-end coding benchmarks over naive splitting.Reaching for a fixed token or line count is the prose-RAG habit that breaks here: it slices a function mid-body, so a chunk holds the top half without its return or imports and the agent reasons over a fragment. Split on the syntax tree at function and class boundaries instead.

Repo maps are the complement: a ranked, compressed outline of the codebase's
symbols, so the agent sees the structure without reading every file. Tools like
aider build exactly this.

## When do embeddings help with code search?

It is not all-or-nothing. If you are new to how [embeddings](/guides/embeddings-for-beginners) work, that primer explains the vectors underneath this section.

- **They help** on large repositories (thousands of files) and on conceptual
  questions where the user does not know the exact term: "where do we handle auth
  refresh?"
- **They hurt or are unnecessary** on small and medium repos, on exact-symbol
  lookups, and on actively edited code where an index goes stale. Grep is exact,
  always fresh, and needs no maintenance.The reflex from prose RAG is to embed the whole repo first and search vectors by default, but on an exact-symbol question a vector index returns plausible near-matches and silently misses the one call site you wanted, and it is stale the moment you edit. Start with grep and structural search, then add embeddings only for large repos or fuzzy conceptual queries.

The emerging consensus is hybrid: structural and lexical search as the backbone,
embeddings layered on for scale and semantic queries, the two fused rather than
one used alone.

## How do I keep a code index fresh?

Whatever you index, do not rebuild the world on every change. Hash files and
re-embed only the ones that changed, deleting the stale vectors for that file and
upserting new ones. Or sidestep the problem entirely the way agentic search does:
recompute retrieval at query time against the live files, so there is no index to
keep in sync. The right answer depends on repo size, but the principle holds:
incremental, never full.

The last guide in this phase steps up a level, from finding context to
[coordinating whole systems of agents](/guides/multi-agent-architectures).