RAG for Coding Agents

Apply textbook RAGRetrieval-Augmented Generation, where you fetch relevant snippets and feed them to the model so its answer is grounded in real context. to a codebase and it quietly underperforms. Code is not prose, and the retrieval that works for documents works against you here. This guide is about why, and how real coding agents find the context they need. If the pipeline is new to you, read RAG fundamentals first.

Why is code retrieval different from text RAG?

Three things make code retrieval its own problem:

• Structure is meaning. A function, class, or method is a unit. Fixed-size chunking splits a function down the middle or merges unrelated code, and either one destroys the chunk's usefulness.
• Exact symbols beat paraphrase. Code questions are often "where is refreshToken called?" The exact name matters more than semantic similarityA match based on meaning rather than spelling, so related but differently-worded text still counts as close., which is the opposite of prose, where paraphrase is the point.
• The answer is spread out. Meaning lives across call graphs, imports, and type definitions in different files. One chunk rarely holds the whole story.

How do coding agents retrieve code?

Two design philosophies have emerged, and the contrast is instructive.

Search the live code, no index. Claude Code deliberately does not build a vector indexA precomputed database of numeric fingerprints of your code, searched by closeness in meaning instead of by exact text.. It searches the way a developer does: ripgrep and glob to find things, then read the files. Its team has been public that an earlier indexed approach was beaten by agentic search, for four reasons. Precision: exact matches with no fuzzy false positives. Simplicity: no index to build or maintain. Freshness: a prebuilt index drifts the moment you start editing. Privacy: nothing leaves the machine to be embedded.

Index with structure. Cursor takes the other path: it hashes files to detect changes, chunks with syntax-aware (tree-sitterA fast parser library that turns source code into a structured syntax tree so tools can split on real code boundaries.) splitters, stores vectors, and combines semantic search with grep. Cursor reports this beats grep alone on codebase questions, with the gap widening on very large repositories.

Both are real positions. The honest summary is that exact, structural search is the backbone, and embeddings help most at scale.

How should I chunk code for retrieval?

If you do index code, do not chunk by line count. Use the syntax tree: split at function, class, and method boundaries so each chunk is a coherent unit. Research on ASTSplitting code along its Abstract Syntax Tree, the structured parse of the source, rather than by raw line count.-aware chunking (recursively splitting large nodes and merging small siblings under a size budget) shows real gains on code retrieval and on end-to-end coding benchmarks over naive splitting.Common gotchaReaching for a fixed token or line count is the prose-RAG habit that breaks here: it slices a function mid-body, so a chunk holds the top half without its return or imports and the agent reasons over a fragment.Split on the syntax tree at function and class boundaries instead.

Repo maps are the complement: a ranked, compressed outline of the codebase's symbols, so the agent sees the structure without reading every file. Tools like aider build exactly this.

When do embeddings help with code search?

It is not all-or-nothing. If you are new to how embeddings work, that primer explains the vectors underneath this section.

• They help on large repositories (thousands of files) and on conceptual questions where the user does not know the exact term: "where do we handle auth refresh?"
• They hurt or are unnecessary on small and medium repos, on exact-symbol lookups, and on actively edited code where an index goes stale. Grep is exact, always fresh, and needs no maintenance.Common gotchaThe reflex from prose RAG is to embed the whole repo first and search vectors by default, but on an exact-symbol question a vector index returns plausible near-matches and silently misses the one call site you wanted, and it is stale the moment you edit.Start with grep and structural search, then add embeddings only for large repos or fuzzy conceptual queries.

The emerging consensus is hybridRunning exact keyword search and meaning-based search together, then merging their results instead of trusting either alone.: structural and lexical search as the backbone, embeddings layered on for scale and semantic queries, the two fused rather than one used alone.

How do I keep a code index fresh?

Whatever you index, do not rebuild the world on every change. Hash files and re-embed only the ones that changed, deleting the stale vectors for that file and upserting new ones. Or sidestep the problem entirely the way agentic search does: recompute retrieval at query time against the live files, so there is no index to keep in sync. The right answer depends on repo size, but the principle holds: incremental, never full.

The last guide in this phase steps up a level, from finding context to coordinating whole systems of agents.