# RAG or a long prompt: which to use

Paste the documents into the prompt when the relevant material is small, stable, and known up front. Reach for retrieval when the source is too large to fit, changes often, or you cannot tell in advance which part holds the answer. This guide gives you the line to draw, and the signals that tell you which side you are on.

## The short version

There is a real choice here because the context window is large. A current Claude model carries a 1 million token window on Opus 4.8 and Sonnet 4.6, which is enough to hold a substantial set of documents in a single prompt. That removes the need for retrieval in many cases that used to require it. It does not remove the need in all of them.

The decision comes down to three questions:

- **Does it fit?** If the material fits in the window with room to work, you can paste it.
- **Does it change?** If the source updates between questions, a pasted copy goes stale; an index can be refreshed.
- **Do you know which part you need?** If you can name the file, paste it. If the answer is somewhere in thousands of documents and you do not know which, you need search.

If all three point to "paste it," paste it. If any one points the other way, lean toward retrieval.

## What "a long prompt" means

Pasting documents in is the simplest possible setup. You put the text in the prompt, ask your question, and the model answers from what is in front of it. No index, no embeddings, no extra service to run.

With the Claude Code CLI you do this constantly without thinking of it as a strategy. You reference a file and Claude reads its full contents into context:

```bash
claude "explain what src/auth/session.ts does and where it is called from"
```

You can point Claude at several files at once, and it reads each one in full:

```bash
claude "compare the retry logic in src/api/client.ts and src/api/queue.ts"
```

You can also pipe text in, which is the same idea from a different direction:

```bash
tail -200 app.log | claude -p "summarise the errors in this log"
```

In every case the full text lands in the window. There is no retrieval step. The model sees everything you handed it, in order, and works from that.

## What does retrieval add?

Retrieval sits a search step in front of the model. Instead of sending the whole corpus, you find the relevant passages first and send only those. [RAG fundamentals](/guides/rag-fundamentals) walks the full pipeline; the short shape is: chunk the documents, embed them, store the vectors, then at question time fetch the closest chunks and ground the answer in them.

The cost of retrieval is real: a store to run, an embedding model to pick, chunk boundaries to tune, and a retrieval stage to evaluate on its own. You take that cost on when the long-prompt approach stops working, not before.

## The three signals, in detail

### Does the material fit in the window?

A 1 million token window is large, but it is not a filing cabinet. Two limits matter.

First, the hard limit. If your source is bigger than the window, you cannot paste it, and the question is settled: you retrieve. A single design doc fits. A 2,000-page document set does not.

Second, the soft limit. Even when material fits, filling the window with everything is wasteful and dilutes the signal. Models attend best to the start and end of a long context and least to the middle, so a fact buried in the middle of a giant paste is the fact most likely to be missed. Sending fewer, more relevant passages produces a sharper answer than sending everything and hoping. [Context engineering](/guides/context-engineering) covers how to fill the window with exactly what the model needs.Do not treat "it fits in the window" as the only test. A 1M-token paste that technically fits still hides the answer in the low-attention middle, so quality drops silently with no error to warn you. Judge by how much of the paste the question needs, not by whether the bytes squeeze in.

A rough rule: if the relevant material is a handful of documents you can name, paste them. If it is "somewhere in our whole knowledge base," you have outgrown pasting.

### Does the source change between questions?

A pasted document is a snapshot. It is correct at the moment you paste it and frozen after that. Ask the same question tomorrow against a copy you pasted today and you may get an answer from yesterday's facts.

An index can be refreshed. When a source document changes, you re-index that document and the next question sees the new version. This is the deciding factor for anything live: support tickets, product inventory, pricing, a wiki under active editing. The content moves, so a static paste is wrong by the time you read the answer.

If the source is stable, a research paper, a signed contract, a finished spec, freshness does not bite, and pasting stays fine.

### Do you know which part you need?

This is the signal people miss. Fit and freshness are easy to check; this one is about the shape of the question.

If you can point at the file, you do not need search. "What does the checkout flow do" against `docs/checkout-flow.md` is a paste. You named the document; retrieval would only re-find what you already have.

If the answer could be in any of thousands of documents and you cannot say which, search is the point. "Which of our 4,000 support articles covers expired refund tokens" is not a paste, because the work is finding the right article, not reading a known one. Retrieval does that finding step. Sending all 4,000 articles, even if they fit, would bury the answer and cost far more.

## A comparison you can scan

| Situation | Paste the docs | Use retrieval |
| --- | --- | --- |
| Material fits in the window | Yes | Optional |
| Material is larger than the window | No | Yes |
| Source is stable between questions | Yes | Yes |
| Source changes often | No (goes stale) | Yes (re-index) |
| You can name the relevant file | Yes | Overkill |
| Answer is hidden in a large set | No (buries it) | Yes |
| You need source citations | Workable | Stronger |
| Setup cost | None | Store, embeddings, tuning |

The pattern: pasting wins on simplicity and loses on scale and freshness. Retrieval wins on scale and freshness and costs you a system to run.

## The middle ground most people land on

The two are not rivals. The common setup uses retrieval to choose a small set of documents, then pastes those documents in full. You get search to find the right material and a clean, complete paste for the model to reason over. You avoid sending the whole corpus, and you avoid the chunk-boundary problems that come from splitting documents too finely.

With the Claude Code CLI, the retrieval half of that pattern is usually a tool the model can call rather than a vector store you build. The [Model Context Protocol](/guides/mcp-fundamentals) lets Claude reach a search service, a database, or a docs system and pull back the relevant pieces on demand. You register the server once and Claude decides when to search:

```json
{
  "mcpServers": {
    "company-docs": {
      "command": "npx",
      "args": ["-y", "@example/docs-search-mcp"]
    }
  }
}
```

Now a question like "find the article on expired refund tokens and explain the fix" turns into a search step (the MCP tool finds the article) followed by a read step (Claude pulls in the matched document). That is retrieval feeding a long prompt, with no vector store of your own to maintain. [Install your first MCP server](/guides/install-your-first-mcp-server) walks the setup end to end.

## How does prompt caching change the maths?

If you paste the same large block on many calls, the cost argument against pasting softens. Prompt caching stores a stable prefix so repeat requests reuse it.

The numbers are concrete. A cache write costs 1.25x the base input price for the default 5 minute window, or 2x for the 1 hour option. A cache read costs 0.1x the base input price. So a large document set you paste once and reuse many times within the cache window is read back at a tenth of the input cost on every later call. The cached content has to be a stable prefix and meet a minimum length (1,024 tokens on Opus 4.8 and Sonnet 4.6), which a large paste comfortably clears.The cache only helps if the big paste comes first and stays byte-for-byte identical. Put the user's question or a timestamp ahead of it and you break the prefix, so every call pays full price and the cheap-to-paste argument quietly vanishes.

This makes "paste the same big context repeatedly" cheaper than it looks, and it pushes the line a little toward pasting for stable sources you query often. [Prompt caching](/guides/prompt-caching) covers the mechanics and the pitfalls.

## Worked examples

**Reviewing one pull request.** The diff and the touched files fit easily, they are exactly the material you want, and they will not change mid-review. Paste it. There is nothing for retrieval to add.

**Answering questions over a 50,000-document support archive.** Too large for the window, changing daily as agents add articles, and you never know which article holds the answer. This is retrieval, full stop.

**A 200-page contract you are negotiating this week.** It fits, it is stable for the week, and you can name it. Paste it, and let caching cover the repeated questions you ask against it.

**A product wiki your team edits all day.** It might fit today, but it changes hourly and you query specific topics. Retrieve, so each answer sees the current page.

## Mistakes that push people to the wrong side

A few patterns lead to the wrong call.

**Reaching for retrieval too early.** Building a vector store for a handful of stable documents is effort spent on a problem you do not have. If the material fits and does not change, paste it and move on. You can always add retrieval later when the corpus grows.

**Pasting a corpus that has outgrown the window.** The opposite failure. The window is large, so people keep pasting more and more until answers quietly degrade. The signal is not whether it technically fits; it is whether the relevant material is buried among a lot of text the question does not need. When the paste is mostly noise, retrieval sharpens it.

**Trusting a pasted copy of a moving source.** Someone exports a wiki to a file, pastes that file for weeks, and never notices it drifted. If the source changes, do not freeze a copy. Point retrieval at the live source so every answer sees the current state.

**Splitting documents that should stay whole.** When you do build retrieval, over-aggressive [chunking](/guides/chunk-your-documents) can sever a fact from the sentence that qualifies it. The hybrid pattern, retrieve to pick whole documents and paste them in full, sidesteps this by keeping each retrieved document intact.

**Ignoring citations until the end.** If you need the answer to point back to a source, decide that up front. Retrieval gives each passage an identifier you can carry into the answer. A flat paste of merged text makes "where did this come from" harder to answer.

## How to test the decision before you build

You do not have to guess. You can check the paste-it path in minutes with the Claude Code CLI before committing to any retrieval infrastructure.

Paste the candidate material and ask the questions you expect real users to ask:

```bash
claude "using only docs/checkout-flow.md and docs/refunds.md, answer: how long is a refund token valid, and what happens when it expires?"
```

If Claude answers correctly and cites the right place, the long-prompt path works for that question. Run your hardest questions this way. Watch for two failure shapes:

- **The answer is wrong or vague even though the fact is in the documents.** This is the lost-in-the-middle problem. The material is too much for the question, and retrieval would help by narrowing it.
- **You cannot decide which documents to paste.** If choosing the inputs is itself hard, that is the search problem in disguise, and it is the clearest sign you need retrieval.

When the paste path passes your questions, you are done; ship the simple thing. When it fails on the shapes above, you have learned exactly why retrieval is needed, which makes the retrieval system easier to scope.

## A prompt to make the call

When you are unsure, hand Claude the constraints and let it reason through the same three signals:

```text
I need to answer questions over a body of documents. Help me decide whether to paste the documents into the prompt or use retrieval.

Here are the facts:
- Total size of the source: 
- How often it changes: 
- For a typical question, can I name the file that holds the answer, or is finding it part of the work?
- Roughly how many questions I expect to ask against the same content

Walk through three checks in order: does it fit in the window, does it change between questions, and do I know which part I need. Then recommend pasting, retrieval, or a hybrid (retrieve to pick documents, paste them in full), and say plainly why.
```

## The rule to remember

Pasting is the default. It is simpler, has no moving parts, and a 1 million token window absorbs more than people expect. Move to retrieval only when one of the three signals forces it: the material does not fit, the source changes between questions, or finding the right document is itself the task. When you do move, the hybrid (retrieve to select, paste to read) gives you most of the benefit of each side. For the retrieval mechanics underneath all of this, read [RAG fundamentals](/guides/rag-fundamentals) next.