Agentic retrieval
The base model is frozen at training time. The most important capability layered on top of it is retrieval: reaching outside the model’s parametric memory to pull in current, specific, or private information.
The interesting shift is that retrieval is no longer a rigid pipeline. It’s a loop, driven by the model’s own reasoning.
The model decides what it needs to know, uses tools to go get it, reads what came back, and retrieves differently if the first try wasn’t enough.
This is broadly called agentic retrieval (or “agentic RAG,” or “agentic search,” depending on who you’re reading). Anthropic frames the same idea as context engineering: designing the information environment and the tools so the model can reach what it needs at the right moment.
From classic RAG to agentic retrieval
Classic RAG (retrieval-augmented generation) is a straight line: embed the question as a vector, look up the nearest chunks in a vector store, hand them to the model with the original question, generate an answer. One retrieval step, no feedback. If the first lookup missed, the answer suffers and the system doesn’t try again.
Agentic retrieval rearranges the pieces:
- The model decides whether retrieval is needed at all.
- It decides what to retrieve and how (file read, grep, web search, MCP tool call, vector lookup), whichever fits the question.
- It reads the result and decides whether it has enough.
- If not, it retrieves again, differently: a sharper query, a different tool, a different file.
That feedback loop is what makes modern coding agents work. Claude Code, Cursor, and Codex all operate this way. The retrieval is driven by the model itself rather than by a separate pipeline built in front of it.
Just-in-time vs. just-in-case
A useful framing from Anthropic:
- Just-in-case retrieval: build a big index up front (usually a vector store), retrieve the top-k chunks on every query, hope the relevant one is in there. Precomputed. Always running.
- Just-in-time retrieval: keep lightweight references (file paths, URLs, query strings, IDs), and load data on demand via tools when the model decides it’s needed. Nothing precomputed; everything fetched when it matters.
Just-in-time wins for most real work. The model sees exactly what it asked for, fresh, with no chunking loss or index staleness. Just-in-case still earns its place when the corpus is too large to navigate by hand and the queries are too fuzzy for exact matching; see the retrieval toolkit for when to reach for each.
Why the loop matters
A single retrieval step bets everything on one guess. An agentic loop absorbs partial failures:
- First lookup returns nothing useful → the model rephrases and tries again.
- First file read surfaces an obsolete function → the model greps for the current name.
- First web search turns up marketing pages → the model narrows with site-specific terms.
Each step teaches the next. The model is reasoning about its own retrieval, which is how it recovers from imperfect matches instead of confidently answering from bad context.
Implications for what you build
If you’re building anything on top of an LLM (a product, an internal tool, a personal workflow), the shift from pipeline-RAG to agentic retrieval changes the design:
- Don’t build a big static index when a tool call would do. Expose your data as tools (read-a-record, search-by-field, list-by-filter) and let the model query them the way a competent human would. This is the MCP pattern; see Modes and abilities.
- Keep references cheap and metadata rich. The model navigates by file paths, URLs, and IDs. Well-named files with short descriptions are easier to reach for than opaque chunks in a vector store.
- Let the model fail and retry. A harness that shows the model a failed search and lets it try again outperforms one that returns a single answer and moves on.
Resources
- Anthropic: Effective context engineering for AI agents
- LlamaIndex: Agentic retrieval guide: beyond naive RAG
- LlamaIndex: Did filesystem tools kill vector search?, a 2026 benchmark comparing file-system agents against vector RAG
- arXiv: Agentic Retrieval-Augmented Generation: A Survey