Chapter 05
Advanced RAG
The basic pipeline retrieves once and answers once. The 2026 frontier makes retrieval smarter, more contextual, and more autonomous — to the point where the field increasingly calls it context engineering rather than just “RAG”.
Contextual retrieval
A chunk ripped from a document loses its surroundings — “the limit is 45 days” is useless if you don't know which limit. Contextual retrieval prepends a short, chunk-specific summary of its document context before embedding, so each vector carries document-wide intent.
Before
"Pro-rated refunds within 45 days of renewal."
— ambiguous in isolation.
After
"[Billing policy, Enterprise plans] Pro-rated refunds
within 45 days of renewal." — self-describing, retrieves far more reliably.
Anthropic's Contextual Retrieval pairs this with contextual BM25 and reranking, cutting retrieval failures substantially. The context line is cheap to generate (and prompt-cacheable across a document's chunks).
Agentic RAG
Instead of one fixed retrieve-then-generate step, the model decides when and what to search. It can search, read, judge whether it has enough, reformulate, search again, and only then answer — a reason → retrieve → reflect loop. Retrieval becomes a tool the model calls.
With Claude, this is a tool-use loop: define a search_docs tool and let the model
drive it until it's confident.
import anthropic
client = anthropic.Anthropic()
tools = [{
"name": "search_docs",
"description": ("Search the knowledge base. Call this whenever you need "
"facts you don't have. You may search multiple times to refine."),
"input_schema": {
"type": "object",
"properties": {"query": {"type": "string"}},
"required": ["query"],
},
}]
messages = [{"role": "user", "content": question}]
while True:
resp = client.messages.create(
model="claude-opus-4-8", max_tokens=1024,
tools=tools, messages=messages,
)
if resp.stop_reason != "tool_use": # model is ready to answer
break
messages.append({"role": "assistant", "content": resp.content})
results = []
for block in resp.content:
if block.type == "tool_use": # model chose to search
hits = retrieve(block.input["query"]) # your hybrid+rerank retriever
results.append({"type": "tool_result", "tool_use_id": block.id,
"content": "\n\n".join(hits)})
messages.append({"role": "user", "content": results})
answer = next(b.text for b in resp.content if b.type == "text")This handles multi-hop questions (“compare our refund policy to our competitor's”) that one-shot retrieval can't — at the cost of more model calls and latency.
GraphRAG
Vector search struggles with questions that require connecting facts across many documents (“what themes recur across all incident reports?”). GraphRAG extracts entities and relationships into a knowledge graph, then retrieves by traversing it — following connections rather than matching text.
Great for global, “connect-the-dots” questions and explainable reasoning paths. The classic knock is indexing cost — Microsoft's LazyGraphRAG (2025) cut that to a tiny fraction by deferring graph summarisation until query time, making it practical on large corpora. Many production systems now blend graph + vector retrieval.
Adaptive RAG
Not every question deserves the same pipeline. Adaptive RAG puts a lightweight classifier up front that routes each query by complexity:
- Simple / known → answer directly, no retrieval (fast, cheap).
- Factual lookup → single-shot hybrid retrieval + rerank.
- Complex / multi-hop → the agentic loop, or GraphRAG.
You spend compute where it pays off and stay fast on the easy majority. This routing mindset — matching retrieval strategy to the query — is the emerging default for serious 2026 systems.