RAG Agents: Routing and Multi-Step Retrieval

Routing Questions

Not every question needs the vector store. Summarization tasks, math and small talk can skip retrieval entirely, saving latency and cost.

def route(question):
    if question.startswith("summarize"):
        return "chat"
    if "latest" in question:
        return "database"   # fresh data source
    return "vector"

Agentic Retrieval

An agent picks tools as it works: it may search the vector store, query a database, read a file, then combine results. Iterative retrieval lets the model refine its search when the first pass is weak.

Multi-Step Pattern

• Step 1 - Rewrite the user question into a searchable one.
• Step 2 - Retrieve a broad candidate set.
• Step 3 - Run follow-up searches for missing facts.
• Step 4 - Generate with the collected evidence.

Key Points

• Route trivial questions away from retrieval.
• Agents choose sources dynamically.
• Iterative retrieval fixes weak first searches.
• Use the smallest tool set that answers well.