What should your agent do when every retriever tool comes back empty?

An agentic RAG system reasons over a set of retriever tools and decides which to call. It is built to retrieve-then-generate. But there is a hole in that design: what happens when every tool returns nothing useful?

Without an explicit answer, one of three bad things happens. The agent returns nothing (or raises), discarding the base model's own knowledge. Or it fabricates a context and passes it along as if it were retrieved — a confident, ungrounded answer with no signal that no real documents were used. Or, with no relevance check, it injects the lowest-scoring chunks anyway and blends retrieved noise into the generation.

Why agentic failure is ambiguous

In a single-store pipeline, an empty result is unambiguous: the store had nothing. In an agentic system the agent chose the tools, so a failure could mean a bad query, the wrong tool, or a genuinely out-of-corpus question. That ambiguity is why the fallback has to live at the agent's decision layer, not bolted on after a single retriever.

Retrieve-Check-Route

The pattern adds two pieces to the graph:

• A relevance check after the tool calls — score whether the retrieved context actually answers the query, rather than assuming any returned chunk is good enough.
• A conditional edge that routes on that score. Relevant context goes to grounded generation. Empty or low-relevance results route to a defined fallback: a base-LLM parametric answer (clearly marked as not retrieved) or a controlled "I don't have that in my sources" — your choice, but an explicit one.

The win is that out-of-corpus queries stop being a silent crash or a fabricated answer. The agent makes an honest decision and the user gets either a flagged best-effort answer or a clean no-answer.