RAG Relevance Fallback

What this Claude Code skill does

Designs a conditional fallback branch in a single-store RAG pipeline so that when retrieved documents are irrelevant or absent, execution routes to an LLM-only (parametric) generation step instead of returning nothing or a hallucinated answer. Use when a LangGraph or LangChain RAG agent "returns nothing", errors, or hallucinates on questions outside the vector store; when adding a relevance check

RAG Relevance Fallback is a packaged skill for Claude Code: a SKILL.md plus references and a runnable test that drops into .claude/skills/, so your agent can run this workflow reliably instead of you re-explaining it each time. Install it via MCP, run it as a CLI, or fork it into your own.

Inside the SKILL.md

<h1>RAG Relevance-Gate Fallback</h1>
<h2>Why This Is Hard</h2>
<p>A RAG pipeline that terminates or errors when the vector store contains no
relevant documents fails silently on out-of-domain questions and degrades the
user experience exactly when the user most needs a graceful answer. Three
failure modes make this hard to get right:</p>
<ul>
<li><strong>Empty response on low-relevance queries.</strong> When no chunk exceeds the
  relevance threshold, the pipeline returns nothing rather than attempting an
  LLM-only answer from the model's training weights.</li>
<li><strong>Parametric knowledge never consulted.</strong> Questions the corpus cannot answer
  are abandoned even though the model already knows the answer from its
  parametric knowledge — the system discards a correct answer it could have
  given for free.</li>
<li><strong>Silent irrelevant-chunk pass-through.</strong> Without an explicit relevance check,
  unrelated chunks flow into the prompt and the LLM produces a confidently wrong
  answer <em>grounded in unrelated text</em> — the worst outcome, because it looks
  authoritative.</li>
</ul>
<p>The root cause is treating retrieval as an unconditional prefix to generation.
The fix is to make generation <strong>conditional</strong> on a relevance decision.</p>
<hr />
…