When should you stop adding chunks to your RAG and reach for a knowledge graph?

There's a specific moment in every RAG project where the eval scores stop going up.

You've upgraded the embedding model. You've added a cross-encoder reranker. You've stitched in HyDE for harder queries. You've doubled the chunk size, then halved it. You've moved to a two-million-token context window because the new model can take it.

The line goes flat. Then it goes down.

This is the multiplicative principle making itself visible. And it's the sign that you should stop tuning RAG and start designing a hybrid context layer.

Why does adding context sometimes make answers worse?

The instinct that "more context = better answers" is so widespread it has its own marketing campaigns. Anthropic's 200K-token windows, Google's 2M, OpenAI's evolving limits. The implicit promise: just put more in, get better out.

The reality is that output quality is not the sum of model intelligence and context quality. It's the product:

output_quality = reasoning_tier × context_quality

That changes the math. A great model with poor context still hits a ceiling — fast. A good model with great context outperforms a great model with mediocre context. And once your context becomes long, redundant, or topic-shifted, you're multiplying intelligence by noise.

The "Lost in the Middle" research (Liu et al, 2023) made this concrete: model accuracy on long-context tasks degrades non-linearly as relevant information moves to the middle of the prompt. Doubling context doesn't double accuracy. It often halves it.

What does a hybrid context layer actually look like?

The fix is to stop using RAG for everything and route queries through a three-layer architecture:

Layer 1 — Vector RAG. Best for point queries, factual lookup, single-hop retrieval. Pinecone, Weaviate, pgvector. This is what you already have. Don't throw it out.

Layer 2 — Knowledge graph. Best for entity relationships and multi-hop queries. "Which teams own services that depend on the deprecated API?" Neo4j, Memgraph, Amazon Neptune. Microsoft's GraphRAG (Edge et al, 2024) showed this layer alone closes most of the gap on multi-document questions.

Layer 3 — Timeline or episodic index. Best for sequence, causation, "how did we get here." Time-series stores (TimescaleDB) or graph databases with temporal edges. Event nodes with timestamps, actors, and causal predecessors.

A query router classifies incoming questions and fans out to the right layers. Compound queries hit all three; the model synthesizes structured context blocks rather than chasing semantic similarity through a haystack.

When is RAG-only good enough?

Don't over-engineer. RAG-only is the right architecture if:

• Your knowledge base is under ~10,000 documents and queries are mostly point lookups.
• The "wrong answer" cost is bounded — a chatbot for product FAQs is forgiving in a way that an enterprise decision-support agent is not.
• You can manually curate the source corpus to remove the temporal and relational gaps that would otherwise trip up retrieval.

If those things hold, building a hybrid layer is a premature abstraction. RAG plus good evals is the boring, correct answer.

What's the implementation order?

For teams starting from RAG-only and feeling the ceiling:

1. Audit which queries are failing. Classify them into the four RAG failure patterns: multi-hop, temporal, organizational context, or scale. Use the RAG failure diagnosis skill to do this rigorously.
2. Add the graph layer first. Most orgs have multi-hop relational failures before temporal ones, and the graph is easier to populate from existing data than a timeline.
3. Instrument event capture before building the timeline. Retroactive timeline ingestion is brutal. Going forward, log structured events at the source.
4. Build the query router last. You need real query patterns to know what to route.

The full architecture, with patterns for parallel retrieval, cascading retrieval, and graph-guided RAG, is captured in the Designing Hybrid Context Layers skill — a Claude Code skill that maps query types to the appropriate layer, returns a routing decision tree, and includes an implementation roadmap for teams starting from RAG-only.

The eval line stays flat until you change the shape of what you're retrieving. Then it moves.