Home/Guides/Your model got smarter and your answers…
Guide · 5 min read

Your model got smarter and your answers got worse. Why?

The multiplicative principle, the degradation curve, and why upgrading models without auditing context backfires.

Quick answer

Output quality is reasoning tier × context quality, not summed. Upgrading the model amplifies whatever's in the context — including the gaps. A great model on poor context fails more visibly than a poor model on poor context, because the answers sound right.

The team upgraded from Sonnet to Opus on a Friday afternoon.

By Monday morning, the on-call engineer was filing a regression. The agent's outputs were worse. Not obviously broken — that would have been easy to revert. They were more confident, more plausible, more articulate. And, on a non-trivial fraction of complex queries, more wrong.

The engineer wrote: "It feels like the smarter model is hallucinating more, not less. Is that possible?"

It is. And it's predictable.

What is the multiplicative model?

Most teams reason about model upgrades additively. "Better model = better answers." Some sub-conscious linear function: 1.2× intelligence yields 1.2× quality.

The actual relationship is multiplicative:

output_quality = reasoning_tier × context_quality

This has three consequences that catch teams off guard:

Upgrading the model when context is the bottleneck doesn't fix the bottleneck. It makes the failures harder to spot.

Why does upgrading sometimes make things worse?

The team's Monday-morning regression had a specific shape. The previous (weaker) model would generate hedged answers when the retrieved context was thin. "I see references to X but the documents don't make the chain explicit. The most likely answer is Y, but verify."

The upgraded model — same prompt, same retrieval, same context — produced answers that read like definitive statements. "The chain is X → Y → Z because A and B." Cleaner. More authoritative. And, where the context didn't actually support the chain, confidently fabricating it.

Anthropic's research on model behavior under context stress documents this pattern: more capable models are better at making weak evidence look like strong evidence. That's a feature for many tasks. It's a failure mode for tasks where the evidence really is weak.

What does the degradation curve look like?

If you plot output quality against context complexity for a fixed reasoning tier, you don't get a flat line that drops off a cliff. You get a curve with three phases:

Graceful phase. Context complexity is well within the model's reasoning capability. Output quality is high and stable. Adding context helps.

Degraded phase. Complexity is approaching the model's ceiling. Output quality drops noticeably, but the failures are visible — the answers feel hedged, qualified, "I'm not sure."

Hallucination-prone phase. Complexity is past the ceiling. Output quality drops sharply. The failures stop being visible — the answers come back confident, fluent, and wrong. This is the dangerous phase, and it's where you land when you upgrade the model without auditing the context.

How do you diagnose intelligence-context fit?

The diagnostic has two scores:

Score the context complexity. How many hops does the answer require? How much synthesis across documents? How long is the relevant time horizon? Is the underlying corpus consistent or contradictory?

Score the model tier. What's the model's reasoning capability on independent benchmarks? Long-context retention? Multi-hop accuracy? Calibration on uncertainty?

Plot both against the degradation curve. If you're in the degraded phase, upgrading the model can help — provided you also tighten the context. If you're already in the hallucination-prone phase, upgrading without changing the context will make the visible quality worse, because confident-wrong is harder to catch than hedged-wrong.

This is the diagnostic encoded in the Auditing Intelligence-Context Fit skill — a Claude Code skill that scores both dimensions, plots the result against the degradation curve, and prescribes whether the right move is a model upgrade, context narrowing, or context restructuring.

The team that upgraded on Friday eventually rolled back, audited their retrieval pipeline, fixed two specific gaps in the context, and re-upgraded. The combined change fixed the regressions. The model upgrade alone wouldn't have.

Frequently asked questions

How do I tell if I'm in the degraded phase or the hallucination-prone phase?
Watch the language. Degraded-phase outputs hedge: 'most likely,' 'unclear from the documents,' 'verify before acting.' Hallucination-prone outputs sound definitive even when the underlying context is thin. If you can't tell from a sample of outputs, run an eval where you deliberately starve the context — confident answers under starvation are the giveaway.
Should I always pick the most capable model available?
No. The right model is the one whose reasoning tier matches your context complexity. A pricey model on simple lookups wastes money. A cheap model on complex synthesis fails. Match the tier to the task, then improve context separately.
What if my context complexity isn't easy to score?
Use proxy metrics: average number of documents retrieved per query, average hops to answer, longest causal chain in your eval set. Each of these correlates with where you land on the degradation curve. Perfect scoring isn't required — directional ranking is.

Related skill

AI Ops

Auditing Intelligence-Context Fit

Audits the fit between a model's reasoning capability and the complexity of the context it receives. Use when an AI sys…

Turn this guide into a skill your agent can run

Stop re-explaining the same workflow. Loreto packages it as a Claude Code skill from any source.