Which model should this task actually use?

The default policy in most teams is: use the best model available for everything, because it is the best. It is defensible, it is simple, and above a certain volume it is quietly wasteful in a way nobody measures.

Two kinds of work

Sort your AI calls into two piles.

Shape-of-input tasks. Classification, extraction, formatting, routing, summarising something short. The answer is largely determined by the input; the model's job is to recognise a pattern and produce a well-formed result. A small fast model does this correctly. A large one does it correctly too, more slowly, for more money, and the output is frequently identical.

Depth-of-reasoning tasks. Multi-step planning, debugging something with an unclear cause, weighing options with real trade-offs, anything where the first plausible answer is often wrong. Here the stronger model does not produce a nicer version of the same answer — it produces a different and better one.

The mistake is not choosing wrong. It is not sorting at all, and then paying depth-of-reasoning prices for shape-of-input work, which is usually the majority of the volume.

The test that takes an hour

Take fifty real inputs from one task type. Run them through your current model and through one tier down. Diff the outputs.

Three possible results, all useful:

• Indistinguishable. Move that task down a tier. You have just bought latency and margin for free.
• Different but both acceptable. Move it down and monitor. Most of the difference will be phrasing.
• Meaningfully worse. Keep the larger model, and now you know why you are paying for it.

Do it per task type rather than per application. A single product usually contains both kinds of work, and the average across them hides both.

Latency is a product decision

The cost conversation dominates, but latency is often what users experience. A classification step that takes two seconds instead of two hundred milliseconds turns an interface that feels instant into one that feels like it is thinking — for a result that was never in doubt.

Where a small model is sufficient, it is not a compromise. It is the better product.

Mixing within one workflow

The strongest setups use more than one model in a single flow: something small and fast to classify the request or pull out the entities, something stronger for the step that needs judgement, and often something small again to format the result.

That is one extra call and a routing decision, and it usually costs less end-to-end than sending everything to the largest model — while being faster on the majority path.

What not to optimise

Low-volume, high-stakes calls. If a task runs fifty times a day and a wrong answer is expensive, use the best model and stop thinking about it. The savings are rounding error and the downside is not.

Sorting is worth doing where volume is high, or where latency is felt. Everywhere else, the default is fine — it just should not be a default you have never examined.