Model Routing

Every AI-assisted workflow contains a decision that rarely gets made on purpose: which model should handle this particular task. Most organizations answer it once, by default, and then stop thinking about it. That single unexamined habit quietly accounts for a large share of both AI overspend and avoidable latency. This is not one vendor’s quirk: the Anthropic price list and the OpenAI price list are built the same way, in tiers of differing capability and differing cost per token.

What model routing means

Model routing is the practice of matching each task to the model whose capability, cost and speed fit that task, instead of sending every request to the most capable model available. It is a small idea with large consequences once an organization runs AI at any real volume: dozens or hundreds of requests a day across teams, tools and workflows.

The default habit most people fall into is treating “best model” as synonymous with “right model.” It rarely is. Asking an Opus-class model to summarize a status update or reformat a table is a bit like asking a senior architect to answer routine email: technically capable, but a poor use of judgment, time and budget.

Where “always the best model” breaks

Three factors make the default-to-premium habit expensive in practice.

Cost

It helps to see the rates side by side, because the gap is smaller than intuition suggests and larger than any single invoice reveals. Between the cheapest tier and the flagship it is exactly 5 times.¹

Model tier	Input per 1M tokens	Output per 1M tokens
Fast (Claude Haiku 4.5)	1 USD	5 USD
Mid (Claude Sonnet 5)	2 USD	10 USD
Flagship (Claude Opus 5)	5 USD	25 USD

At individual scale this difference is invisible. At the scale of a team running thousands of requests a month, it grows into a line item you can see in the budget, mostly avoidable for routine tasks.

Latency

Larger models generally take longer to respond. That difference is easy to ignore in a single chat window and impossible to ignore inside an agent that calls a model dozens of times to complete one task, or inside a customer-facing product where every extra second of delay measurably affects completion rates.

Quality that has nowhere to go

For well-defined, low-ambiguity tasks, a frontier model’s extra reasoning capacity does not translate into a better result. The task does not require it, so the additional intelligence goes unused, paid for but never delivered.

Not having a routing policy is not a neutral choice. It is a default policy, and usually an expensive one.

A three-tier framework

Rather than deciding model by model, it helps to route by task shape:

• Routine tasks: lookups, single-file edits, formatting, short status updates, first-draft generation from a template. High volume, low individual stakes, easy to check. These belong on Haiku-class models, including free tiers where available.
• Working tasks: multi-file implementation, structured analysis, drafting substantive sections of a document, reasoning across several steps. This is where most of the actual day-to-day work sits, and where a solid mid-tier general-purpose model earns its cost.
• Strategic tasks: system design, architecture and irreversible decisions, synthesis across long or ambiguous context, anything with financial, legal or reputational exposure. This is a small share of total volume and the one place where paying for frontier reasoning is worth it.

The question worth asking before every task is not which model feels safest to default to, but what it costs to be wrong here, and whether the task calls for deep reasoning or simply competent execution.

When routing does not pay off

Routing carries its own cost: somebody maintains the rules, watches whether they still match reality, and reacts when a vendor retires a model variant. That cost only repays itself once routine tasks make up a clear majority of volume and recur in recognisable shapes.

At a few dozen requests a day, scattered across different kinds of work, the saving on model choice is smaller than the time spent maintaining the rules. The honest decision there is a single mid-tier model as the default, with deliberate reach for the frontier model where the stakes are high.

The threshold to check before adopting routing is simple: how many requests from the past three months map onto repeatable task shapes. If the answer is “fewer than half”, routing will cost more than it returns.

Task class	Examples	Model class	Cost of getting it wrong
Routine	Formatting, short status notes, first draft from a template	Fast and cheap, free tiers included	Low, verified immediately
Working	Structured analysis, editing material passages, multi-step reasoning	A solid mid-tier model	Medium, surfaces at review
Strategic	Architectural and irreversible decisions, synthesis of ambiguous context	Frontier model	High: financial, legal, reputational

From manual routing to automatic cascades

Once the three tiers are defined, the routing decision does not need to depend on someone remembering to make it each time. A cascade architecture attempts the task on the fast, inexpensive model first, checks the result against a simple rule or a lightweight verification step and escalates only the cases that fail that check to a stronger model. At the rates in the table above the arithmetic is simple: every escalated case costs 5 times what a case handled in place costs, so a cascade only earns its keep once the cheap model closes a clear majority of traffic. This is the pattern worth building into production AI systems: routing stops being a habit an individual has to maintain and becomes a property of the system itself, applied consistently whether the request comes from an employee, an internal tool or a customer-facing agent.

Where routing needs a governance layer

Cost and latency are not the only variables in a regulated environment. Some tasks should run on the same validated model configuration every time, independent of price, because consistency and explainability matter more than savings: decisions that touch personal data within the meaning of GDPR, financial disclosures or safety, or that must be defensible to a regulator or an auditor after the fact. For those tasks, routing is not an efficiency question but a governance one. It belongs in written policy, reviewed on a schedule, with a human in the loop wherever a decision is difficult to reverse.

Building a routing matrix for your organization

A workable starting point rarely needs more than four steps: inventory the AI tasks teams actually run over a couple of weeks; classify them by stakes and reversibility rather than novelty; assign a default model tier to each category and write it down; and build explicit escalation rules for the exceptions instead of leaving the choice to whoever happens to be at the keyboard. Because model capability and pricing shift quickly, the matrix is worth revisiting on a quarterly basis. A routing policy set once and never revisited tends to become wrong within months.

RS Management designs and implements these routing systems, from the initial task inventory to the production cascade, as part of our broader AI advisory and delivery work for executive teams and regulated organizations. If your organization is spending more on AI than the work actually requires, or would like a second, more critical opinion on how model choices are made today, we would welcome a conversation.

Footnotes

1. Anthropic, model pricing per million tokens: https://platform.claude.com/docs/en/about-claude/models/overview. ↩