Skip to main content
Most AI workloads include a share of simple calls, such as classification, extraction, routing and short summaries, that do not need a top-tier model. nOps finds owners whose spend is on an expensive model when your organization already runs a model at least two tiers cheaper, and estimates the saving from moving part of the work to it. Unlike tier right-sizing, this assumes only a share of the work moves, not all of it.

Why it matters

The cheapest model in a family can cost several times less per token than the top tier, so routing even part of the simple work to it saves a lot.

How nOps finds it

nOps reads the last 30 days of AI billing per owner and model (people and workloads on AWS Bedrock, Claude Enterprise users, Anthropic API keys and OpenAI API projects). The target is the cheapest model your organization runs on that provider with at least $100 of spend in the window, and it must be at least two tiers below the current model (for Claude, Opus to Haiku). The current model must cost at least 3 times as much for the same token mix, the owner must have spent at least $25 on it, and the target must have a price for every token type used. The saving assumes 40% of the work moves, priced at the rates on your own bills, and must be at least $25 a month. Owners who already have a version upgrade or tier right-sizing recommendation for the same model are left out.

Before you start

A way to choose the model per task or per call (for example a router or a per-feature model setting), and evaluations for the tasks you move.

How to implement

  1. List the tasks the owner runs on the current model and pick the simple, high-volume ones: classification, extraction, tagging, routing, formatting, short summaries.
  2. Test those tasks on the suggested cheapest model with real inputs and compare the results.
  3. Send those tasks to the cheaper model, using a per-task model setting in your code or a routing rule in your AI gateway.
  4. Keep complex reasoning, long documents and agent planning on the current model.
  5. Widen the share you route as you gain confidence, and watch quality and spend.

Risks and considerations

  • Sending a complex task to a small model can produce wrong or incomplete answers. Route by task, not at random.
  • The 40% share is an assumption. Your real saving depends on how much of the work is simple.
  • To roll back, route the tasks back to the original model.

FAQs

Tier right-sizing moves all of an owner’s work one tier down. Cheapest Tier moves part of the work two or more tiers down, to the cheapest model you run.
Billing shows token costs, not what each prompt asks for, so nOps cannot measure how much of the work is simple. 40% is the share it assumes.