Skip to main content
Some models charge more per token once a request goes past a long-context threshold, often 200K input tokens. Long chat threads, pasted documents and agents that keep their whole history push requests into that higher rate. nOps measures how much of each owner’s spend is billed at the long-context rate and estimates the saving from keeping requests under the threshold.

Why it matters

The surcharge applies to the whole request, not only the tokens over the limit, and nothing in the tool warns users when they cross it.

How nOps finds it

nOps uses three sources. On AWS Bedrock it reads the last 32 days of Claude Sonnet long-context usage per model or inference profile, and the saving is the full premium over standard context, taking long-context input as 2 times and output as 1.5 times the standard price; at least $1 of premium is needed. On the OpenAI API it reads the last 30 days of standard-tier long-context lines per project, takes the same 2 times and 1.5 times premiums, and assumes half of that premium can be avoided; the saving must be at least $25 a month. For Claude Enterprise it reads each user’s spend by context window from Anthropic’s analytics over about 32 days; a user is flagged when more than 20% of their spend is in the 200K to 1M tier, and the saving assumes a 25% surcharge of which 55% can be avoided, with a minimum of $5 a month.

Before you start

Know how the team or application builds its prompts: long conversations, large attachments, or agents that resend their whole history.

How to implement

  1. Look at the owner and model in the recommendation and the share of spend billed at the long-context rate.
  2. For people using chat or coding tools, start a new conversation or session for a new task instead of continuing a long thread, and use the tool’s compact or summarize command when a thread gets long.
  3. For applications, summarize older turns, retrieve only the relevant passages of large documents instead of sending them whole, and cap how much history an agent resends.
  4. Check the long-context share again after two to four weeks.

Risks and considerations

  • Some tasks genuinely need the whole context, for example reviewing a large codebase or contract. Trimming those can lower answer quality.
  • The OpenAI and Claude Enterprise estimates assume only part of the long-context spend can be avoided, so the real saving depends on how your team works.
  • Nothing changes in your account, so there is nothing to roll back.

FAQs

Requests the provider bills at its long-context rate: Claude requests in the 200K to 1M token tier, and OpenAI requests billed as long context.
Not currently. nOps found that current Claude models on the Anthropic API bill long requests at the standard rate, so there is no premium to remove.