Skip to main content
This page explains each figure on the AI ROI page. Every number is labelled with how it was measured, so you can tell a billed dollar from an estimate.

AI coding spend

What your organization spends on AI coding tools, per person per day. Spend on AI APIs (such as Amazon Bedrock) is not part of AI ROI: it isn’t tied to a developer. Service accounts, such as Cursor’s code review bot or background agents, count toward total spend but are never tied to a person or a PR.

Settled spend

Claude adjusts the last three days of Claude Code spend after the fact. Cursor’s daily figures are final once the day is over. So a day’s spend is settled once every tool has finished adjusting it.
  • Totals show every day that has arrived.
  • Cost per PR counts settled spend only. A PR merged after the last settled day shows Arriving and is left out of averages until its spend settles.
  • “Vs before” spend changes compare settled days with the same days of the previous period, so the newest, still-arriving days never read as a drop.
Cards that depend on this say “Through Nov 9” (or similar) to show the last settled day.

Pull requests

Merged pull requests from your connected code hosts: GitHub, GitLab and Bitbucket Cloud. nOps reads PR metadata, the commits in each PR (including those squashed away on merge) and the list of files changed with line counts. It never stores file contents.
  • Bots are left out, except AI coding agents that open PRs themselves (GitHub Copilot coding agent, Devin, Codex, Claude, Cursor).
  • A change merged twice counts once. A fix merged into dev and then cherry-picked or hotfixed into prod is one piece of work. nOps keeps the first merge and drops the copy, recognized by a cherry-pick/ or backport/ branch, a title that cites the original PR, or the same author merging an identical diff within three days.
  • Release PRs (for example dev → prod) are used to tell when work reached production, but are not counted as work themselves.

Reached production

A merged PR reached production when a later release PR carried the branch it merged into, or when it merged straight into a release branch. In a repository with no release PRs, merging counts as production.

AI cost per PR

How much AI spend went into a merged PR. nOps uses the best evidence it has: Spend after a person’s last merge in the range is work in progress and isn’t tied to any PR. Spend tied to PRs is the share of AI spend credited to a merged PR. The rest is exploration, research, unfinished work, or spend of someone not matched to a code host.

Cost basis

Work types

Each PR, and each piece of spend, gets one work type: nOps uses the strongest evidence first:
  1. The PR title or branch. A title like fix: … or a branch like feat/….
  2. The linked ticket. A branch that names a Linear or Jira ticket (like ENG-123) takes the ticket’s type and labels. Bug → Fix, Feature → Build, and so on.
  3. The commit types in the PR (for example fix(api): …). The agent sends only the type, never the message.
  4. The AI ROI Agent’s prompt labels. A small model on the developer’s machine labels each prompt. nOps receives only the label. See How prompts are labelled.
  5. What changed, for example only documentation files.
AI spend that never reached a merged PR can only get a work type from the agent’s prompt labels. That is why Where the spend goes shows how much of the split each source decided.

Output points

A measure of how much a PR delivered, so a one-line config change doesn’t count the same as a new service. nOps computes it from the PR’s file list and line counts only, never its contents:
  • Complexity from 1 to 5, from the kinds of files changed (code, tests, config, infrastructure), how many there are, and how many lines changed. Higher complexity weighs much more (level 1 = 1, level 5 = 24).
  • Size: lines changed, with diminishing returns so a huge PR doesn’t dominate.
  • A code change with no test files weighs 20% less.
  • A reverted PR scores 0.
When the file list isn’t available, output falls back to PR size and is labelled as such.

Rework

An estimate of how much merged work needed fixing soon after. A PR is reworked when, within 14 days of merging:
  • it was reverted, or
  • a later merged Fix PR changed or removed lines in a code file this PR was the last to write.
To avoid false matches, nOps ignores files that many PRs touch (8 or more in the range), config files, and fixes spread across more than 8 code files. PRs merged in the last 14 days are pending: they are never counted as clean until the 14 days pass. The rework rate is split between PRs with evidence of AI use and the rest, so you can see whether AI-assisted work needs fixing more often.

AI evidence

How sure nOps is that AI was used on a PR: AI-assisted PRs are verified or inferred.

Written by AI

The share of merged lines your AI tools say they wrote. It adds up, per person, the lines that Claude Code and Cursor report as AI-written and accepted, capped at that person’s own merged lines, then divides by every merged line. It is an estimate, and it runs high for heavy users, because tools count lines that were later rewritten or never merged. GitHub Copilot doesn’t report lines per person, so Copilot users add nothing to it. The card’s ⓘ shows each tool’s part. The AI ROI Agent measures AI-written lines precisely, commit by commit, on the developer’s machine. That measurement feeds Verified evidence, not this share, so the two never double count.

Agent sessions and coaching signals

With the AI ROI Agent installed, nOps also sees:
  • Sessions: one per AI coding session, with prompts, tool calls, edits and duration. A session with no activity for an hour counts as ended.
  • Repeated commands: the same shell command run 8 or more times in a row.
  • Error streaks: 5 or more failing tool calls in a row.
These feed the Stuck sessions flag on the Engineering Overview. They come from counts and command names only, never prompts.

Data freshness