Understanding differences between AI Impact (Comparison), Tool Utilization, and Agent Traces
Last updated: October 28, 2026
AI Impact (Comparison), Tool utilization, and Agent Traces answer three different questions, so their numbers are not meant to match. This article explains what each one measures and how to read them together.
Why it matters
The most common confusion is a high adoption number next to a much lower AI code ratio. Both can be right at once. Knowing which question each feature answers tells you which number to quote and which gap to look into.
Feature | Where to find it | The question it answers |
|---|---|---|
AI Impact (Comparison) | AI Effectiveness → AI insights → Comparison | How much of our merged code did AI agents write, and how do speed, review, and quality differ between high-AI and low-AI work? |
Tool utilization (page title: AI Tool Adoption) | AI Effectiveness → AI insights → Tool utilization | Are developers using the AI tools we pay for? |
Agent Traces | AI Effectiveness → Traces | What happened inside each AI coding session, and which lines did the agent write? |
How it works
AI Impact (Comparison) measures AI code from Agent Traces. Span matches merged lines to the author's recorded sessions and computes an AI Code Ratio on qualified PRs, then compares delivery metrics between PRs with more and less AI. A PR qualifies when there is enough trace coverage across its life to attribute it reliably. See How are PRs considered qualified in the AI Impact report when using traces?
Tool utilization comes mainly from each vendor's admin or usage API: GitHub Copilot, Cursor, Claude Code, Augment, ChatGPT and Codex (ChatGPT & Codex Workspace), and the OpenAI API (OpenAI Platform). It reports active users, AI days per week, and the vendor's count of accepted lines. Where your organization collects traces, Span uses trace data for the people and days the traces cover. Tool utilization does not label individual pull requests as AI-assisted.
Agent Traces is the session-level record: the prompts, tool calls, and file edits from each AI session, plus evaluations, task classification, and cost. It is the source behind the Comparison report's AI Code Ratio. Use it to see what an agent did on a piece of work, and to inspect line-level attribution on a pull request. See Understanding Agent Traces in Span (auspex).
Which one to read
"How much of our shipped code is AI-authored?" The Comparison AI Code Ratio.
"Are we getting value from the seats we pay for?" Tool utilization.
"What did the agent actually do, and which lines did it write on this PR?" Agent Traces.
Tool utilization needs only your vendor connections and source control. The Comparison report and Agent Traces need Agent Traces turned on for your organization, with auspex installed on developer machines.
Things to know
High adoption, lower AI Code Ratio. For example, 80% of developers were active in Cursor last week, while the AI Code Ratio is 25%. The first counts who used the tool; the second measures how much of the merged diff Span attributed to an agent. Using a tool doesn't mean most of the merged code came from it.
Accepted lines run higher than merged AI lines. Accepted lines include suggestions accepted on branches that never merged, code later rewritten, and deleted lines. Cursor tab completions count as 2 lines each. See Understanding Accepted AI Lines vs. All Merged Lines.
Qualified PRs start low after rollout. PRs opened before trace collection started can't qualify. That is a coverage gap, not a drop in AI usage.
Missing data can show as zero in Tool utilization. A person whose tool isn't connected, or whose vendor account didn't match a Span person, shows no AI usage there.
Cloud agents. Span detects PRs from cloud agents such as Devin, Codegen, the Copilot coding agent, and Codex Cloud from markers they leave on the PR, and counts them as AI tool usage for the person who requested them. To track an agent's own bot account, see Tracking Agents in Span.
Worked example: in one month, developers accepted 12,000 AI lines across Cursor and Copilot, and your merged PRs contained 10,000 lines. Accepted AI Lines vs. All Merged Lines shows 120%. In the same month, Span attributed 2,500 of 10,000 qualified merged lines to agent sessions, so the Comparison AI Code Ratio is 25%. Both are correct; they measure different things.
Related articles
How AI Code Ratio is calculated
How are PRs considered qualified in the AI Impact report when using traces?
Understanding Accepted AI Lines vs. All Merged Lines
Understanding Agent Traces in Span (auspex)
Need help?
Contact your Span account team or reach out to support@span.app.