Understanding Agent Traces in Span (auspex)

Last updated: November 16, 2026

An agent trace is the full record of an engineer's session with an AI coding tool: the prompts sent, MCP calls made, files edited, skills used, and tokens spent. Span collects traces with its agent, auspex, and connects each session to the pull request it produced.

Why it matters

A pull request used to be the unit of engineering work. Today much of the consequential work happens upstream, inside the agent session: scoping the task, gathering context, steering the agent. The agent trace makes that work visible.

Span pulls traces from your AI coding tools into one system of record and, when relevant, connects each session to the PR it produced. That connection is the foundation of Span's AI Effectiveness suite: questions about AI adoption, quality, and cost move from anecdote to evidence.

  • For engineering leaders: see how teams actually use AI coding tools, and correlate AI-generated work with outcomes such as PR cycle time and code quality.

  • For engineers: review your own sessions, learn from ones that went sideways, and see which lines of your PRs an agent wrote.

How it works

auspex hooks into each supported AI tool on the engineer's machine, redacts secrets and personal data on the device, and uploads events to Span. Span's pipeline rebuilds each session, links it to a person by work email, and analyzes it. auspex captures five tools: Claude Code, Cursor, Codex CLI, GitHub Copilot CLI, and VS Code Copilot Chat.

Redaction and privacy mode

auspex redacts secrets and personal data on the device before anything is uploaded, and Span runs a second redaction pass on its own servers before storage. Redaction removes values with a recognizable shape, such as email addresses, keys, and ID numbers. For work that should never be captured, any engineer can turn on privacy mode. Privacy mode discards everything: while it's on, auspex captures, stores, and sends nothing from AI coding sessions on that machine, and only a heartbeat with no captured content keeps reporting that the device is healthy. The change takes effect immediately and persists across restarts.

auspex privacy on
auspex privacy off
auspex privacy

The first command discards all capture, the second resumes capture, and the third shows the current state. Your organization can never turn an engineer's privacy mode off. It can force privacy mode on for every device by asking your Span account team; while that is active, engineers can't turn privacy mode off, and auspex privacy reports that it is enforced by your organization. For what is redacted, custom redaction patterns, and organization-enforced privacy mode, see auspex: Default redaction, custom patterns, and privacy mode.

AI attribution

Because Span analyzes the trace alongside the diff, it can identify which specific lines in each PR an agent generated. Span matches the edits from the PR author's agent sessions against the PR's diff, using the chronology of file changes. You can see the results on the PR catalog page. The PR author can view a line-by-line diff showing attribution by file and by line.

Traces

The Traces page (AI Effectiveness → Traces) is a searchable, filterable record of AI-assisted work across your organization. When a PR surprises you, when an engineer wants to learn from a session that went sideways, or when a reviewer needs to understand intent before approving, the underlying trace is one click away.

  • Secrets and personal data are redacted on the device and again on Span's servers before storage.

  • By default, prompts and full trace conversations are visible only to the engineer who authored the trace. Other people see the trace marked Only visible to author, with a placeholder name instead of its title.

  • Click a trace to open its side panel, with the trace's metadata and its evaluation scores. Everyone with access to the trace sees the scores. Only the author sees the reasoning behind each score, because that reasoning is generated from the session's contents.

  • The author can click View trace to see the full conversation with accompanying citations.

Effectiveness scorecard

Span runs agent evaluations against your traces, asking structured questions of each session and synthesizing results across thousands of interactions. An LLM judge scores eligible sessions from 1 to 5 on three themes: prompt clarity, environment readiness, and quality stewardship. A theme score below 3.5 counts as below threshold. Scores roll up into trends you can slice by team, repo, or topic area.

This is where leaders answer questions like "is our prompt quality improving?" or "which teams are getting leverage from AI, and which are stuck?" When a score dips, you can drill into the specific traces driving the change.

From the scorecard you can see scores over time for the overall score or for each theme. Filter by team or timeframe to adjust the charts and the scores in the Breakdown table. You can view the breakdown by team or by individual; this is the primary view for comparing how effectively different teams or repos use agents. Click the Traces icon on the right to see the traces that contribute most to lowering the score, which shows where improvements can be made.

Task classification

Span classifies each trace by the kind of work performed, so you can see where agents are applied most. A trace can carry more than one category. The categories are:

  1. Feature implementation

  2. Debugging

  3. Refactoring

  4. Testing

  5. Documentation

  6. DevOps

  7. Data work

  8. Environment setup

  9. Question answering

  10. Exploration

  11. Code review

  12. QA verification

  13. Design & planning

  14. Other

A trace without a category shows as Unclassified.

Model usage and autonomy ratio

You can also see a breakdown of model usage and the autonomy ratio: the number of agent tool calls per human prompt, averaged across sessions. A higher number means the agent works longer on its own between messages from the engineer. Typed prompts and typed slash commands count as prompts; messages the tool generates on its own don't. A sub-agent's tool calls count in its parent session.

Harness health

Harness health (formerly Environment Readiness, marked Beta) runs once a week on your busiest repositories and lists the changes to each repository's context files, rules, commands, tools, and access that would stop people from having to step in for the agent. Each recommendation is ranked by how often people stepped in and comes with evidence from the sessions and a fix prompt; you can filter the list by repository and principle. For details, see Harness health: weekly fixes for your AI coding setup.

Setup

  1. Approve the security doc.

  2. Get your organization's traces token. Every install path needs the same credential: a single org-wide value (a span_… string) shared by every tool and every device. Get it from the Traces settings page: the Trace collection card has a View token button, and the token shows regardless of which tools are toggled on. The page is limited to Span admins and people whose permission group can manage integrations. If it doesn't load for you, ask one of them for the token.

  3. Check your Claude Code settings (see below).

  4. Install auspex with the guide that matches how you manage devices (see below).

  5. Ask Span to turn on traces. Once auspex is installed, reach out to your Span account team so we can enable trace processing for your organization. We recommend installing on a few people's machines first to confirm everything works. For larger organizations, start with a pilot.

  6. Grant trace permissions. Who can see traces, their contents, and trace reporting is controlled by four Agent Trace permissions. Out of the box, only the Owners and Admins permission groups have them, so engineers can't open the Traces page, even for their own sessions, until you grant Agent Trace Data to their permission group. See Agent Trace permissions.

Before you install: check your Claude Code settings

Claude Code has a settings behavior that can silently stop auspex from capturing its activity. The device looks healthy and every other tool keeps reporting, but no Claude Code session is recorded.

If any of the following is true, or you don't know the answer, confirm with your Claude Code administrator or your Span account team before you roll out:

  • Do you use Claude Code remote settings (the organization policy configured at claude.ai/admin-settings/claude-code)? Any policy published there, even one that never mentions hooks, can suppress auspex's capture.

  • Have you disabled or restricted hooks in any managed setting, with disableAllHooks, allowManagedHooksOnly, or strictPluginOnlyCustomization?

  • Do your remote settings set "managedSourcesBehavior": "merge"? This setting keeps auspex's hooks running alongside your own policy. If it is absent, the default "first-wins" applies and auspex's Claude Code capture stops.

For the full explanation, how to tell whether a device is affected, and the workarounds, see auspex: Claude Code Remote Settings and Managed Hooks (macOS) or auspex: Claude Code Remote Settings and Managed Hooks (Windows). The behavior is the same on both platforms, and on Linux.

Using OpenCode?

auspex captures OpenCode 2.x through a plugin that install places for you, on MDM-managed and self-installed devices alike. Read Capture OpenCode with auspex if any of these apply:

  • You set OPENCODE_CONFIG_DIR.

  • Your opencode.json lists "-auspex" under plugins, which turns the plugin off.

Install the auspex agent

Once the security doc is approved, use the guide that matches how you manage devices:

To see what's new in each release, check the release notes. Before you upgrade, check Upgrading auspex for releases that need a manual step. If you use an MDM that isn't listed, reach out so we can support you.

Things to know

  • Only the five supported tools are captured. Chat in the Claude desktop app, Copilot inline completions, and other AI tools don't produce traces.

  • Privacy mode is all or nothing. auspex has no per-file or per-repository exclusions. A session run while privacy mode is on never reaches Span, so it doesn't appear on the Traces page and isn't used for AI attribution.

  • Traces take a few hours to appear. Most sessions appear within about 2 hours, and within 4 hours in the worst case. A trace stays hidden until Span has screened it.

  • Evaluation scores arrive about a day after a session ends. Span waits until a session has been quiet for 20 hours, in case the engineer resumes it, then scores it on its next run (every 3 hours). For example, a session that ends at 6 PM Monday is scored between about 2 PM and 5 PM Tuesday.

  • Not every session is scored. Span scores a session only if it is linked to a PR or classified as feature implementation, has at least 1 tool call or 2 prompts, and has 1,250 events or fewer. A session is scored once; later activity doesn't change its score.

  • Autonomy ratio averages sessions, not tool calls. Each session weighs the same. For example, a session with 30 tool calls and 3 prompts (ratio 10) and a session with 4 tool calls and 4 prompts (ratio 1) average to 5.5, even though the first session did far more work.

  • Attribution credits the PR author's sessions. A session run by someone other than the PR's author isn't credited to that PR. A session with no PR has no attribution.

  • Traces need a matching person. Span links each session to a person by the work email on the device. A session whose email doesn't match anyone in Span still counts in organization totals, but in no team or person view.

  • Screened traces still count. Trace Screening keeps a sensitive trace's contents away from everyone and everything but its author, including evaluations, coaching, and Harness health. The trace still counts in metrics calculated from trace metadata, such as session counts and cost.

FAQ

Why does auspex capture on the device instead of through our model provider's API or an LLM gateway?

Span does use model provider APIs, for cost and usage. Traces come from the device because that's where the code is. A gateway sees what the model was asked and what it proposed, such as an edit to src/app.ts. It can't tell which repository that path is in, which branch the engineer has checked out, or whether the edit was applied. It also misses changes the agent makes through shell commands, scripts, and MCP tools, which never pass through the model as an edit. Without that, Span can't connect a session to the pull request it produced.

On the device, auspex records the repository, branch, and commit at the moment of each edit. It does this without sitting between engineers and their models, so if collection fails, the agent keeps working.

Related articles

Need help?

Contact your Span account team or reach out to support@span.app.