Trace evaluation labels: Strong, Typical, Needs attention, and why a trace isn't scored yet
Last updated: November 16, 2026
Each agent trace Span evaluates gets a score from 1 to 5, shown as a label: Strong, Typical or Needs attention. A trace without a score yet shows Analyzing or Pending, and hovering over it tells you why.
Why it matters
The labels let you scan the Traces page for sessions worth a closer look without reading every score. When a trace has no score, the hover text tells you whether to wait, link a pull request, or stop expecting one.
For engineers: see which of your sessions scored well, and why one hasn't been scored.
For managers and admins: find the sessions pulling a team's scores down, and answer "why isn't this trace scored?" without opening a support ticket.
How it works
Where you see the labels. On AI Effectiveness → Traces, each trace row has an overall score icon and one column per evaluation theme (prompt clarity, environment readiness and quality stewardship). Hover over a theme column's header for its full name. Open a trace and select its Evaluation tab to see the label written out for the overall score and for each theme.
What each label means
Label | Score | What it means |
|---|---|---|
Strong | 4.0 or higher | The session did well on this theme. |
Typical | 3.5 to 3.9 | The session met the bar. |
Needs attention | Below 3.5 | The session is below threshold on this theme. These are the traces worth reviewing first. |
Analyzing | No score yet | Span hasn't scored this theme yet. Hover for the reason. |
Span rounds scores to one decimal place for display. Needs attention uses the exact score, so a 3.49 shows as 3.5 but still counts as Needs attention.
Why a trace isn't scored yet
Hover over Analyzing in a theme column, or over Pending on the trace's Evaluation tab, to see the reason:
Hover text | What it means | What you can do |
|---|---|---|
This trace is waiting to be evaluated. | The trace is queued. Span waits until a session has been quiet for 20 hours, in case the engineer picks it back up, then scores it on its next run (every 3 hours). | Wait. Scores usually arrive about a day after the session ends. |
Evaluation is waiting for a linked pull request. | Span scores a session once it's linked to a merged pull request, or classified as feature implementation. This one has neither yet. | Merge the pull request the session produced. Span links a session to a PR when the PR's diff matches the session's code edits, and only credits the PR author's own sessions. |
Evaluation couldn't complete. Span will try again. | An attempt failed and Span will retry on its own. | Wait. If retries keep failing, the trace moves to Not evaluated. |
Why a trace won't be scored
Some traces show Not evaluated on the Evaluation tab, or a dash in the theme columns. Hover for the reason:
Hover text | What it means |
|---|---|
This trace doesn't have enough recorded activity to evaluate. | The session needs at least 1 tool call or 2 prompts. |
This trace exceeds the size supported for evaluation. | Sessions with more than 1,250 events aren't scored. |
The required trace data isn't available under your organization's retention settings. | The session's raw events were deleted under your retention window before Span could score it. |
No linked pull request was found within the 30-day evaluation window. | No pull request linked to the session within 30 days. If a PR links to it later, Span can still score it. |
Evaluation failed after repeated attempts. Span has stopped retrying automatically. | Contact support with a link to the trace. |
A theme can also be skipped on its own. Not applicable means the theme doesn't apply to that session. Insufficient evidence means the session didn't show enough to score it. Neither counts as Needs attention, and neither is included in below-threshold counts.
Things to know
Screened traces aren't scored. A trace that matches a Trace Screening category stays with its author and never reaches evaluations.
A session is scored once. Later activity, or a pull request linked after the session was scored, doesn't change its score.
Everyone with access to a trace sees its labels. Only the trace's author sees the reasoning behind each score, because that reasoning comes from the session's contents.
Worked example: an engineer ends a session at 3:00 PM on Tuesday and merges its pull request that evening. Until about 11:00 AM Wednesday, the theme columns show Analyzing with "This trace is waiting to be evaluated." It's scored by about 2:00 PM Wednesday. It scores 4.2 for prompt clarity (Strong), 3.7 for quality stewardship (Typical) and 3.1 for environment readiness (Needs attention). If the engineer had never merged a pull request and the session wasn't feature implementation, the hover text would read "Evaluation is waiting for a linked pull request." After 30 days with no PR, it would read "No linked pull request was found within the 30-day evaluation window."
Related articles
Understanding Agent Traces in Span (auspex)
How long before a trace shows up in Span?
Trace Screening: Automatically restrict personal or sensitive traces
Need help?
Contact your Span account team or reach out to support@span.app.