% of PRs Merged with Tests

Overview

The % of PRs Merged with Tests report measures the percentage of your team's merged pull requests that include test code changes. This metric provides visibility into your organization's testing discipline and code quality practices.

What This Report Shows

Primary Metric:

• % of PRs Merged with Tests - The percentage of merged PRs containing at least one line of test code

Supporting Metrics:

• Total PRs Merged with Tests (count)

• Total Merged PRs (denominator)

Calculation: (PRs with Tests ÷ Total Merged PRs) × 100

This is classified as a positive metric - higher percentages indicate better testing practices.

How to Access This Report

1. Navigate to Insights > Productivity > Velocity → Delivery in the main menu

2. Locate the "% of PRs merged with tests" metric card

3. View:

   • Current percentage value

   • Historical trend visualization

   • Percentile ranking (organizational comparison)

   • Filter by team, individual, or time period

⚠ Important: This metric requires configuration of test file patterns in your organization settings (see Configuration section below).

How the Metric is Calculated

What Counts as a "Test"

A PR is counted as having tests if it:

• Adds at least one line to a file matching your test patterns, OR

• Modifies at least one line in a file matching your test patterns

Test File Detection

Span uses gitignore-style pattern matching to identify test files:

1. Applies your organization's explicit test file patterns

2. Combines with gitignore rules to exclude ignored files

3. Filters out generated files (lockfiles, build outputs)

4. Counts added/modified lines in matching files

5. Marks PR as "has tests" if total > 0

🔴 Critical Requirement: Your organization must explicitly configure test file patterns. Without configuration, the metric cannot distinguish test files from production code.

Configuration: Setting Up Test Detection

Where to Configure

1. Navigate to Organization Settings → Global Settings → Metrics

2. Find the "% PRs With Tests Metric" section

3. Enter test file patterns using gitignore syntax

4. Maximum pattern length: 5,000 characters

Pattern Syntax (Gitignore Format)

Common Patterns by Language

JavaScript/TypeScript:

*.spec.ts
*.test.js
test/
__tests__/

Python:

test_*.py
*_test.py
tests/

Java:

*Test.java
src/test/**

Ruby:

*_spec.rb
spec/

Go:

*_test.go

Configuration Notes

✅ Changes apply retroactively to all historical PRs
✅ Patterns are evaluated with your gitignore configuration
✅ Metric unavailable until at least one pattern is configured
🔒 Requires: Organization Metrics Settings Management permission

Key Insights from This Metric

High Percentages (80%+)

• Strong testing culture and discipline

• Quality-conscious teams consistently including test coverage

• Lower risk of untested code in production

• Alignment with industry best practices

Moderate Percentages (50-80%)

• Balanced approach with room for improvement

• May include legitimate non-tested PRs (docs, config)

• Worth investigating specific cases

Low Percentages (<50%)

• May indicate legitimate cases (documentation, auto-generated files) OR

• Could signal testing discipline gaps

• Potential risk from untested changes

Questions to Investigate

• Are specific teams/individuals consistently merging untested code?

• Do certain PR types legitimately have fewer tests?

• Is the trend improving or declining?

• Does test coverage correlate with revert rates or defects?

Related Metrics to Review Together

Code Quality Metrics

• PR Revert Rate - High reverts + low test coverage = risk

• % PRs Merged Without Review - Unreviewed + untested = highest risk

• PR Diff Size - Large PRs without tests carry more risk

Delivery Metrics

• PRs Merged / Week - Track velocity vs. testing discipline

• Total Lines of Code - Test code growth vs. production code

Automated Monitoring

• Low PRs with Tests Alert - Triggered when test rate drops 20%+ below historical average

Interpreting the Data

Understanding the Value

Example: "78.5% of PRs merged with tests" = ~4 out of 5 merged PRs included test modifications

Trend Analysis

Drill-Down Dimensions

Segment the metric by:

• Individual contributor - Identify coaching opportunities

• Team - Compare testing discipline across org

• Repository - See which codebases need attention

• Time period - Identify when practices changed

Important Limitations

What This Metric Does NOT Measure

❌ Test quality - Only presence, not effectiveness
❌ Code coverage % - Line or branch coverage percentages
❌ Test execution results - Whether tests pass/fail
❌ Functional correctness - Whether code actually works

When Interpretation Gets Tricky

1. Configuration-only PRs

• Auto-generated dependency updates

• Infrastructure configuration changes

• Legitimately may not need test changes

2. Refactoring PRs

• Pure refactoring without behavior changes

• May not modify tests unless test code itself needs updating

3. Documentation PRs

• README or docs updates

• Will lower percentage but are low-risk

4. Emergency Hotfixes

• Urgent production fixes

• May follow different testing workflows

5. Pattern Mismatches

• If test patterns don't match your repo structure

• May misclassify files

Best Practices

Using This Metric Effectively

✅ DO:

• Combine with defect rate, revert rate, and quality metrics

• Understand your codebase context and PR types

• Review configuration regularly as repos evolve

• Set realistic targets based on your team practices

• Investigate outliers and significant changes

❌ DON'T:

• Use as the sole quality indicator

• Apply arbitrary percentage targets

• Ignore legitimate non-test PRs

• Compare teams with different codebase types

• Assume all PRs need test changes

Setting Realistic Targets

Consider:

• Your codebase type (frontend vs. backend vs. infrastructure)

• Team maturity and existing practices

• PR type distribution (features vs. docs vs. config)

• Industry benchmarks for similar organizations

Quick Reference by Role

Getting Started Checklist

•  Configure test file patterns in Organization Settings

•  Verify patterns match your repository structure

•  Set baseline measurement for current state

•  Define target percentage based on codebase type

•  Set up alerts for significant drops

•  Review metric monthly alongside quality metrics

•  Update patterns as repo structure changes

Summary

The "% of PRs Merged with Tests" metric provides clear visibility into your organization's testing discipline. By properly configuring test file patterns and monitoring trends over time, you can identify quality improvements, spot teams needing support, and ensure your engineering organization maintains high standards for shipped code.

Remember: This metric is one piece of your quality puzzle. Use it alongside code review metrics, defect rates, and cycle time to get the complete picture of your engineering effectiveness.