TEST-002 critical test-requirements

Bug fix commits include regression tests

80%+ of bug fix commits have associated test changes (same commit or ±1 commit) to prevent regressions

Question to ask

"How many bugs have you fixed twice?"

Verification guide

Severity: Critical

Check automatically:

  1. Sample recent bug fix commits:

    # Get last 20 bug fix commits with their hashes
    git log --oneline --grep="fix" -n 20
    
  2. For each fix commit, check if test files were modified in that commit OR adjacent commits:

    # Check the fix commit itself
    git show --name-only --pretty="" <commit-sha> | grep -E "\.(test|spec)\.(ts|js|py)$|tests/|__tests__/"
    
    # Check 1 commit before
    git show --name-only --pretty="" <commit-sha>~1 | grep -E "\.(test|spec)\.(ts|js|py)$|tests/|__tests__/"
    
    # Check 1 commit after
    git show --name-only --pretty="" <commit-sha>^1 | grep -E "\.(test|spec)\.(ts|js|py)$|tests/|__tests__/"
    
  3. Calculate ratio: What percentage of fix commits have test changes in the commit or adjacent commits?

Cross-reference with:

  • TEST-001 - tests must exist and run for this to matter
  • Section 9 (Development Workflow) - PR process should enforce this

Pass criteria:

  • 80%+ of bug fix commits have associated test changes (same commit or ±1 commit)

Fail criteria:

  • Most bug fix commits have no associated test changes
  • Pattern of fixes without regression tests

If low ratio, ask user: "Only X% of recent bug fix commits have associated test changes. Is there a process to require regression tests for bug fixes? This prevents the same bug from recurring."

Evidence to capture:

  • Number of fix commits sampled
  • Number with associated test changes
  • Percentage
  • Example fix commits without tests (for review)

Section

08. Testing & Code Metrics

Monitoring & Health