Run mutation testing only on release PRs and auto-create blocker issues for surviving mutants #336

New issue

Closed

opened 2026-04-11 08:18:45 -07:00 by jwilger-ai-bot · 0 comments

jwilger-ai-bot commented

2026-04-11 08:18:45 -07:00

(Migrated from github.com)

Problem

Mutation testing (cargo mutants) currently runs as part of the normal development workflow, but it's expensive and doesn't need to gate every PR. Meanwhile, surviving mutants that slip into a release represent real coverage gaps that should be tracked and resolved.

Desired Behavior

Remove mutation testing from regular PR CI — it's too slow for per-PR feedback and adds noise during active development.
Add mutation testing to the release PR workflow — when a release PR is created (by release-please or equivalent), run cargo mutants as a required check.
Block the release if any mutants survive — the mutation testing check must pass with zero surviving mutants for the release PR to merge.
Auto-create GitHub issues for surviving mutants — when mutation testing finds survivors, automatically create one issue per surviving mutant (or grouped by file/function) with:
- Label: P0-critical + bug
- Title identifying the surviving mutant location
- Body with the mutant details and the test gap it represents
- These issues block the release — they must be resolved before the next release attempt

Context

This project requires 100% mutant kill rate — zero surviving mutants. Running mutation testing on every PR is expensive and often irrelevant for small changes. Moving it to the release gate ensures coverage gaps are caught before shipping while keeping the development feedback loop fast.

Acceptance Criteria

Mutation testing removed from per-PR CI checks
Mutation testing added as required check on release PRs
Release PR blocked when any mutants survive (zero tolerance)
Surviving mutants automatically create P0-critical issues with actionable details
Issues include enough context to reproduce and fix the coverage gap

## Problem Mutation testing (`cargo mutants`) currently runs as part of the normal development workflow, but it's expensive and doesn't need to gate every PR. Meanwhile, surviving mutants that slip into a release represent real coverage gaps that should be tracked and resolved. ## Desired Behavior 1. **Remove mutation testing from regular PR CI** — it's too slow for per-PR feedback and adds noise during active development. 2. **Add mutation testing to the release PR workflow** — when a release PR is created (by release-please or equivalent), run `cargo mutants` as a required check. 3. **Block the release if any mutants survive** — the mutation testing check must pass with zero surviving mutants for the release PR to merge. 4. **Auto-create GitHub issues for surviving mutants** — when mutation testing finds survivors, automatically create one issue per surviving mutant (or grouped by file/function) with: - Label: `P0-critical` + `bug` - Title identifying the surviving mutant location - Body with the mutant details and the test gap it represents - These issues block the release — they must be resolved before the next release attempt ## Context This project requires 100% mutant kill rate — zero surviving mutants. Running mutation testing on every PR is expensive and often irrelevant for small changes. Moving it to the release gate ensures coverage gaps are caught before shipping while keeping the development feedback loop fast. ## Acceptance Criteria - [ ] Mutation testing removed from per-PR CI checks - [ ] Mutation testing added as required check on release PRs - [ ] Release PR blocked when any mutants survive (zero tolerance) - [ ] Surviving mutants automatically create `P0-critical` issues with actionable details - [ ] Issues include enough context to reproduce and fix the coverage gap