Suggest missing CI linters in AI reviews #51

New issue

Closed

opened 2026-05-03 09:01:27 -07:00 by jwilger · 1 comment

jwilger commented

2026-05-03 09:01:27 -07:00

Owner

Problem

We plan to remove actual linter execution from Auto Review's responsibilities, but reviews should still help projects improve CI coverage. Today the default review flow does not explicitly suggest which linters should be added to CI gates when those linters are absent.

Desired behavior

During the default AI review, inspect the project's existing CI gates and the project/language ecosystem enough to identify relevant linters or static checks that are not currently running in CI.
Emit review findings recommending specific missing CI linter/check additions.
Default these findings to warning severity.
Suppress or lower the recommendation when project memory shows the project has explicitly declined that linter/check.
Keep this advisory only; Auto Review should not resume responsibility for running those linters itself.

Acceptance criteria

Missing-CI-linter recommendations are part of the default review items.
Recommendations name the specific linter/check and the CI gate where it appears absent.
Explicit project-memory declines prevent repeated warning-level recommendations for the declined linter/check.
Behavior is covered by tests for both the default warning case and the memory-declined suppression case.

Related to removing bundled linter execution from the review pipeline and re-scoping sandbox requirements after that change.

## Problem We plan to remove actual linter execution from Auto Review's responsibilities, but reviews should still help projects improve CI coverage. Today the default review flow does not explicitly suggest which linters should be added to CI gates when those linters are absent. ## Desired behavior - During the default AI review, inspect the project's existing CI gates and the project/language ecosystem enough to identify relevant linters or static checks that are not currently running in CI. - Emit review findings recommending specific missing CI linter/check additions. - Default these findings to warning severity. - Suppress or lower the recommendation when project memory shows the project has explicitly declined that linter/check. - Keep this advisory only; Auto Review should not resume responsibility for running those linters itself. ## Acceptance criteria - Missing-CI-linter recommendations are part of the default review items. - Recommendations name the specific linter/check and the CI gate where it appears absent. - Explicit project-memory declines prevent repeated warning-level recommendations for the declined linter/check. - Behavior is covered by tests for both the default warning case and the memory-declined suppression case. ## Related context Related to removing bundled linter execution from the review pipeline and re-scoping sandbox requirements after that change.

jwilger referenced this issue from a pull request that will close it,

2026-05-03 18:33:37 -07:00

feat(review): suggest missing CI linters #65

jwilger commented

2026-05-03 18:44:05 -07:00

Author

Owner

Design refinement from PR #65 review:

The prompt guidance in #65 is useful, but it should not be the final shape of this issue. Missing-CI-linter suggestions need concrete CI evidence rather than relying on changed workflow files or RAG context.

Proposed implementation direction:

When the semantic review is kicked off from the CI workflow step, pass a capped CI-results payload into Auto Review.
Include finished job results so the review can see what actually ran, not just what workflow files appear to define.
Preserve non-failing warnings/annotations/log excerpts from completed jobs so green builds can still surface advisory issues.
Represent skipped/pending/unavailable jobs explicitly so the reviewer avoids overconfident “missing” claims when absence is ambiguous.
Keep Auto Review advisory-only: it should inspect reported CI results and recommend coverage improvements, not run linters itself.

Suggested acceptance detail to add/verify:

CI-triggered reviews include a CI results context section listing completed jobs, conclusions, and capped warning output.
Missing-check recommendations are based on concrete evidence from completed/declared CI data.
Pending/skipped/unavailable jobs are treated as uncertain rather than automatically missing.
Existing project-memory declines still suppress repeated warning-level recommendations for declined checks.

Design refinement from PR #65 review: The prompt guidance in #65 is useful, but it should not be the final shape of this issue. Missing-CI-linter suggestions need concrete CI evidence rather than relying on changed workflow files or RAG context. Proposed implementation direction: - When the semantic review is kicked off from the CI workflow step, pass a capped CI-results payload into Auto Review. - Include finished job results so the review can see what actually ran, not just what workflow files appear to define. - Preserve non-failing warnings/annotations/log excerpts from completed jobs so green builds can still surface advisory issues. - Represent skipped/pending/unavailable jobs explicitly so the reviewer avoids overconfident “missing” claims when absence is ambiguous. - Keep Auto Review advisory-only: it should inspect reported CI results and recommend coverage improvements, not run linters itself. Suggested acceptance detail to add/verify: - CI-triggered reviews include a `CI results` context section listing completed jobs, conclusions, and capped warning output. - Missing-check recommendations are based on concrete evidence from completed/declared CI data. - Pending/skipped/unavailable jobs are treated as uncertain rather than automatically missing. - Existing project-memory declines still suppress repeated warning-level recommendations for declined checks.

jwilger closed this issue

2026-05-03 18:47:17 -07:00

jwilger referenced this issue from a commit

2026-05-03 18:47:19 -07:00

feat(review): suggest missing CI linters (#65)