feat(config): add fine-grained PR metadata check controls #277

New issue

Closed

opened 2026-05-19 16:47:23 -07:00 by jwilger · 0 comments

jwilger commented

2026-05-19 16:47:23 -07:00

Owner

Problem

Repos can currently enable or disable the PR metadata quality check as a single boolean via .auto_review.yaml (pr_metadata_check: true/false). The default check bundles several distinct expectations together, such as title shape, non-empty body, body not copying the title, and explaining why the change is needed.

Repos should be able to keep the default behavior while tuning these expectations more precisely.

Proposed enhancement

Add repo-config support for fine-grained PR metadata review controls:

Preserve the current default set of metadata checks for existing repos.
Allow each default metadata check to be independently enabled/disabled, or grouped where the checks only make sense as an all-or-nothing unit.
Allow repos to add additional natural-language metadata rules that are included in the metadata quality check.
Keep the metadata prompt treating title/body and repo-provided rules as untrusted/configured data as appropriate; avoid letting PR text inject instructions.
Provide validation for typo'd or unsupported metadata-check config keys via auto-review config validate --strict.

Possible config shape

Exact schema TBD, but something like:

pr_metadata_check:
  enabled: true
  checks:
    title_conventional_or_imperative: true
    body_required: true
    body_not_title_copy: true
    body_explains_why: true
  additional_rules:
    - "Security-sensitive changes must describe the threat model impact."
    - "Schema migrations must mention rollback risk."

Backward compatibility should preserve existing boolean config:

pr_metadata_check: true

Acceptance criteria

Existing pr_metadata_check: true/false behavior continues to work.
Repos can disable at least one default metadata expectation without disabling the whole metadata gate.
Repos can add one or more custom metadata rules to the check.
Config validation documents and validates the new shape.
User/operator docs and .auto_review.example.yaml describe the new options.
Tests cover backward compatibility, selective default-check disabling, custom rule prompt injection, and strict validation.

## Problem Repos can currently enable or disable the PR metadata quality check as a single boolean via `.auto_review.yaml` (`pr_metadata_check: true/false`). The default check bundles several distinct expectations together, such as title shape, non-empty body, body not copying the title, and explaining why the change is needed. Repos should be able to keep the default behavior while tuning these expectations more precisely. ## Proposed enhancement Add repo-config support for fine-grained PR metadata review controls: - Preserve the current default set of metadata checks for existing repos. - Allow each default metadata check to be independently enabled/disabled, or grouped where the checks only make sense as an all-or-nothing unit. - Allow repos to add additional natural-language metadata rules that are included in the metadata quality check. - Keep the metadata prompt treating title/body and repo-provided rules as untrusted/configured data as appropriate; avoid letting PR text inject instructions. - Provide validation for typo'd or unsupported metadata-check config keys via `auto-review config validate --strict`. ## Possible config shape Exact schema TBD, but something like: ```yaml pr_metadata_check: enabled: true checks: title_conventional_or_imperative: true body_required: true body_not_title_copy: true body_explains_why: true additional_rules: - "Security-sensitive changes must describe the threat model impact." - "Schema migrations must mention rollback risk." ``` Backward compatibility should preserve existing boolean config: ```yaml pr_metadata_check: true ``` ## Acceptance criteria - Existing `pr_metadata_check: true/false` behavior continues to work. - Repos can disable at least one default metadata expectation without disabling the whole metadata gate. - Repos can add one or more custom metadata rules to the check. - Config validation documents and validates the new shape. - User/operator docs and `.auto_review.example.yaml` describe the new options. - Tests cover backward compatibility, selective default-check disabling, custom rule prompt injection, and strict validation.

jwilger added this to the 1.1 milestone

2026-05-19 16:47:37 -07:00

jwilger added the

enhancement

label

2026-05-19 16:47:41 -07:00

jwilger referenced this issue from a pull request that will close it,

2026-05-20 07:53:27 -07:00

feat(config): add fine-grained PR metadata controls #281

jwilger closed this issue

2026-05-20 15:01:34 -07:00

jwilger referenced this issue from a commit