Slipstream/auto_review
1
0
Fork 0
Code Issues 2 Pull requests Projects Releases 27 Packages Activity Actions

feat(config): add fine-grained PR metadata controls #281

Merged
jwilger merged 12 commits from issue-277-pr-metadata-controls into main 2026-05-20 15:01:34 -07:00
Conversation 1 Commits 12 Files changed 12 +730 -46
jwilger commented 2026-05-20 07:53:27 -07:00
Owner
Copy link

Why:

  • Repositories need to tune PR metadata quality checks without disabling the entire metadata gate.
  • RGR subagent guardrails also needed hard stops for empty results, missing session-id leases, and recursive delegation discovered while working this issue.
  • Closes #277

What:

  • Preserve legacy pr_metadata_check: true/false and add object-form metadata controls with enabled, checks.body_required, and additional_rules.
  • Pass structured metadata config from repo config through orchestrator into the review pipeline.
  • Allow body_required: false to suppress only empty-body-specific metadata failures while preserving mixed/non-body failures.
  • Include repo-provided additional metadata rules in the Cheap prompt as configured untrusted data that cannot override output, JSON, or system boundaries.
  • Extend strict config validation to reject unsupported nested pr_metadata_check keys.
  • Document the new config shape in .auto_review.example.yaml and docs/USER-GUIDE.md.
  • Harden opencode RGR workflow by rejecting empty RED author results, adding fallback test-author leases, and preventing recursive RGR specialist delegation.

Validation:

  • just fmt
  • just clippy
  • just test
  • just opencode-test
  • cargo test -p ar-review -- --test-threads=1 (used to confirm one env-sensitive test serially)
Why: - Repositories need to tune PR metadata quality checks without disabling the entire metadata gate. - RGR subagent guardrails also needed hard stops for empty results, missing session-id leases, and recursive delegation discovered while working this issue. - Closes #277 What: - Preserve legacy `pr_metadata_check: true/false` and add object-form metadata controls with `enabled`, `checks.body_required`, and `additional_rules`. - Pass structured metadata config from repo config through orchestrator into the review pipeline. - Allow `body_required: false` to suppress only empty-body-specific metadata failures while preserving mixed/non-body failures. - Include repo-provided additional metadata rules in the Cheap prompt as configured untrusted data that cannot override output, JSON, or system boundaries. - Extend strict config validation to reject unsupported nested `pr_metadata_check` keys. - Document the new config shape in `.auto_review.example.yaml` and `docs/USER-GUIDE.md`. - Harden opencode RGR workflow by rejecting empty RED author results, adding fallback test-author leases, and preventing recursive RGR specialist delegation. Validation: - just fmt - just clippy - just test - just opencode-test - cargo test -p ar-review -- --test-threads=1 (used to confirm one env-sensitive test serially)
Why:
- Empty rgr-test-author results can otherwise look like successful delegation and cause retry loops with no ledger-ready RED evidence.

What:
- Reject empty rgr-test-author task results in the discipline plugin.
- Add a regression test for the silent empty-result failure mode.

Validation:
- just opencode-test
Why:
- Repositories need a backward-compatible path from the legacy boolean metadata gate to fine-grained PR metadata controls.

What:
- Allow pr_metadata_check to deserialize from either the legacy boolean or an object with enabled.
- Cover object-form disabled parsing with a focused regression test.

Validation:
- cargo test -p ar-review parses_object_pr_metadata_check_enabled_false
Why:
- The task tool may not expose a delegated subagent session id before rgr-test-author needs to claim its test-edit lease.
- Without a fallback lease, RED authoring can block before any focused failing test exists.

What:
- Add a fallback test-author lease path when delegation lacks a session id.
- Cover the fallback lease claim with an opencode plugin test.

Validation:
- just opencode-test
Why:
- RGR specialist subagents that hit a guardrail must return control to the orchestrator instead of spawning another specialist and repeating the same blockage.

What:
- Deny Task usage for RGR test author, diagnostic implementer, and reviewer agents.
- Tell RGR specialists to stop and report blockers instead of nested delegation.

Validation:
- just opencode-test
Why:
- Repositories need a backward-compatible config shape for selectively tuning PR metadata checks.

What:
- Add structured pr_metadata_check parsing with checks.body_required and additional_rules.
- Preserve legacy boolean deserialization and keep orchestrator enabled checks using the enabled flag.

Validation:
- cargo test -p ar-review parses_pr_metadata_check_false
- cargo test -p ar-review parses_object_pr_metadata_check_enabled_false
- cargo test -p ar-review parses_fine_grained_pr_metadata_check_controls
- cargo test -p ar-orchestrator
Why:
- Repositories need to disable the required PR body expectation without disabling the entire metadata gate.

What:
- Pass structured metadata-check config into the review pipeline.
- Suppress only empty-body-specific metadata failures when body_required is disabled.
- Preserve mixed/non-body metadata failures as blocking.

Validation:
- cargo test -p ar-review disabled_body_required_check_allows_empty_body_metadata
- cargo test -p ar-review disabled_body_required_check_preserves_mixed_metadata_failures
Why:
- Repositories need custom PR metadata requirements included in the metadata quality check without weakening prompt boundaries.

What:
- Thread additional metadata rules into the Cheap metadata prompt.
- Frame repo-provided rules as configured untrusted data that cannot override output, JSON, or system boundaries.

Validation:
- cargo test -p ar-review pr_metadata_prompt_includes_additional_rules_as_configured_data
Why:
- Strict config validation should catch typos in nested PR metadata check controls instead of silently accepting unsupported settings.

What:
- Reject unknown pr_metadata_check and checks keys in strict config parsing.
- Add allowlist drift tests for structured metadata config fields.

Validation:
- cargo test -p ar-review strict_metadata_allowlists_match_struct_fields
- cargo test -p ar-review strict_rejects_unknown_pr_metadata_check_keys
Why:
- The review pipeline now needs the full structured metadata-check config rather than only the enabled flag.

What:
- Carry PrMetadataCheck through workspace preparation into ReviewArgs.
- Use the structured default on workspace-prep fallback.

Validation:
- cargo test -p ar-orchestrator
Why:
- Operators need examples for tuning PR metadata checks and validating nested config keys.

What:
- Document object-form pr_metadata_check controls and additional rules.
- Update the example config and strict-validation guidance.

Validation:
- docs-only change reviewed
Why:
- The metadata-control config changes need to satisfy the repository formatting gate.

What:
- Apply cargo fmt formatting to config.rs.

Validation:
- just fmt
- just test
fix(review): satisfy metadata clippy gate
All checks were successful
CI / Classify changed paths (pull_request) Successful in 1s
Details
CI / Clippy (pull_request) Successful in 41s
Details
CI / opencode plugin tests (pull_request) Successful in 14s
Details
CI / Test (pull_request) Successful in 54s
Details
CI / Dependency policy (pull_request) Successful in 9s
Details
CI / Build (pull_request) Successful in 31s
Details
CI / Build PR artifacts (no token) (pull_request) Successful in 2s
Details
auto_review auto_review: no findings
CI / Format check (pull_request) Successful in 6s
Details
CI / Request auto_review semantic review (pull_request) Successful in 1s
Details
5f12f52224
 
Why:
- The full verification gate rejects needless struct updates in metadata-check tests.

What:
- Remove redundant default updates from PrMetadataChecks test initializers.

Validation:
- just clippy
auto-review approved these changes 2026-05-20 07:56:38 -07:00
auto-review left a comment
Copy link

This PR introduces fine-grained controls for PR metadata checks, allowing repositories to customize their metadata quality checks without disabling the entire gate. The changes appear well-structured and safe to merge.

Walkthrough

  • Configuration Changes: The .auto_review.example.yaml file now supports an object form for pr_metadata_check, allowing for more detailed configuration options such as enabled, checks.body_required, and additional_rules.
  • Code Changes: The codebase has been updated to handle the new configuration structure, including changes in dispatcher.rs, config.rs, and pipeline.rs to pass and validate the structured metadata config.
  • Testing: New tests have been added to ensure the correct parsing and handling of the new configuration options, including tests for strict validation of unknown keys and the behavior of the body_required option.

LLM usage and cost

This PR introduces fine-grained controls for PR metadata checks, allowing repositories to customize their metadata quality checks without disabling the entire gate. The changes appear well-structured and safe to merge. ## Walkthrough - **Configuration Changes**: The `.auto_review.example.yaml` file now supports an object form for `pr_metadata_check`, allowing for more detailed configuration options such as `enabled`, `checks.body_required`, and `additional_rules`. - **Code Changes**: The codebase has been updated to handle the new configuration structure, including changes in `dispatcher.rs`, `config.rs`, and `pipeline.rs` to pass and validate the structured metadata config. - **Testing**: New tests have been added to ensure the correct parsing and handling of the new configuration options, including tests for strict validation of unknown keys and the behavior of the `body_required` option. ## LLM usage and cost - Reasoning (gpt-4o) in=43306 out=16791 cost=$0.468395 - Cheap (gpt-4o-mini) in=537 out=44 cost=$0.000107 Estimated total USD: $0.468502 via https://api.openai.com and https://api.openai.com
Sign in to join this conversation.
Reviewers
No reviewers
auto-review
Labels
Clear labels   
bug

   
discussion

   
enhancement
  
bug
Something is not working

  
duplicate
This issue or pull request already exists

  
enhancement
New feature

  
help wanted
Need some help

  
invalid
Something is wrong

  
question
More information is needed

  
wontfix
This won't be fixed

No labels
bug
discussion
enhancement
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Slipstream/auto_review!281
No description provided.