Slipstream/eventcore

Fork 0

test: harden doctests and guard docs against fabricated APIs #426

Merged

jwilger merged 1 commit from docs/doctest-harness into main

2026-06-14 21:23:20 -07:00

jwilger commented

2026-06-14 21:16:42 -07:00

Owner

Summary

Follow-up to #424. Prevents the documentation drift that PR from recurring, by making doc examples compile-checked where feasible and adding a permanent guard against fabricated APIs in the prose docs.

Rustdoc hardening

Converted 24 rustdoc ignore examples to no_run, so CI's existing cargo test --doc --workspace --all-features now compiles them against the real API (previously they were never compiled — the exact gap that let stale examples survive). Conversions added only minimal hidden setup. The remaining ignore examples were deliberately left: they document pub(crate) items (which rustdoc doesn't compile) or need a live store/heavy scaffolding that would obscure the example.

Doctest counts now compiled: eventcore_types 13, eventcore_testing 6, eventcore 11, plus memory/macros/postgres — all green.

Manual doc-accuracy guard

Added eventcore-examples/tests/doc_accuracy_test.rs — a compiled test (run by the normal nextest suite, so no CI changes) that scans the manual, blueprints, and published READMEs and fails if any Rust code block references an EventCore API that does not exist:

fabricated types (EventToWrite, ExpectedVersion, ReadOptions, PoolConfig/MetricsConfig/SecurityConfig/EventCoreConfig, SchemaRegistry, …)
wrong-crate facade imports (eventcore::EventStore et al. — these live in eventcore_types)
fabricated store methods (write_events/list_all_streams/health_check/…)
stale forms (StreamId::from/new, .events_written, emit!)

Why a symbol guard rather than full compilation

The manual's code blocks are illustrative, mostly-partial teaching snippets (axum handlers, app-owned structs) that cannot compile standalone, so a blanket "compile every block" (mdBook/skeptic) would drown in false positives or require annotating hundreds of blocks. The guard targets the actual failure mode — references to APIs that don't exist — with zero false positives. Self-contained, fully compiled examples already live in the other eventcore-examples integration tests and in crate rustdoc.

When EventCore genuinely gains one of these names, drop it from the lists in the test (the failure message says so).

Validation

Full pre-commit gate green (fmt, clippy, nextest incl. the new guard, test --doc, check, audit). 24 converted doctests compile; guard passes on current docs.

🤖 Generated with Claude Code

## Summary Follow-up to #424. Prevents the documentation drift that PR from recurring, by making doc examples compile-checked where feasible and adding a permanent guard against fabricated APIs in the prose docs. ## Rustdoc hardening Converted **24** rustdoc `ignore` examples to `no_run`, so CI's existing `cargo test --doc --workspace --all-features` now **compiles** them against the real API (previously they were never compiled — the exact gap that let stale examples survive). Conversions added only minimal hidden setup. The remaining `ignore` examples were deliberately left: they document `pub(crate)` items (which rustdoc doesn't compile) or need a live store/heavy scaffolding that would obscure the example. Doctest counts now compiled: `eventcore_types` 13, `eventcore_testing` 6, `eventcore` 11, plus memory/macros/postgres — all green. ## Manual doc-accuracy guard Added `eventcore-examples/tests/doc_accuracy_test.rs` — a compiled test (run by the normal `nextest` suite, so no CI changes) that scans the **manual, blueprints, and published READMEs** and fails if any Rust code block references an EventCore API that does not exist: - fabricated types (`EventToWrite`, `ExpectedVersion`, `ReadOptions`, `PoolConfig`/`MetricsConfig`/`SecurityConfig`/`EventCoreConfig`, `SchemaRegistry`, …) - wrong-crate facade imports (`eventcore::EventStore` et al. — these live in `eventcore_types`) - fabricated store methods (`write_events`/`list_all_streams`/`health_check`/…) - stale forms (`StreamId::from`/`new`, `.events_written`, `emit!`) ### Why a symbol guard rather than full compilation The manual's code blocks are illustrative, mostly-partial teaching snippets (axum handlers, app-owned structs) that cannot compile standalone, so a blanket "compile every block" (mdBook/skeptic) would drown in false positives or require annotating hundreds of blocks. The guard targets the actual failure mode — references to APIs that don't exist — with zero false positives. Self-contained, **fully compiled** examples already live in the other `eventcore-examples` integration tests and in crate rustdoc. When EventCore genuinely gains one of these names, drop it from the lists in the test (the failure message says so). ## Validation Full pre-commit gate green (fmt, clippy, nextest incl. the new guard, `test --doc`, check, audit). 24 converted doctests compile; guard passes on current docs. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

jwilger added 1 commit

2026-06-14 21:16:42 -07:00

test: harden doctests and guard docs against fabricated APIs

CI / Detect Changes (pull_request) Successful in 5s

Details

CI / Request auto_review semantic review (pull_request) Successful in 2s

Details

auto_review auto_review: no findings

CI / Format (pull_request) Successful in 23s

Details

CI / Clippy (pull_request) Successful in 3m31s

Details

CI / Security Audit (pull_request) Successful in 37s

Details

CI / Test (pull_request) Successful in 5m29s

Details

CI / Mutation (pull_request) Has been skipped

Details

CI / CI Gate (pull_request) Successful in 3s

Details

7cf8a7b041

Prevents the documentation drift fixed in #424 from recurring.

- Convert 24 rustdoc `ignore` examples to `no_run` so CI's existing
  `cargo test --doc --workspace` compiles them against the real API. The
  remaining `ignore` examples document pub(crate) items (which rustdoc does
  not compile) or require a live store, so they are left as-is.
- Add eventcore-examples/tests/doc_accuracy_test.rs: a compiled guard (run by
  the normal test suite) that scans the manual, blueprints, and published
  READMEs and fails if any Rust code block references an EventCore API that
  does not exist — fabricated types (EventToWrite, ExpectedVersion, ReadOptions,
  PoolConfig/MetricsConfig/SecurityConfig/EventCoreConfig, SchemaRegistry, …),
  wrong-crate facade imports (eventcore::EventStore et al.), fabricated store
  methods (write_events/list_all_streams/health_check/…), and stale forms
  (StreamId::from/new, .events_written, emit!).

The manual's code blocks are illustrative, mostly-partial snippets that cannot
compile standalone, so the guard checks symbol existence rather than full
compilation; the self-contained, compile-checked examples live in the other
eventcore-examples integration tests and in crate rustdoc.

auto-review approved these changes

2026-06-14 21:17:04 -07:00

auto-review left a comment

This PR enhances the documentation testing by converting ignored Rust examples to no_run for compilation checks and adds a new test to guard against fabricated API references in documentation. The changes appear safe to merge.

Walkthrough

eventcore-examples/tests/doc_accuracy_test.rs: Introduces a new test to ensure documentation accuracy by checking for fabricated API references.
Various lib.rs files: Converts ignore to no_run in Rust code examples to ensure they are compiled, preventing stale examples from persisting.

LLM usage and cost

Reasoning (gpt-4o) in=6598 out=1625 cost=$0.057365
Cheap (gpt-4o-mini) in=890 out=56 cost=$0.000167
Estimated total USD: $0.057532 via https://api.openai.com and https://api.openai.com

This PR enhances the documentation testing by converting ignored Rust examples to `no_run` for compilation checks and adds a new test to guard against fabricated API references in documentation. The changes appear safe to merge. ## Walkthrough - **eventcore-examples/tests/doc_accuracy_test.rs**: Introduces a new test to ensure documentation accuracy by checking for fabricated API references. - **Various `lib.rs` files**: Converts `ignore` to `no_run` in Rust code examples to ensure they are compiled, preventing stale examples from persisting. ## LLM usage and cost - Reasoning (gpt-4o) in=6598 out=1625 cost=$0.057365 - Cheap (gpt-4o-mini) in=890 out=56 cost=$0.000167 Estimated total USD: $0.057532 via https://api.openai.com and https://api.openai.com

jwilger merged commit 4fd9d0f549 into main

2026-06-14 21:23:20 -07:00

jwilger referenced this pull request from a commit

2026-06-14 21:23:22 -07:00

test: harden doctests and guard docs against fabricated APIs (#426)