Slipstream/eventcore

Fork 0

perf(eventcore-postgres): batch INSERT in append_events (#360) #406

Merged

jwilger merged 1 commit from perf/360-postgres-batch-insert into main

2026-06-13 07:23:01 -07:00

jwilger commented

2026-06-13 07:19:13 -07:00

Owner

Summary

Replaces the per-event INSERT loop in PostgresEventStore::append_events with a single multi-row INSERT per chunk, built via sqlx::QueryBuilder and chunked at 1000 events to stay well under Postgres' 65535 bound-parameter limit. For a 100-event append this turns 100 SQL round-trips into one statement.

Correctness

The BEFORE INSERT version-assignment trigger is unchanged. It still assigns gap-free sequential versions and enforces optimistic concurrency per row in VALUES order. This was verified empirically against PostgreSQL 17: a multi-row INSERT fires the FOR EACH ROW trigger per row in order, and each row's trigger sees the prior rows of the same statement (plpgsql advances the command snapshot between invocations), so versions and conflict detection behave identically to the per-row loop.

The type-erased event payload (Box<dyn Any>) is dropped before the insert loop — only the in-memory store needs it — so the borrows held across .await stay Send.

Tests

Existing contract + internals tests (sequential version assignment, concurrent version conflicts, conflict-preserves-atomicity, cross-stream ordering) all pass. Adds an integration test that appends a 1200-event batch to a single stream (crossing the 1000-row chunk boundary) and reads it back.

Workflow note

Phases 1-3 are not applicable: behavior-preserving internal optimization, no public API change, acceptance criteria specified in the issue. Existing contract + integration tests are the regression safety net (Phase 4 TDD).

Closes #360

## Summary Replaces the per-event `INSERT` loop in `PostgresEventStore::append_events` with a single multi-row INSERT per chunk, built via `sqlx::QueryBuilder` and chunked at 1000 events to stay well under Postgres' 65535 bound-parameter limit. For a 100-event append this turns 100 SQL round-trips into one statement. ## Correctness The `BEFORE INSERT` version-assignment trigger is **unchanged**. It still assigns gap-free sequential versions and enforces optimistic concurrency per row in VALUES order. This was verified empirically against PostgreSQL 17: a multi-row INSERT fires the `FOR EACH ROW` trigger per row in order, and each row's trigger sees the prior rows of the *same* statement (plpgsql advances the command snapshot between invocations), so versions and conflict detection behave identically to the per-row loop. The type-erased event payload (`Box<dyn Any>`) is dropped before the insert loop — only the in-memory store needs it — so the borrows held across `.await` stay `Send`. ## Tests Existing contract + internals tests (sequential version assignment, concurrent version conflicts, conflict-preserves-atomicity, cross-stream ordering) all pass. Adds an integration test that appends a 1200-event batch to a single stream (crossing the 1000-row chunk boundary) and reads it back. ## Workflow note Phases 1-3 are not applicable: behavior-preserving internal optimization, no public API change, acceptance criteria specified in the issue. Existing contract + integration tests are the regression safety net (Phase 4 TDD). Closes #360

jwilger added 1 commit

2026-06-13 07:19:13 -07:00

perf(eventcore-postgres): batch INSERT in append_events (#360 )

CI / Detect Changes (pull_request) Successful in 3s

Details

CI / Request auto_review semantic review (pull_request) Successful in 2s

Details

auto_review auto_review: no findings

CI / Format (pull_request) Successful in 15s

Details

CI / Clippy (pull_request) Successful in 1m52s

Details

CI / Security Audit (pull_request) Successful in 27s

Details

CI / Test (pull_request) Successful in 3m5s

Details

CI / Mutation (pull_request) Has been skipped

Details

CI / CI Gate (pull_request) Successful in 2s

Details

653df11ccd

Replace the per-event INSERT loop in `PostgresEventStore::append_events`
with a single multi-row INSERT per chunk, built via `sqlx::QueryBuilder`
and chunked at 1000 events to stay well under Postgres' 65535
bound-parameter limit. For a 100-event append this turns 100 SQL
round-trips into one statement.

The BEFORE INSERT version-assignment trigger is unchanged: it still
assigns gap-free sequential versions and enforces optimistic concurrency
per row in VALUES order. This was verified empirically — a multi-row
INSERT fires the FOR EACH ROW trigger per row in order, and each row's
trigger sees the prior rows of the same statement (plpgsql advances the
command snapshot between invocations), so versions and conflict checks
behave identically to the per-row loop.

The type-erased event payload is dropped before the insert loop (only
the in-memory store needs it) so the borrows held across awaits stay
Send. Adds an integration test that appends a 1200-event batch to one
stream (crossing the 1000-row chunk boundary) and reads it back.

Workflow note: Phases 1-3 are not applicable — behavior-preserving
internal optimization with no public API change and acceptance criteria
specified in issue #360. Existing contract + integration tests (version
assignment, conflict detection, atomicity, ordering) are the regression
safety net (Phase 4 TDD).

Closes #360

auto-review approved these changes

2026-06-13 07:19:23 -07:00

auto-review left a comment

This PR optimizes the append_events function in PostgresEventStore by batching event inserts, reducing SQL round-trips and improving performance. The changes are well-tested and maintain existing functionality, making it safe to merge.

Walkthrough

eventcore-postgres/src/lib.rs:
- Replaced the per-event INSERT loop with a batched multi-row INSERT using sqlx::QueryBuilder.
- Events are chunked at 1000 to stay within PostgreSQL's parameter limit.
- The BEFORE INSERT trigger behavior remains unchanged, ensuring sequential version assignment and conflict detection.
- Unused import StreamWriteEntry could be removed for clarity.
eventcore-postgres/tests/batch_append_test.rs:
- Added a test to verify that a large batch of events is correctly persisted across chunk boundaries.
- Confirms that the BEFORE INSERT trigger assigns gap-free sequential versions and that all events are stored durably.

LLM usage and cost

Reasoning (gpt-4o) in=2490 out=349 cost=$0.017685
Cheap (gpt-4o-mini) in=620 out=52 cost=$0.000124
Estimated total USD: $0.017809 via https://api.openai.com and https://api.openai.com

This PR optimizes the `append_events` function in `PostgresEventStore` by batching event inserts, reducing SQL round-trips and improving performance. The changes are well-tested and maintain existing functionality, making it safe to merge. ## Walkthrough - **eventcore-postgres/src/lib.rs**: - Replaced the per-event `INSERT` loop with a batched multi-row `INSERT` using `sqlx::QueryBuilder`. - Events are chunked at 1000 to stay within PostgreSQL's parameter limit. - The `BEFORE INSERT` trigger behavior remains unchanged, ensuring sequential version assignment and conflict detection. - Unused import `StreamWriteEntry` could be removed for clarity. - **eventcore-postgres/tests/batch_append_test.rs**: - Added a test to verify that a large batch of events is correctly persisted across chunk boundaries. - Confirms that the `BEFORE INSERT` trigger assigns gap-free sequential versions and that all events are stored durably. ## LLM usage and cost - Reasoning (gpt-4o) in=2490 out=349 cost=$0.017685 - Cheap (gpt-4o-mini) in=620 out=52 cost=$0.000124 Estimated total USD: $0.017809 via https://api.openai.com and https://api.openai.com

jwilger merged commit a9d3da0f60 into main

2026-06-13 07:23:01 -07:00

jwilger deleted branch perf/360-postgres-batch-insert

2026-06-13 07:23:01 -07:00

jwilger referenced this pull request from a commit

2026-06-13 07:23:01 -07:00

perf(eventcore-postgres): batch INSERT in append_events (#360) (#406)