perf: eliminate double serialization in event append path #361

New issue

Open

opened 2026-04-12 11:56:04 -07:00 by jwilger-ai-bot · 0 comments

jwilger-ai-bot commented

2026-04-12 11:56:04 -07:00

(Migrated from github.com)

Problem

Events are serialized twice during execute():

StreamWrites::append() (eventcore-types/src/store.rs:89) calls serde_json::to_value(&event) — produces a serde_json::Value
Each backend's append_events() then calls serde_json::to_string(event_data) to convert the Value to a JSON string for SQL binding

At ~78 ns per serialize pass plus to_value overhead, this adds ~150-200 ns per event unnecessarily. For a 100-event batch, that's ~15-20 µs of waste.

Proposed Solution

Options (in order of preference):

Serialize directly to String in StreamWrites::append() and store the string, not the Value. Backends bind the string directly.
Have backends accept the Value and use serde_json::to_writer to a pre-allocated buffer instead of to_string.

Option 1 is simpler and eliminates the intermediate Value allocation entirely.

Expected Impact

~150 ns/event saved across all backends. Most impactful for high-throughput in-memory and SQLite workloads.

Location

eventcore-types/src/store.rs — StreamWrites::append() and StreamWriteEntry struct
eventcore-postgres/src/lib.rs — append_events() SQL binding
eventcore-sqlite/src/lib.rs — append_events() SQL binding

Benchmark Baseline

Run cargo bench -p eventcore-bench --bench store_operations -- 'store/append' and cargo bench -p eventcore-bench --bench serialization to measure before/after.

## Problem Events are serialized **twice** during `execute()`: 1. `StreamWrites::append()` (`eventcore-types/src/store.rs:89`) calls `serde_json::to_value(&event)` — produces a `serde_json::Value` 2. Each backend's `append_events()` then calls `serde_json::to_string(event_data)` to convert the `Value` to a JSON string for SQL binding At ~78 ns per serialize pass plus `to_value` overhead, this adds ~150-200 ns per event unnecessarily. For a 100-event batch, that's ~15-20 µs of waste. ## Proposed Solution Options (in order of preference): 1. Serialize directly to `String` in `StreamWrites::append()` and store the string, not the `Value`. Backends bind the string directly. 2. Have backends accept the `Value` and use `serde_json::to_writer` to a pre-allocated buffer instead of `to_string`. Option 1 is simpler and eliminates the intermediate `Value` allocation entirely. ## Expected Impact ~150 ns/event saved across all backends. Most impactful for high-throughput in-memory and SQLite workloads. ## Location - `eventcore-types/src/store.rs` — `StreamWrites::append()` and `StreamWriteEntry` struct - `eventcore-postgres/src/lib.rs` — `append_events()` SQL binding - `eventcore-sqlite/src/lib.rs` — `append_events()` SQL binding ## Benchmark Baseline Run `cargo bench -p eventcore-bench --bench store_operations -- 'store/append'` and `cargo bench -p eventcore-bench --bench serialization` to measure before/after.