Event schema versioning and upcasting support #325

New issue

Closed

opened 2026-04-05 10:33:33 -07:00 by jwilger-ai-bot · 1 comment

jwilger-ai-bot commented

2026-04-05 10:33:33 -07:00

(Migrated from github.com)

Problem

When event schemas evolve over time (new fields, changed types, removed fields), previously persisted events may fail to deserialize against the updated Rust types. Currently, eventcore stores events as JSON blobs via serde with no built-in versioning, schema registry, or upcasting mechanism. If a variant's structure changes incompatibly, serde_json::from_str fails hard during stream replay.

For additive changes (new fields with defaults), applications can use #[serde(default)] at the application layer. But for non-backwards-compatible changes (field removal, type changes, semantic incompatibility), there's no path forward without manual workarounds.

Desired Behavior

An upcasting system that allows applications to:

Version events — associate a schema version with each event variant
Register upcasters — transformations from old event shapes to new ones
Transparent replay — during stream replay, old events are automatically upcast to the latest version before being passed to CommandLogic::apply or Projector handlers

Application code only handles the latest event version. Old versions are transparently transformed.

Possible Approaches

Type-tagged storage: store event type name + version alongside the JSON payload (the event_type column already exists but isn't used for version routing)
Upcast registry: application registers fn(serde_json::Value) -> serde_json::Value transformations keyed by (event_type, from_version, to_version)
Chain upcasting: v1 → v2 → v3 applied in sequence, so only adjacent-version transforms need to be written

Context

This came up in a consuming project (stochastic_macro) that follows outside-in TDD, where event fields are added incrementally across implementation slices. Additive changes work with #[serde(default)], but non-backwards-compatible changes are currently blocked.

## Problem When event schemas evolve over time (new fields, changed types, removed fields), previously persisted events may fail to deserialize against the updated Rust types. Currently, eventcore stores events as JSON blobs via serde with no built-in versioning, schema registry, or upcasting mechanism. If a variant's structure changes incompatibly, `serde_json::from_str` fails hard during stream replay. For additive changes (new fields with defaults), applications can use `#[serde(default)]` at the application layer. But for non-backwards-compatible changes (field removal, type changes, semantic incompatibility), there's no path forward without manual workarounds. ## Desired Behavior An upcasting system that allows applications to: 1. **Version events** — associate a schema version with each event variant 2. **Register upcasters** — transformations from old event shapes to new ones 3. **Transparent replay** — during stream replay, old events are automatically upcast to the latest version before being passed to `CommandLogic::apply` or `Projector` handlers Application code only handles the latest event version. Old versions are transparently transformed. ## Possible Approaches - **Type-tagged storage**: store event type name + version alongside the JSON payload (the `event_type` column already exists but isn't used for version routing) - **Upcast registry**: application registers `fn(serde_json::Value) -> serde_json::Value` transformations keyed by (event_type, from_version, to_version) - **Chain upcasting**: v1 → v2 → v3 applied in sequence, so only adjacent-version transforms need to be written ## Context This came up in a consuming project (stochastic_macro) that follows outside-in TDD, where event fields are added incrementally across implementation slices. Additive changes work with `#[serde(default)]`, but non-backwards-compatible changes are currently blocked.

jwilger-ai-bot commented

2026-04-11 17:15:10 -07:00

(Migrated from github.com)

Completed in #343 and #344 — ADR-0035 documents enum variant approach for schema evolution, and Event::event_type_name() added for stable storage.