jwilger/eventcore

Fork 0

ADR: Effect Pattern for Core APIs #298

Merged

jwilger merged 1 commit from adr/effect-pattern-for-core-apis into main

2026-02-17 08:25:07 -08:00

jwilger commented

2026-02-17 08:17:20 -08:00

(Migrated from github.com)

Status

Proposed

Date

2026-02-17

Context

EventCore's core functions (execute and run_projection) currently accept trait implementations as direct parameters and call their methods internally:

// Command execution
pub async fn execute<C, S>(store: S, command: C, policy: RetryPolicy)
    -> Result<ExecutionResponse, CommandError>
where C: CommandLogic, S: EventStore

// Projection running
pub async fn run_projection<P, B>(projector: P, backend: &B)
    -> Result<(), ProjectionError>
where B: EventReader + CheckpointStore + ProjectorCoordinator

Inside these functions, infrastructure interactions happen directly:

execute calls store.read_stream() and store.append_events() as part of the five-phase pipeline (stream resolution, read/version capture, state reconstruction, business logic, atomic append)
run_projection calls backend.read_events(), backend.load(), backend.save(), and backend.try_acquire() inside its poll loop

While CommandLogic::handle() is already pure (state in, events out), the execution pipeline and projection infrastructure that surround it are tightly coupled to trait method calls on injected dependencies.

Key Forces:

Testability: Testing the execution pipeline or projection loop requires providing full trait implementations, even when testing orchestration logic rather than storage behavior.
Observability: Understanding what the pipeline does requires tracing instrumentation inside the functions. There is no reified record of operations performed.
Composability: Adding cross-cutting concerns (caching, batching, metrics, circuit breaking) requires modifying function internals or wrapping trait implementations.
Determinism: The execution pipeline interleaves pure logic (state reconstruction, business rules) with effectful operations (reads, writes). Separating these enables deterministic replay and testing.
Alignment with domain purity: CommandLogic::handle() already follows a functional core pattern. The surrounding pipeline should match this discipline.
ADR-010 alignment: Free functions remain the primary API style. This ADR changes what those functions do internally (yield effects instead of calling traits), not the function-based API design.

Decision

Restructure EventCore's core functions so they emit effect requests instead of directly calling trait methods. A shell/runtime loop routes each effect to the appropriate trait implementation, feeds the result back, and repeats until the function completes.

Effect Types

Event Store Effects (used by execute):

ReadStream { stream_id } — replaces direct EventStore::read_stream calls
AppendEvents { writes } — replaces direct EventStore::append_events calls

Projection Effects (used by run_projection):

TryAcquireLock { subscription_name } — replaces direct ProjectorCoordinator::try_acquire calls
LoadCheckpoint { name } — replaces direct CheckpointStore::load calls
SaveCheckpoint { name, position } — replaces direct CheckpointStore::save calls
ReadEvents { filter, page } — replaces direct EventReader::read_events calls
Sleep { duration } — replaces direct tokio::time::sleep calls

Shell/Runtime Loop

The public API signatures remain free functions. Internally, each function becomes a state machine that yields effects:

// Public API unchanged
pub async fn execute<C, S>(store: S, command: C, policy: RetryPolicy)
    -> Result<ExecutionResponse, CommandError>
where C: CommandLogic, S: EventStore
{
    let mut machine = ExecutePipeline::new(command, policy);
    loop {
        match machine.next() {
            Step::Effect(effect) => {
                let result = handle_effect(&store, effect).await;
                machine.resume(result);
            }
            Step::Done(outcome) => return outcome,
        }
    }
}

The internal state machines (ExecutePipeline, ProjectionLoop, etc.) are pub(crate) — not part of the public API (consistent with ADR-030).

What Changes

Component	Before	After
`execute` internals	Calls `store.read_stream()` / `store.append_events()` directly	Yields `StoreEffect`, shell dispatches to `EventStore`
`run_projection` internals	Calls `backend.read_events()` / `backend.load()` / `backend.save()` / `backend.try_acquire()` directly	Yields `ProjectionEffect`, shell dispatches to backend traits
`CommandLogic::handle()`	Already pure	Unchanged
Public API signatures	`execute(store, command, policy)`	Unchanged
Backend trait definitions	`EventStore`, `EventReader`, etc.	Unchanged

What Does NOT Change

Public free-function API style (ADR-010)
Backend trait definitions in eventcore-types
CommandLogic trait and domain event model (ADR-012)
Crate layering (ADR-030)
Existing backend implementations (eventcore-memory, eventcore-postgres)

Rationale

Why effects over direct trait calls:

The effect pattern separates describing what to do from performing it — the same principle behind CommandLogic::handle() being pure, extended to the surrounding pipeline.

Effect-based testing: Test the execution pipeline by providing canned responses rather than implementing full EventStore traits. Pipeline logic tests become fast, deterministic, and focused.
Effect logging: The sequence of effects provides a complete, machine-readable trace — useful for debugging, auditing, and replay.
Middleware composition: Cross-cutting concerns wrap the effect handler rather than the trait implementation, enabling composition without modifying backends.
Deterministic replay: Given recorded effect results, the pipeline can be replayed exactly, enabling snapshot testing of complex multi-step execution flows.

Why keep the public API unchanged:

Users should not need to understand effects to use EventCore. The shell loop is an implementation detail. execute(store, command, policy) remains the primary API.

Why internal state machines:

Making the state machine explicit (rather than relying on async fn suspension) gives control over yield points (effects, not arbitrary .await points), enables testing without an async runtime, and clearly separates the machine (pure) from the shell (effectful).

Trade-offs Accepted:

Internal complexity: More code than direct trait calls, offset by improved testability and composability
Indirection: One extra match per effect, negligible performance impact
Contributor learning curve: Documented pattern, well-established in functional programming

Consequences

Positive

Pipeline testability without full trait implementations
Effect logs provide complete operation traces
Middleware wraps effect handlers for cross-cutting concerns
Deterministic replay from recorded effect results
Entire stack follows functional core / imperative shell
Public API surface unchanged

Negative

More internal moving parts (state machines, effect types, shell loops)
Contributors must understand the effect pattern
Significant internal refactoring of execute and run_projection

Neutral

Backend implementers unaffected (same traits)
Application developers unaffected (same functions)
Change is entirely internal to the eventcore crate

Alternatives Considered

Keep direct trait calls, add middleware trait

Introduce a Middleware<S: EventStore> wrapping store calls. Solves composability but not testability or observability. Rejected.

Async generator / stream-based approach

Use async-stream or nightly generators. Premature dependency on unstable features. Can migrate later if generators stabilize. Rejected.

Full algebraic effects (a la Eff/Koka)

Massive complexity for a focused library. No established Rust ecosystem support. Over-engineered. Rejected.

ADR-010: Free Function API Design — public API remains free functions; effects are internal
ADR-012: Event Trait for Domain-First Design — domain purity principle extended to pipeline
ADR-030: Layered Crate Public API — effect types are pub(crate), not exported

## Status Proposed ## Date 2026-02-17 ## Context EventCore's core functions (`execute` and `run_projection`) currently accept trait implementations as direct parameters and call their methods internally: ```rust // Command execution pub async fn execute<C, S>(store: S, command: C, policy: RetryPolicy) -> Result<ExecutionResponse, CommandError> where C: CommandLogic, S: EventStore // Projection running pub async fn run_projection<P, B>(projector: P, backend: &B) -> Result<(), ProjectionError> where B: EventReader + CheckpointStore + ProjectorCoordinator ``` Inside these functions, infrastructure interactions happen directly: - `execute` calls `store.read_stream()` and `store.append_events()` as part of the five-phase pipeline (stream resolution, read/version capture, state reconstruction, business logic, atomic append) - `run_projection` calls `backend.read_events()`, `backend.load()`, `backend.save()`, and `backend.try_acquire()` inside its poll loop While `CommandLogic::handle()` is already pure (state in, events out), the execution pipeline and projection infrastructure that surround it are tightly coupled to trait method calls on injected dependencies. **Key Forces:** 1. **Testability**: Testing the execution pipeline or projection loop requires providing full trait implementations, even when testing orchestration logic rather than storage behavior. 2. **Observability**: Understanding what the pipeline does requires tracing instrumentation inside the functions. There is no reified record of operations performed. 3. **Composability**: Adding cross-cutting concerns (caching, batching, metrics, circuit breaking) requires modifying function internals or wrapping trait implementations. 4. **Determinism**: The execution pipeline interleaves pure logic (state reconstruction, business rules) with effectful operations (reads, writes). Separating these enables deterministic replay and testing. 5. **Alignment with domain purity**: `CommandLogic::handle()` already follows a functional core pattern. The surrounding pipeline should match this discipline. 6. **ADR-010 alignment**: Free functions remain the primary API style. This ADR changes what those functions *do internally* (yield effects instead of calling traits), not the function-based API design. ## Decision Restructure EventCore's core functions so they emit **effect requests** instead of directly calling trait methods. A shell/runtime loop routes each effect to the appropriate trait implementation, feeds the result back, and repeats until the function completes. ### Effect Types **Event Store Effects** (used by `execute`): - `ReadStream { stream_id }` — replaces direct `EventStore::read_stream` calls - `AppendEvents { writes }` — replaces direct `EventStore::append_events` calls **Projection Effects** (used by `run_projection`): - `TryAcquireLock { subscription_name }` — replaces direct `ProjectorCoordinator::try_acquire` calls - `LoadCheckpoint { name }` — replaces direct `CheckpointStore::load` calls - `SaveCheckpoint { name, position }` — replaces direct `CheckpointStore::save` calls - `ReadEvents { filter, page }` — replaces direct `EventReader::read_events` calls - `Sleep { duration }` — replaces direct `tokio::time::sleep` calls ### Shell/Runtime Loop The public API signatures remain free functions. Internally, each function becomes a state machine that yields effects: ```rust // Public API unchanged pub async fn execute<C, S>(store: S, command: C, policy: RetryPolicy) -> Result<ExecutionResponse, CommandError> where C: CommandLogic, S: EventStore { let mut machine = ExecutePipeline::new(command, policy); loop { match machine.next() { Step::Effect(effect) => { let result = handle_effect(&store, effect).await; machine.resume(result); } Step::Done(outcome) => return outcome, } } } ``` The internal state machines (`ExecutePipeline`, `ProjectionLoop`, etc.) are `pub(crate)` — not part of the public API (consistent with ADR-030). ### What Changes | Component | Before | After | |-----------|--------|-------| | `execute` internals | Calls `store.read_stream()` / `store.append_events()` directly | Yields `StoreEffect`, shell dispatches to `EventStore` | | `run_projection` internals | Calls `backend.read_events()` / `backend.load()` / `backend.save()` / `backend.try_acquire()` directly | Yields `ProjectionEffect`, shell dispatches to backend traits | | `CommandLogic::handle()` | Already pure | Unchanged | | Public API signatures | `execute(store, command, policy)` | Unchanged | | Backend trait definitions | `EventStore`, `EventReader`, etc. | Unchanged | ### What Does NOT Change - Public free-function API style (ADR-010) - Backend trait definitions in `eventcore-types` - `CommandLogic` trait and domain event model (ADR-012) - Crate layering (ADR-030) - Existing backend implementations (`eventcore-memory`, `eventcore-postgres`) ## Rationale **Why effects over direct trait calls:** The effect pattern separates *describing* what to do from *performing* it — the same principle behind `CommandLogic::handle()` being pure, extended to the surrounding pipeline. 1. **Effect-based testing**: Test the execution pipeline by providing canned responses rather than implementing full `EventStore` traits. Pipeline logic tests become fast, deterministic, and focused. 2. **Effect logging**: The sequence of effects provides a complete, machine-readable trace — useful for debugging, auditing, and replay. 3. **Middleware composition**: Cross-cutting concerns wrap the effect handler rather than the trait implementation, enabling composition without modifying backends. 4. **Deterministic replay**: Given recorded effect results, the pipeline can be replayed exactly, enabling snapshot testing of complex multi-step execution flows. **Why keep the public API unchanged:** Users should not need to understand effects to use EventCore. The shell loop is an implementation detail. `execute(store, command, policy)` remains the primary API. **Why internal state machines:** Making the state machine explicit (rather than relying on `async fn` suspension) gives control over yield points (effects, not arbitrary `.await` points), enables testing without an async runtime, and clearly separates the machine (pure) from the shell (effectful). **Trade-offs Accepted:** - **Internal complexity**: More code than direct trait calls, offset by improved testability and composability - **Indirection**: One extra match per effect, negligible performance impact - **Contributor learning curve**: Documented pattern, well-established in functional programming ## Consequences ### Positive - Pipeline testability without full trait implementations - Effect logs provide complete operation traces - Middleware wraps effect handlers for cross-cutting concerns - Deterministic replay from recorded effect results - Entire stack follows functional core / imperative shell - Public API surface unchanged ### Negative - More internal moving parts (state machines, effect types, shell loops) - Contributors must understand the effect pattern - Significant internal refactoring of `execute` and `run_projection` ### Neutral - Backend implementers unaffected (same traits) - Application developers unaffected (same functions) - Change is entirely internal to the `eventcore` crate ## Alternatives Considered ### Keep direct trait calls, add middleware trait Introduce a `Middleware<S: EventStore>` wrapping store calls. Solves composability but not testability or observability. Rejected. ### Async generator / stream-based approach Use `async-stream` or nightly generators. Premature dependency on unstable features. Can migrate later if generators stabilize. Rejected. ### Full algebraic effects (a la Eff/Koka) Massive complexity for a focused library. No established Rust ecosystem support. Over-engineered. Rejected. ## Related Decisions - ADR-010: Free Function API Design — public API remains free functions; effects are internal - ADR-012: Event Trait for Domain-First Design — domain purity principle extended to pipeline - ADR-030: Layered Crate Public API — effect types are `pub(crate)`, not exported