feat(eventcore-fs): git-mergeable file-based event store backend #390
No reviewers
Labels
No labels
adr
automated
bug
chore
dependencies
documentation
enhancement
epic
github-actions
P1-high
P2-medium
P3-low
release
research
rust
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Slipstream/eventcore!390
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/eventcore-fs-backend"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Adds
eventcore-fs, a file-based event store backend that persists eachappend_eventstransaction as one immutable JSONL file underevents/, namedby a transaction UUID7. Because every transaction is a uniquely named file that
is never edited, a
git mergeof two clones'events/directories is a pureadditive union with zero textual conflicts — the foundation for offline
collaboration on event-sourced developer tools committed to git.
The work was designed in full before any code (the on-disk format is immutable,
so it had to reserve the merge-mode header fields from the first commit), then
implemented in two layers via outside-in TDD.
Phase 0 — design (ADRs 0038–0046 + blueprint)
ADR-0040 locking & coordination (Layer 1)
projection-after-merge, ADR-0044 replica identity, ADR-0045 merge
mode off-trait, ADR-0046 git integration (Layer 2)
blueprints/fs-merge-mode.md(incorporating the discovery brief + BDDscenarios);
store-backends.mdupdated with the new backendLayer 1 — single-writer backend (complete)
gives all-or-nothing multi-stream writes
StreamVersionand global order are computed at read time by linearizinga transaction DAG (ADR-0039); single-writer mode is the degenerate
linear-chain case, so the shared 19-test contract suite passes unchanged
.lock+ per-subscriptionfs4advisory locks (ADR-0040)
FileEventStore,FileCheckpointStore,FileProjectorCoordinatorLayer 2 — merge mode (core complete)
linearizebreaks ties between concurrenttransactions by
(created_at, replica_id, transaction_id), so any two clonesholding the same file set compute byte-identical canonical order
detect_forks/statusreport streams with concurrentsame-base transactions (off-trait API, ADR-0045)
reconcile<E, R>(resolver)hands each fork'styped divergent branches to the application, which returns resolution events
(or
Unresolvable); the library records an N-parent merge transaction thatcollapses the fork and replicates through git
a single resolved head (termination proven by test)
open()writes.gitignore(keeping the replica id, locks,index, and tmp out of git — the primary copy-trap defense) and a defensive
.gitattributes merge=uniontopology_generation()lets projections detectstructural merges and rebuild (ADR-0043)
Quality gates
eventcore-fs(19 contract + unit + merge-mode integration);full workspace green (postgres excluded — needs a live DB)
cargo clippy --all-targets --all-features -D warningscleancargo fmtcleancargo mutants)Scoped follow-ups (each has an ADR; not blocking this PR)
These are the more advanced refinements the ADRs anticipate; the core feature is
complete and shippable without them:
cursor-based projections never rewind after a merge (the topology-generation
signal is implemented; the cursor is the larger remaining surface)
in depth against
cp -r, beyond the.gitignorealready in place) andreconcile-time
ReplicaIdentityConflictdetectiondangling-transaction reporting in
status()for partial/aborted git merges🤖 Generated with Claude Code
Adds the reconcile API (ADR-0042): the application supplies a resolver that inspects each fork's divergent branches (typed events) and returns either resolution events or Unresolvable. Resolution events are recorded as an N-parent merge transaction listing the fork heads as parents, so detect_forks thereafter reports the fork as resolved, and the merge file replicates through git like any other transaction. - reconcile<E, R>(resolver) drives every unresolved fork - ForkContext / BranchView expose the typed divergent events - ResolutionOutcome::{Resolve, Unresolvable}; ReconcileReport summary - detect_forks now filters resolved forks (a transaction descending from all heads), via re-introduced ancestor reachability - the per-event version arithmetic is factored into one build_envelopes helper shared by the append and merge paths Integration tests prove a fork collapses into a merge that both clones converge on, that the resolver sees both branches' events, and that a declined fork stays unresolved. Zero surviving mutants.This PR introduces
eventcore-fs, a file-based event store backend designed for offline collaboration via git merges. The implementation appears robust, with a focus on deterministic linearization and domain-owned reconciliation. The changes seem safe to merge, but adding CI checks for testing and linting would enhance reliability.Walkthrough
eventcore-fsbackend is added, allowing event histories to be stored as immutable JSONL files. This enables offline collaboration by leveraging git's merge capabilities.LLM usage and cost
Estimated total USD: $0.124475 via https://api.openai.com and https://api.openai.com
Pre-merge checks
feat(eventcore-fs): file-based event store backend with git-merge reconciliationto feat(eventcore-fs): git-mergeable file-based event store backendAddressed the pre-merge metadata check: shortened the PR title from 80 to 64 characters, now under the 72-char conventional-commit header limit.
Re the CI suggestion in the review: the repository CI already runs Test, Clippy, Mutation, Format, and Security Audit jobs on every PR (see the checks on this PR), so no additional CI for tests/linting is needed.
@auto-review re-review
Queued a forced review at
9719f46c4e. This intentionally bypasses CI gating; watch the commit-status badge for progress.This PR introduces a new file-based event store backend,
eventcore-fs, which allows for git-mergeable offline collaboration by storing each transaction as an immutable JSONL file. The implementation appears robust, with comprehensive testing and documentation. It is safe to merge.Walkthrough
eventcore-fsas a new package and included necessary dependencies likefs4for file locking.LLM usage and cost
Estimated total USD: $0.124369 via https://api.openai.com and https://api.openai.com
New commits pushed, approval review dismissed automatically according to repository settings
The PR updates the status of several ADR documents from 'Proposed' to 'Accepted', indicating that the design decisions have been finalized and approved. This change is straightforward and does not introduce any code changes, making it safe to merge.
LLM usage and cost
Estimated total USD: $0.016509 via https://api.openai.com and https://api.openai.com
@ -0,0 +10,4 @@workspace = true[dependencies]eventcore-types = { version = "0.8.0", path = "../eventcore-types" }Won't this version constraint be problematic when the version numbers are bumped with the next release? How can we make it reference "whatever version is published along with this" (i.e. the workspace version)?
@ -0,0 +20,4 @@uuid = { version = "1.23.1", features = ["v7", "serde"] }fs4 = { version = "1.1.0", default-features = false, features = ["sync"] }nutype = { version = "0.7.0", features = ["serde"] }Let's also ensure that the versions for all of the other dependencies are up to date with the latest versions and also match the versions used for other crates in eventcore that use the same dependencies. Can that be enforced such that these version numbers are always shared between the crates? Does that even matter in Rust? Or could each of our crates depend on a different version of X and the whole thing would still work after all transitive dependencies are resolved?
@ -0,0 +1602,4 @@"existing git metadata is preserved, not overwritten");}}This file is huge. Wouldn't it be better if we could break this down into modules? Is there a sensible way to do so that isn't just random? I think smaller files are easier to deal with in agentic development harnesses, aren't they?
New commits pushed, approval review dismissed automatically according to repository settings
This PR introduces a new file-based event store backend,
eventcore-fs, which allows for git-mergeable event storage. The changes appear well-structured and modular, with clear separation of concerns across different files. The update to thechronodependency version should be verified for compatibility.Walkthrough
Δ since
7940ae1:Dependency Update:
Cargo.toml: Updatedchronodependency to version0.4.42.New Modules:
config.rs: Handles store configuration and directory layout.coordination.rs: Manages locking and coordination for the event store.error.rs: Defines error types for the file backend.format.rs: Specifies the on-disk transaction file format.index.rs: Implements the in-memory linearized read model.merge.rs: Provides fork detection and reconciliation API types.Refactoring:
lib.rs: Refactored to use new modules, improving code organization and readability.LLM usage and cost
Estimated total USD: $0.102834 via https://api.openai.com and https://api.openai.com
Thanks for the review — addressed all three:
1.
eventcore-types = { version = "0.8.0", path = ... }drifting on release bumps.This matches every sibling backend crate (
eventcore-memory,-sqlite,-postgresall hardcode the sameversion = "0.8.0"). The version is required for a path dependency that is also published to crates.io, and release-plz bumps all of them in lockstep on each release per ADR-025 (every crate usesversion.workspace = true, so they always move together). So it does not drift in practice — itʼs the same mechanism the rest of the workspace already relies on.2. Dependency versions: up to date, matching siblings, and enforcement.
Aligned
chronoto0.4.42(matching theeventcorecrate); the rest already match the sibling crates. On enforcement: in Rust this matters less than it looks — Cargoʼs resolver unifies SemVer-compatible requirements via the lockfile, so e.g.serde "1.0"andserde "1.0.228"resolve to a single compiled copy. Divergent major versions would coexist (multiple copies compiled — wasteful but still correct). The idiomatic way to enforce shared versions is[workspace.dependencies]+dep.workspace = true. This workspace does not use that today (there is no[workspace.dependencies]section), so adopting it is a workspace-wide change touching every crate — I would recommend doing that as its own PR rather than bundling it here. I am also adding a generation guardrail so new crates match sibling versions from the start (separate guardrails PR, below).3.
lib.rsis huge → modularize.Done in
8494efe. Split by responsibility intoerror,config,format,index,merge, andcoordinationmodules;lib.rsdropped from ~1605 to 754 lines (~400 non-test — theFileEventStorefacade, trait impls, and re-exports). Pure refactor: public API unchanged, all 37 tests pass, clippy/fmt clean, zero surviving mutants.Per our review-feedback protocol, the giant-file and dependency-consistency lessons are getting generation guardrails in a separate
chore/guardrail-update-*PR so future agents avoid these in the first place (cheaper than catching them in review).This PR introduces new guidelines for maintaining version consistency across crates and organizing modules by responsibility. These changes aim to improve dependency management and code modularity.
LLM usage and cost
Estimated total USD: $0.021574 via https://api.openai.com and https://api.openai.com