Relationships
#4019 Datastore v3: data writes the datastore filter keeps local are never fsynced
Opened by zack · 10/27/2026
Description
Follow-up from swamp-club#4017's adversarial review (ADV-1).
swamp-club#4017 makes v2 filesystem data-version writes durable: version files and the directory entries that reach them are fsynced before the latest marker and the catalog publish the version. On v3 it leaves FileSystemUnifiedDataRepository's writes unsynced, because commit replay normally covers them. Each committed key is rewritten by the read model's catch-up and checked after a reboot.
That premise does not hold for data keys the checkout's datastore filter keeps local. datastoreKeyFilter (src/infrastructure/datastore_v3/v3_datastore.ts:353-369) puts data/ keys outside the datastore when the configured directories omit data, or when an exclude pattern matches them (design/enablers/datastores.md, around line 237). The commit-log unit drops any key whose admits() is false: it "stays in the checkout and is never committed" (src/infrastructure/datastore_v3/commit_log_unit_of_work.ts:307-313). The read model never projects it either (sqlite_read_model.ts unprojectedReason returns local).
Those writes still run inside a bound v3 unit, so typedStagingActive(markDirty) is true, and #4017's gate leaves them unsynced. No commit exists to replay them, and nothing records them as unsynced. A power cut before page-cache write-back can therefore leave zero-byte raw and metadata.yaml published as latest for those records: the swamp-club#3955 end state, on v3.
Expected
A data-version write whose key the v3 unit does not admit is made durable the way #4017 makes v2 writes durable: synced before the marker and the catalog publish it.
Notes
- FileSystemUnifiedDataRepository cannot tell today, because filtering happens inside the unit's stageOps. It needs a way to ask the bound unit whether it admits a key, or the gate needs to become "unit admits this key" instead of typedStagingActive.
- This builds on #4017's sync option and repository hooks, so it lands after #4017.
- Narrow configuration: only v3 repos whose datastore directories or exclude patterns keep data/ local.
Open
No activity in this phase yet.
Sign in to post a ripple.