Skip to main content
← Back to list
01Issue
BugTriagedSwamp CLIPublic
Assigneeszack

Relationships

#4017 Filesystem datastore (v2): data-version writes are not fsynced, so a power cut can publish zero-byte versions

Opened by zack · 10/27/2026

Description

Split out of swamp-club#3955 as the prevention half. #3955 keeps detection and repair (doctor datastores, the read warning, and --repair).

The v2 filesystem datastore (the default format) writes each data version with atomicWriteTextFile and atomicWriteFile (src/infrastructure/persistence/atomic_write.ts). Those write a temp file and rename it, but never fsync the file or its directory. UnifiedDataRepository then advances the latest marker and upserts the SQLite catalog row. On ext4 with delayed allocation, a power cut can keep the rename, the marker and the catalog row but lose the file data. raw and metadata.yaml come back as zero bytes while is_latest points at them, which is the end state reported in #3955. v3 already writes its own files durably (DurableWriteBatch, swamp-club#3617); the v2 path never got the fsync.

This is not a regression: atomic_write.ts has never fsynced since it was introduced (#346).

Fix (steps 1-4 of the approved #3955 plan)

  1. Add an opt-in sync mode to the atomic write helpers: fsync the temp handle, rename, fsync the directory. Injectable hooks for tests. The default is unchanged for the other callers.
  2. On v2, every data-version write path (save, saveDeferred, append, finalizeVersion and its deferred form, rename) fsyncs the version files and the directory entries that reach them before the latest marker and the catalog publish the version. The marker is written durably too. v3 is unchanged, because commit replay covers it.
  3. DataWriter syncs its own append and stream handles before closing them when the allocation says the store is durable. Only the getFilePath subprocess path is reopened to sync.
  4. Correct design/primitives/data.md: the write sequence, and the 'durable once await returns' claim.

Acceptance

  • Hook-order unit tests show content and metadata synced before the marker write and the catalog upsert on v2, and no syncs on v3.
  • A save loop timed before and after in a scratch repo (not asserted in a test).
02Bog Flow
✓OPEN◉TRIAGED○IN PROGRESS○SHIPPED+ 1 MOREASSIGNED+ 3 MOREFINDINGS

Triaged

10/27/2026, 3:54:16 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
zack assigned zack10/27/2026, 3:49:47 PM

Sign in to post a ripple.