Relationships
#3674 v3: chunk durability barrier costs a full drive-cache flush per command on macOS; use an ordering barrier
Opened by stack72 · 10/20/2026
Background
swamp-club#3617 made the v3 filesystem datastore sync each new chunk file and its shard directory before the commit that names it lands. That closes the S1 bug where a power cut left a commit naming a torn chunk. On macOS, Deno's FsFile.sync() is F_FULLFSYNC, which waits for the drive's whole write cache, so every command that stores a new non-inline chunk now pays a full cache flush. Main synced nothing.
Measured cost (vs main 270eb2fe6)
- In-process bench (
scripts/bench_v3_chunk_barrier.ts), macOS: 1 new chunk 0.5 → 8.4 ms; 200 new chunks 52 → 181 ms. - swamp-uat#496 v3 write on macOS: 566 → 762 ms (+196 ms); direct A/B +217 ms, measured while other processes wrote 1–2 GB/s to the disk. On an idle Mac roughly 10–25 ms per command.
- Linux (Docker Desktop VM): no measurable difference; bare metal not measured.
commits.db itself runs SQLite with fullfsync=0, so the commit only gets a plain fsync on macOS. What the barrier needs is ordering (the chunk on disk before any commit naming it), not a full cache flush.
Scope
- On macOS, replace
F_FULLFSYNCfor chunk files and their shard directories withF_BARRIERFSYNCvia FFI (fcntlon a descriptor opened through libc), falling back toF_FULLFSYNCon ENOTSUP. - Leave Linux and Windows as they are.
- Keep the guarantee: a chunk is ordered to disk before any commit that names it.
Done when
- The swamp-uat#496 v3 write line on macOS is within a few ms of main, on both an idle and a busy disk.
- A test pins the ENOTSUP fallback path.
Gate
This must land before datastore rework Phase 4 I (dogfood and benchmark).
Open
No activity in this phase yet.
Sign in to post a ripple.