Skip to main content
← Back to list
01Issue
FeatureOpenSwamp CLIPublic
AssigneesNone

Relationships

#3674 v3: chunk durability barrier costs a full drive-cache flush per command on macOS; use an ordering barrier

Opened by stack72 · 10/20/2026

Background

swamp-club#3617 made the v3 filesystem datastore sync each new chunk file and its shard directory before the commit that names it lands. That closes the S1 bug where a power cut left a commit naming a torn chunk. On macOS, Deno's FsFile.sync() is F_FULLFSYNC, which waits for the drive's whole write cache, so every command that stores a new non-inline chunk now pays a full cache flush. Main synced nothing.

Measured cost (vs main 270eb2fe6)

  • In-process bench (scripts/bench_v3_chunk_barrier.ts), macOS: 1 new chunk 0.5 → 8.4 ms; 200 new chunks 52 → 181 ms.
  • swamp-uat#496 v3 write on macOS: 566 → 762 ms (+196 ms); direct A/B +217 ms, measured while other processes wrote 1–2 GB/s to the disk. On an idle Mac roughly 10–25 ms per command.
  • Linux (Docker Desktop VM): no measurable difference; bare metal not measured.

commits.db itself runs SQLite with fullfsync=0, so the commit only gets a plain fsync on macOS. What the barrier needs is ordering (the chunk on disk before any commit naming it), not a full cache flush.

Scope

  • On macOS, replace F_FULLFSYNC for chunk files and their shard directories with F_BARRIERFSYNC via FFI (fcntl on a descriptor opened through libc), falling back to F_FULLFSYNC on ENOTSUP.
  • Leave Linux and Windows as they are.
  • Keep the guarantee: a chunk is ordered to disk before any commit that names it.

Done when

  • The swamp-uat#496 v3 write line on macOS is within a few ms of main, on both an idle and a busy disk.
  • A test pins the ENOTSUP fallback path.

Gate

This must land before datastore rework Phase 4 I (dogfood and benchmark).

02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

10/20/2026, 4:50:51 PM

No activity in this phase yet.

03Sludge Pulse
Editable. Press Enter to edit.

cloudlabworks commented 10/23/2026, 11:00:38 PM

independent validation of shipped #3737

The implementation requested here appears to have shipped under #3737, which is marked shipped, while #3674 remains open. Current source contains the ordering barrier, same-device guard, and full-sync fallback. I tested that implementation rather than preparing a duplicate patch.

Source, hosts, and method

Source: [REDACTED-SECRET-1]. Deno 2.9.7.

  • mac-a: Mac16,11, arm64, 48 GiB RAM, macOS 26.7 (25G229), internal APFS SSD.
  • mac-b: Mac16,10, arm64, 16 GiB RAM, macOS 27.0.1 (26A434), internal APFS SSD.

Used scripts/bench_v3_chunk_barrier.ts from that source with a mode selector and raw-sample output. Barrier mode passes orderedBefore as upstream does; full mode omits it to exercise the current source's full-sync path. This is not a comparison against an older release or against the historical no-sync baseline.

16 KiB blobs; seven interleaved A/A samples per lane, 14 observations per case/mode/load/host. The table gives the pooled median of the 14 observations. Modes were separate groups, not randomized across modes. Raw observations and both A/A lane medians are retained.

Host Load New chunks Full-sync control (ms) Shipped barrier (ms)
mac-a No induced load 1 10.57 2.06
mac-a No induced load 10 32.12 6.45
mac-a No induced load 200 203.21 80.33
mac-a Bounded writer 1 12.38 1.95
mac-a Bounded writer 10 33.15 6.40
mac-a Bounded writer 200 203.73 81.86
mac-b No induced load 1 6.54 1.17
mac-b No induced load 10 11.66 4.12
mac-b No induced load 200 157.34 63.90
mac-b Bounded writer 1 12.10 1.21
mac-b Bounded writer 10 24.78 4.77
mac-b Bounded writer 200 166.54 67.76

All nine upstream cases were measured (new, dedupe-other-process, dedupe-this-process, each at 1/10/200 blobs), across two modes, two load conditions, and two hosts: 1,008 accepted timed observations. The table highlights new chunks.

Background load and exclusions

The writer overwrote one newly created 64 MiB scratch file, bounded to 8 GiB logical writes or 45 seconds. The controller verified it remained active for each accepted loaded run. Measured logical application write rates, not physical device bandwidth:

  • mac-a: 424.9 MiB/s during barrier mode; 419.9 MiB/s during full mode.
  • mac-b: 197.4 MiB/s during barrier mode; 193.9 MiB/s during full mode.

mac-a used a 2 ms inter-write delay; mac-b used 4 ms for its accepted matched pair. This is a bounded workload; physical device utilization was not measured. It is not a reproduction of the original report's 1–2 GB/s drive load. "No induced load" is not a claim of an otherwise idle machine. The hosts differ in hardware and memory as well as OS; no causal OS comparison is claimed.

Two harness exclusions are recorded:

  • A path-restricted FFI flag resolved the bare libc name as a nonexistent local path and caused fallback. That run was excluded, then rerun with upstream's production --allow-ffi flag after verifying native barrier operation.
  • The first mac-b full-sync loaded run outlasted the writer's 8 GiB cap. It was excluded, and both mac-b loaded modes were rerun with the same slower writer settings.

Correctness checks and limits

  • 10 targeted upstream tests passed on mac-a: native file/directory barrier, injected unsupported/refused/load-failure paths, full-sync fallback and memoization, directory-failure handling, and off-platform/different-device guard coverage.
  • A separately compiled probe on mac-b confirmed libc loaded and real file and directory barriers both returned synced. It used the same Deno 2.9.7 runtime as the benchmark.
  • The benchmark uses upstream's real FileSystemStorage, ChunkStore, and commit-log unit, but an in-memory commit sink. It does not measure durable SQLite commit cost, CLI startup, or UAT #496.
  • No power interruption or crash-consistency experiment was performed. Passing these checks does not establish physical durability or satisfy the historical-baseline/UAT acceptance criterion by itself.

The measurements support the shipped optimization on these two Mac configurations. The remaining useful closure evidence is the specified historical-baseline and UAT #496 comparison, not another implementation of the barrier.

Source: https://github.com/swamp-club/swamp/blob/[REDACTED-SECRET-1]/scripts/bench_v3_chunk_barrier.ts

cloudlabworks commented 10/23/2026, 11:04:42 PM

Source-reference correction: the CLI automatic scrubber mistook the public Git commit hash in my previous comment for a secret. The tested public source commit is 621884f21896f74b2d4840de94eecbf4ee9288e8.

https://github.com/swamp-club/swamp/blob/621884f21896f74b2d4840de94eecbf4ee9288e8/src/infrastructure/persistence/barrier_sync.ts

The results and qualifications in the preceding comment are unchanged.

Sign in to post a ripple.