Skip to main content
← Back to list
01Issue
BugOpenSwamp CLIPublicTeam
AssigneesNone

Relationships

#4162 Filesystem control-plane put races empty-parent cleanup (cron pending-run ENOENT)

Opened by nckrtl · 10/30/2026

Filesystem control-plane put races delete's empty-parent cleanup, causing cron dual-write ENOENT.

Observed with 20261028.205235.0-sha.1602a617 on Linux/systemd, local filesystem datastore. Serve intermittently logs Control-plane dual-write failed for cron pending run ... No such file or directory: writefile .swamp/_control/pending-runs/.<uuid>.tmp while scheduling and claiming concurrently.

FileSystemControlPlaneStore.put() does mkdir(dirname(path), {recursive:true}) then atomicWriteFile(). A concurrent delete() of the directory's last record calls cleanupEmptyParentDirs() between those two operations. The atomic temporary-file write then fails with NotFound. This reproduces without retention or GC. Recreating the directory is temporary because normal record deletion removes it again. putIfAbsent() already handles a similar NotFound window when staging, but put() does not.

Deterministic minimal test, run in the Swamp source checkout:

import { assertRejects } from "@std/assert";
import { FileSystemControlPlaneStore } from "./src/infrastructure/persistence/fs_control_plane_store.ts";

Deno.test("control put races cleanup of an empty parent", async () => {
  const root = await Deno.makeTempDir();
  const store = new FileSystemControlPlaneStore(root);
  const mkdir = Deno.mkdir;
  try {
    await store.put("pending-runs/old", new TextEncoder().encode("old"));
    let once = true;
    Deno.mkdir = async (...args: Parameters<typeof Deno.mkdir>) => {
      await mkdir(...args);
      if (once && String(args[0]).endsWith("/pending-runs")) {
        once = false;
        await store.delete("pending-runs/old");
      }
    };
    // Current behavior: NotFound. Expected after a fix: put succeeds and
    // get("pending-runs/new") returns the complete new bytes.
    await assertRejects(
      () => store.put("pending-runs/new", new TextEncoder().encode("new")),
      Deno.errors.NotFound,
    );
  } finally {
    Deno.mkdir = mkdir;
    await Deno.remove(root, { recursive: true });
  }
});

Please make atomic control-plane puts safe against concurrent empty-parent cleanup, and add this interleaving as a regression. No production records or credentials are needed for the reproduction.

02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

10/30/2026, 9:06:15 AM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.