Skip to main content
← Back to list
01Issue
FeatureOpenSwamp CLIPublic
AssigneesNone

Relationships

#4310 serve: give the definition index a change signal (file watcher with a sync barrier) so a write to a definitions directory costs O(1), not a stat of every file in it

Opened by zack · 11/2/2026

Follow-up to #4307.

serve's DefinitionIdIndex (src/infrastructure/persistence/definition_id_index.ts) validates by stat. When a definitions directory's stat moves, it re-lists the directory and stats every file in it. Deno.readDir returns no inode, and a save that renames a temp file over an existing name shows up only in that file's own stat. So each write to a type directory costs O(N) stats, where N is the number of definitions in it. For 2s after a write (RUN_INDEX_RACY_WINDOW_MS), every index check re-lists again, because sharing a listing would mean trusting a directory stat younger than the filesystem's timestamp resolution.

#4307 makes each re-list cheaper (only changed entries are re-indexed, and stats run sync in time slices), but the cost per write is still proportional to the directory size. A workflow that gives each run its own behaviorName keeps growing the auto-definitions directory, so serve CPU per run still grows with the number of runs ever made, just with a smaller slope.

Direction

Give the index a change signal so one write costs O(1):

  • Watch the definition roots (Deno.watchFs) and mark only the paths in events as dirty.
  • Add a sync barrier before trusting the watcher: write a cookie file and wait for its event, as watchman does. Otherwise a check that starts after a write completes but before the write's event arrives would answer stale, and a lookup by id that misses in the index is final since #4280.
  • Fall back to the current stat-based re-list on watcher overflow, error, or an unsupported filesystem (network filesystems, inotify watch limits).

Constraints

  • A miss must stay correct for writes from any process: the CLI, workers, and v3 datastore pulls, not only serve's own saves.
  • Must hold on Linux, macOS and Windows.
02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

11/2/2026, 6:07:54 PM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.