Relationships
#2513 worker prune: remote datastore keeps deleted worker records (no-path markDirty() skips deletions)
Opened by stack72 · 9/24/2026
Description
swamp worker prune deletes worker records locally, but the remote (S3/GCS) datastore keeps them, so a later hydration pull brings them back.
In src/cli/commands/worker_prune.ts, the delete deps are built with repoContext.markDirty (around line 160), so modelDelete marks each deleted file by path. Then, around line 287, the command calls syncService.markDirty() with no path right before pushChanged. In the S3 extension (datastore/s3/extensions/datastores/_lib/s3_cache_sync.ts, swamp-extensions repo):
- A no-path
markDirty()setsbulkInvalidated(around lines 1702-1707). - The full-walk push computes deletions only when the dirty-path set overflowed (around lines 3040-3055). The comment there says a no-path
markDirty()"is a modification signal, not a deletion signal". - After the push,
dirtyPathsis cleared (around line 3178), so the per-path deletion marksmodelDeletesent are dropped.
The GCS extension has the same guard (gcs_cache_sync.ts around lines 2887 and 3011).
worker prune is the only CLI command that deletes files and then marks without a path. The other CLI commands that do the same (access grant and group, token mint, worker token create and revoke, model create and others) follow writes, which the full walk still uploads, only more slowly. swamp-club#2415 removed the equivalent no-path calls from serve; this CLI site was not in its scope.
This is a code trace across both repositories, not a reproduction against a live bucket.
Expected
swamp worker prune relies on the per-path marks from modelDelete and drops the no-path markDirty(), so the push takes the scoped path that deletes the remote objects.
Lower-priority notes
- Serve's
WorkerGcService(the "Worker GC" block insrc/cli/commands/serve.ts, andsrc/serve/worker_gc_service.ts) never callspushChangedafter a sweep. Serve has no no-pathmarkDirty()calls left, so its per-path deletion marks go out with the next handler push. The result is a delay, not a loss: on an idle server the remote stays stale until some mutation pushes. Pushing after a sweep that deleted something, as themodel deletehandler does, would close that gap. - Unverified: serve's worker GC builds its delete deps with
createModelDeleteDeps(repoDir, datastoreResolver, undefined, repoContext.markDirty), without injectingrepoContext's definition or data repositories.integration/serve_deps_rules_test.tsrequires serve handlers to inject them, because fresh repositories can miss datastore-resolved paths under managedConfig plus a namespace. Whether that causes a real miss for worker definitions has not been checked.
Found while planning swamp-club#2413.
Open
No activity in this phase yet.
Sign in to post a ripple.