#788 swamp data gc prunes the catalog but never deletes objects from S3 datastores (markDirty hook not wired) — sync manifest never shrinks

Summary

On an S3-backed datastore, swamp data gc prunes the local catalog (data-version index) but never deletes the underlying objects from S3. It reports a large bytesReclaimed, but the objects remain in the bucket and the file-level sync manifest (.datastore-index.json) — which enumerates every object under the prefix and is pulled/pushed under the global lock on every sync — does not shrink. So GC does not relieve the sync/lock cost it appears to address.

This is distinct from the global-lock contention in #666 (lock scope). This is the GC ↔ datastore object lifecycle: reclaimed versions are orphaned in S3.

Root cause (swamp core)

createDataGcDeps() in src/libswamp/data/gc.ts constructs the data repository without a markDirty hook:

const unifiedDataRepo = new FileSystemUnifiedDataRepository(
  repoDir,
  dsPath(SWAMP_SUBDIRS.data),
  catalogStore,            // <- 4th arg (markDirty) omitted
);

FileSystemUnifiedDataRepository.delete() calls notifyDirty(), which is a no-op when the hook is absent (src/infrastructure/persistence/unified_data_repository.ts):

private async notifyDirty(relPath?: string): Promise<void> {
  if (this.markDirty) await this.markDirty(relPath);   // markDirty undefined under GC
}

With no dirty signal, the end-of-command sync push has nothing to propagate, so the S3 DELETEs are never issued. Additionally, for lazy-cached S3 datastores the version content isn't present locally, so Deno.remove(versionDir) hits NotFound and only the catalog row (catalogStore.removeVersion) is dropped. bytesReclaimed is computed from catalog/size accounting, not from bytes actually freed in the backend.

Reproduction

1. Repo on @swamp/s3-datastore (MinIO), namespaced, with substantial version history.
2. swamp data gc --force → logs e.g. deleted 7150 expired items, 100621 excess versions reclaimed (192171800 bytes) then push complete, no changes.
3. Inspect the bucket: object count and total size under the prefix are unchanged; {prefix}/.datastore-index.json is rewritten at the same (large) size.

Observed (real numbers)

• GC reported ~183 MB / 100k+ versions reclaimed; sync logged push complete, no changes.
• Post-GC catalog: ~10k rows / ~5 MB.
• Bucket reality, same prefix: 321,678 objects / 470 MB (…/data/ alone 281,583 objects / 270 MB); {prefix}/.datastore-index.json still 154 MB, rewritten on the GC push.
• Sync wall-time unchanged — the global-lock hold is gated by the 154 MB manifest, which still lists all ~321k objects.

Expected

GC should physically delete reclaimed objects from the datastore backend (or mark them dirty so the push deletes them), so that object count, total size, and the sync manifest all shrink — and bytesReclaimed reflects bytes actually freed.

Impact

The mitigation users are told to apply for large-index sync cost (swamp data gc) silently has no effect on S3 datastores. Reclaimed-version objects accumulate forever, the per-file sync manifest keeps growing, and global-lock hold time per sync never improves. Directly compounds #666.

Notes

• A monolithic root .datastore-index.json (103 MB) continues to be rewritten after a namespace migration even though the repo is namespaced — possibly related (the manifest layer seems to track objects independently of catalog/namespace state).
• ephemeral lifetime logs Ephemeral lifetime is not yet implemented, so ephemeral-tagged data is never collected either.

Environment

• swamp 20260617.212026.0-sha.396e0952
• @swamp/s3-datastore (MinIO backend, conditional-write locking), darwin/arm64, Deno 2.8.x
• Adjacent to #666 (single global datastore lock).