#2026 writeCatalogExport takes 55-150s per workflow step — serializes entire catalog as pretty-printed JSON

Summary

writeCatalogExport in repo_context.ts runs in the per-step flush (via stepLockHook → acquireModelLocks → flush). It iterates every row in the SQLite catalog, serializes each with JSON.stringify(row, null, 2) (pretty-printed), and writes them one-by-one to .catalog-export.json.

At 102k rows (post-GC; was 255k before swamp run gc), this takes 55s. It runs after EVERY workflow step, making a 0.5s method take 55s+ wall clock.

Measured breakdown (fleet-health, single-step workflow)

Root cause

1. Full snapshot every step — the export writes ALL catalog rows, not just the ones that changed. For a step that writes 3 data artifacts, 102k unchanged rows are re-serialized.
2. Pretty-printed JSON — JSON.stringify(row, null, 2) produces ~3x more bytes than compact JSON. The file is consumed by swamp datastore catalog pull, which doesn't need indentation.
3. Per-row file.write — each row is a separate Deno.writeFile call instead of buffered/batched writes.

Suggested fixes (any one would help, all three together would bring it to <1s)

1. Incremental export — only re-export rows that changed since the last export (track a catalog version/sequence number)
2. Compact JSON — JSON.stringify(row) instead of JSON.stringify(row, null, 2)
3. Buffer writes — accumulate rows in memory and write once, or use a streaming JSON writer with a write buffer

Context

• Serve: v20260904.044433.0
• Catalog: 102k rows (after swamp run gc --older-than 3d; was 255k before)
• The pull-side bottleneck (per-prefix watermarks) was fixed in keeb/swamp-mongodb-datastore@9fde8a9 — this is the remaining overhead