#1600 swamp serve: data.query fails permanently with "UNIQUE constraint failed: catalog.*" — one duplicate catalog 5-tuple aborts every catalog rebuild

Summary

On a swamp serve node (MongoDB datastore, SQLite _catalog.db), every data.query (CEL predicate scan) fails with:

UNIQUE constraint failed: catalog.namespace, catalog.type_normalized, catalog.model_id, catalog.data_name, catalog.version

after ~75s. data.get / data.latest (lookup by name) keep working. Any workflow with a data.query assert gate therefore breaks: the assert errors (rather than evaluating to false), so downstream steps blocked on condition: succeeded never run and the whole workflow fails — even for a benign, low-severity, allowFailure-style gate.

Impact

A single poisoned local record wedges all data.query on the node, persistently, across restarts (local data persists in the HOME volume). In our case it silently broke a daily digest workflow whose gate is size(data.query('modelName == "reading" && specName == "message" && tags.workflowRunId == run.id')) > 0 — the compose step succeeded, but the gate threw and blocked the send.

Root cause (observed)

There is one duplicate 5-tuple in the node's local _catalog.db: the same (model, name, version) exists as a local record with one id and as a datastore record with a different id — i.e. a version number was reused by two writes that raced during a boot-reconciliation cron-replay storm (serve replays every missed cron on boot; overlapping replays wrote the same next-version twice).

The catalog rebuild does a plain INSERT per record scanned (local files + datastore sync). The second insert of the duplicate 5-tuple aborts the entire build, so the query fails. A different node (a laptop CLI) pointed at the same datastore rebuilds fine, because it never materialized the duplicate local record — confirming the datastore itself is clean and the corruption is node-local.

Notes / non-fixes

• --log-level debug gives no per-row trace — the rebuild is a single SQL operation, ~5 log lines then the error.
• Deleting _catalog.db + restart does not fix it — the rebuild re-hits the duplicate among local files.
• Removing one model's local dir does not fix it — any data.query triggers a full repo catalog build, so the dup can be in any model.
• Recovery required moving all local model data aside and letting the node re-sync from the clean datastore.

Repro sketch

On a serve node, induce two writes of the same (model, name) that both compute the same next version (e.g. boot-reconciliation replaying overlapping scheduled runs of a workflow that writes a versioned step-output), then run any data.query.

Asks

1. A read (data.query) must not fail on a catalog INSERT. The catalog build should dedupe / upsert (INSERT … ON CONFLICT DO NOTHING/UPDATE) or skip-and-log a duplicate 5-tuple, rather than aborting the whole rebuild. One bad record should not wedge every query on the node.
2. Prevent the version reuse at write time — serialize next-version assignment per (model, name), or detect and reject a reuse, so overlapping boot-replay runs can't create the duplicate in the first place.