Lab #2975: findBySpec/findByTag lose a workflow step's output after a version is deleted, pruned or rolled back

Follow-up to swamp-club#2520 (PR swamp-club/swamp#2800).

Background

The data catalog keeps two latest flags per row (CatalogStore in src/infrastructure/persistence/catalog_store.ts):

is_latest: one row per data name, the highest promoted version. Used by data query, data get, data.latest() and readModelData.
is_step_latest: each workflow step's latest version of the name. Used only by the CEL helpers data.findBySpec and data.findByTag (via the latestPerStep query option), so every step's output stays visible (swamp-club#1761).

Problem

Paths that remove a version re-promote only the surviving highest version through catalogUpsert / upsertNewVersion. They never restore is_step_latest for another step whose latest was the removed row.

Example: one data name written by two steps.

v1  step s1   is_latest 0  is_step_latest 0   (superseded by s1 v3)
v2  step s2   is_latest 0  is_step_latest 1
v3  step s1   is_latest 1  is_step_latest 1

findBySpec returns v2 and v3. After v3 is removed, v2 is re-promoted, but v1 stays at is_step_latest 0, so findBySpec returns only v2 and step s1's output silently disappears. A full catalog rebuild (computeLatestFlags) would correctly mark v1 as s1's step latest.

Affected removal paths in src/infrastructure/persistence/unified_data_repository.ts:

swamp data delete with a version (delete-specific-version, re-promotes newLatest)
GC pruning (collectGarbage) and the write-time version cap (pruneExcessVersions)
rollbackVersions for deferred writes. Here a lower version of the same step promoted while a higher deferred write of that step was in flight stays at is_step_latest 0, because upsertNewVersion counts the in-flight row as a higher row in scope. If the deferred write is then rolled back, the step has no step latest.

Only the per-step view is affected. is_latest (data query, data get, data.latest, readModelData) is restored correctly by these paths.

The delete case predates #2520: the old per-step is_latest had the same hole. It is documented as a known gap in design/enablers/data-query.md (Step-aware versioning).

Suggested fix

After removing a version, recompute both flags for that one (namespace, type, model, name) group inside the same transaction: load the group's remaining rows, run computeLatestFlags (src/domain/data/data_query_service.ts), and write back changed rows. This is what enforceUniqueLatest already does catalog-wide. Then remove the known-gap note from the design doc.

Tests

catalog_store / unified_data_repository tests for delete-version, GC prune, version-cap prune and rollback, each leaving every step with a step latest that matches computeLatestFlags.
Extend the fast-check property in src/domain/data/data_query_service_property_test.ts to interleave removals with promotions.

Session-start 'swamp model search --json' in the AGENTS.md template is unfulfillable - wire it to a hook or drop it

Flaky property test: analyzeExpression dot/bracket agreement fails when the generated model name is a CEL keyword

Accept usernames for user subjects in grant files

Docs: enable-managed-config should say to re-run config migrate when its push fails

Docs: grant file reference shows a stale format and omits resources and subjects

Flaky property test: analyzeExpression hyphenated dot access fails when modelName generates a CEL keyword ("in")

extension quality and push crash with IsADirectory when additionalFiles lists a directory

extension push: a required global argument renders with a stray "" ("text": "string""")

Docs: extension publishing reference is stale on bare specifiers, quality output and fmt config (swamp-club#3022)

swamp-extensions: replace bare import specifiers with npm:/jsr: specifiers in first-party extensions

extension push: 'Requested visibility' line prints to stdout without the logger prefix, twice per run

A nested structural swamp that outlives the run that started it keeps skipping that run's lock while the holder writes

Show signal waits to remote clients and the dashboard

Settle expired signal waits from swamp serve

Resume signalled workflow runs automatically under swamp serve

extension push/quality: structured acceptances for agents, a clearer title than For next time, and an optional reason

extension push: the undecided private-entitlement verdict still says the registry may start a trial at publish

seed:collective cannot produce a superseded personal trial or a returned trial, so the UAT cannot cover them

Replayable state fragments: Git-reviewable desired state for resources changed by model methods

extension push: registry metadata and content hash label files relative to the repo dir, so a push with --extensions-dir records ../ paths

registry: store and show declared acceptances sent in contentMetadata.acceptances at confirm (swamp-club#3021)

extension push --dry-run: authentication verdict prints 'Signed in as .' when the credential is an API key with no username

serve --remote-only: step output from workers is never persisted to the run log (history logs always empty)

serve HA: worker enrollment on a replica that lacks the enrollment token's definition may create a second definition (unverified)

serve HA: pull a token's definition on an auth miss so a peer accepts a new token at once

extension push: version-drift check reports 'no previously published version found' when the registry call failed

Search and list output follows directory iteration order in the definition, workflow, vault and output repositories

extension push: accept attestation-backed adversarial-review evidence (lookup by extension and content hash), generated provenance, and carry the evidence to the registry at confirm (design: Lab #3023)

Workflow load errors print the raw Zod issue dump instead of a readable message

Decide a stricter naming rule for workflow step and job names

serve: run scheduled workflows concurrently (per-workflow serialization), and report queue delay

Show expired approval gates as expired, with a Cancel action in the dashboard

digitalocean codegen leaves an orphaned security_secret.ts model file after its endpoint left the spec

extension promote accepts a yanked version and reports it promoted

Docs: doctor install reports a stale or failing autoupdate scheduler

extension push output: double-quoted adjacent interpolations, repo-relative paths, and --json emits three JSON documents

auto-resolve: the Installing line prints the extension's entire multi-line description

s3-datastore/serve: readResource without a version returns an older version; the latest marker is rewritten backwards

Publish agent-facing projections of the manual: /llms.txt, .md pages, and llms-full.txt

Let agent-runner drive more agent CLIs (Kilo Code and others)

s3-datastore: every fast-path miss re-downloads every _index shard, so a busy serve pulls the whole index every poll (~230 GB/day S3 egress)

Feedback: Swamp as an approval-gated control plane for a small fleet

extension safety analyzer's Deno.Command( warning is a plain substring match

Tell timeouts apart from cancels in method-run records, and review the hidden 30 s fallback timer for step-called model methods

Run from a git worktree: definitions from the worktree, state from the shared repo

S3/GCS datastore: a push that fails after uploading drops its recorded deletes, so the retry brings deleted data back

Tests: bring the in-memory remote's default semantics up to @swamp/s3-datastore and @swamp/gcs-datastore 2026.10.01.1

Tell the pushing client when its extension contentMetadata fails validation

Extension datastore: query still returns an item after its delete is pulled

Cancel, reject and supersede of a parent run should settle its suspended nested child runs

Relationships

#2975 findBySpec/findByTag lose a workflow step's output after a version is deleted, pruned or rolled back

Background

Problem

Suggested fix

Tests

Shipped