Lab #1802: data catalog: multiple is_latest=1 rows per (model,data) — latest-demotion skips workflow step-output rows

Summary

swamp data query / data.findByTag can return two rows for the same (model, data) instance because the data catalog holds multiple rows with is_latest = 1. swamp data versions <model> <name> recomputes latest from files and correctly shows one, so the two views disagree. Downstream this produces duplicated output — in our case, duplicate rows in a README generated from data.findByTag('specName','cheatsheet').

Environment

swamp 20260824.001920.0-sha.f0180ad4
Datastore: @webframp/gitlab-datastore v2026.08.21.2 (shared across two machines)
macOS (darwin 25.6.0)

Steps to reproduce / evidence

Have a data instance that receives a compile-style model-method write (marked latest), then shortly after an approve workflow step-output write of the same spec (tags: source=step-output, workflow=…, step=approve).

Query the catalog:

SELECT model_name, data_name, COUNT(*)
FROM catalog WHERE is_latest=1
GROUP BY namespace, type_normalized, model_id, data_name
HAVING COUNT(*) > 1;

In our repo this returned 9 instances, each with two is_latest=1 rows — always the model-method row + the approve step-output row.

Run a fresh compile (model-method) on an affected instance. It demotes every prior model-method row to is_latest=0 but leaves the most-recent workflow step-output (approve) row still is_latest=1, and adds the new row as latest → still two latest rows. A workflow-step write appears to demote nothing at all.

Expected

Exactly one is_latest=1 per (namespace, model, data). A new write should demote all prior latest rows regardless of owner_type / source (model-method vs. workflow step-output).

Actual

is_latest demotion is effectively scoped by owner/source: model-method writes don't demote workflow step-output rows, and workflow-step writes don't demote prior rows. Approve/step-output rows accumulate as permanent additional latests.

Impact

Any consumer using data.findByTag / data query on latest gets duplicate records.
Recurs on every approve, so data-only cleanup does not hold — especially on a shared datastore where a second machine keeps writing.
No CLI command recomputes is_latest: gc (lifetime/version-cap only), datastore compact (vacuum), datastore migrate-index (index layout) don't, and data delete no longer has a --version flag (whole-artifact only), so there is no supported way to repair a corrupted catalog.

typed-secrets query view: route a row's JSON content by whether the adapter adapted the record, not by the service's namespace rule

Scheduled flaky-test detection is failing

Update public OpenTelemetry manual for remote waterfalls and execution metrics (#3786)

typed-secrets: the accessor adapter reads a legacy row stamped with the empty namespace as foreign, so its tagged fields stay reference text on the typed path

Registry scorecard: score secret-safe-docs (rubric v4) and keep sensitive on extracted arguments

Share the extension quality rubric between the swamp CLI and swamp-club instead of mirroring it

Registry dependency audit: audit the locked version push records in the archive, not the registry's newest match for a range specifier (swamp-club half of #3662)

@swamp/cve/dirtyfrag: report and README print the old swamp model command spelling

@swamp/cve/php-rce-2026-10: match RHEL fixes for the PHP RCEs

cve/emudhra page: Windows and macOS support, and shared CVE layer fixes (follow-up to #3710 and #3766)

Docs: extra-headers how-to systemd snippet puts a path in SWAMP_SERVE_EXTRA_HEADERS

CVE pages: horizontally scrolling code blocks are not keyboard-reachable

typed-secrets 4.6: turn the typed path on for every entry point, remove the internal option, and delete the regular-expression rewrite, the value search and the plaintext resolution of tagged fields

typed-secrets 4.5: the inline resolutions refuse or mask: placement, task selectors, child references and deferred targets refuse secrets, assert messages mask them, and the deprecated resolvers go

typed-secrets 4.4: persistence and resume hold references: rehydration stops reading the vault, every persisted copy uses the persisted form, and run events are masked

Docs: extension fmt never writes deno.lock and warns on a stale one (swamp-club#3717)

Docs: extension manual pages describe push and quality resolving only the deno.json import map

Server-driven warning and info messages to CLI users, targeted by swamp version (exploration)

Serve: delegate namespace-scoped extension pulls and reload without granting administrator access

workflow cancel --server ignores --ca-cert and SWAMP_CA_CERT

Preserve active worker execution across serve restarts and upgrades; reconnect and reattach dispatches

v3: chunk durability barrier costs a full drive-cache flush per command on macOS; use an ordering barrier

vary/forEach object values with no key/name/id produce data names Windows refuses

issue-lifecycle start drops issue comments: context.comments is always empty

swamp core: a non-string sensitive value reads back as a JSON string

Generated vault keys carry raw data-name and vary characters that strict vault providers reject

bundleExtension 'bundles on different caches do not wait for each other' test fails under CPU load

extension install and extension pull leave different bundles for the same extension version

Registry drops contentMetadata.extensions (method grafts) at confirm-push

Multi-model skill eval: gpt-5.4 and gemini jobs fail preflight every run (OpenAI credits exhausted, Google API key invalid)

local_source_failed drops the indexing error: a transient local extension failure leaves no cause (follow-up to 1672)

Finer-grained method locking: a long-running method on one definition blocks unrelated objects managed by the same definition

Surface (or warn about) values a model method returns besides dataHandles, with sensitive-field support

workflow placement: run a workflow transparently on a specified host, local or via swamp workers (incl. nested child workflows)

issue-lifecycle: pr_merged and ship report success and advance the phase when swamp-club is unreachable, writing nothing upstream

auth login leaves auth.json apiKey empty and stores the key only in ~/.config/swamp/api-key, which the auth gate never reads

swamp-extensions CI: Review Integrity explains a stalled or failed audit (streamed progress, timeout, did-not-finish comment)

Workflow run cancellation on concurrency-key conflict

Read non-string sensitive resource fields back with their type

Run index repair-by-stat misses an in-place, same-size record rewrite that restores mtime within the same second as its entry: ctime is reported in whole seconds

Configurable storage tiers: runs, outputs, data versions and audit move through an ordered list of user-defined tiers, and stay queryable across them

A global argument that evaluates to swamp-namespace expression text is silently dropped, and literal() cannot pass it through

serve: the per-IP WebSocket burst limit can't be configured, so workers behind one NAT IP throttle each other

extension pull: safety analysis scans the bundle the extension runs

verification: run the upgrade-path test in parallel, and skip the dynamic test for no-op bumps

S3-backed serve: a poll that pulls one file invalidates the whole data catalog, so the next query re-reads every version

swamp-extensions: CI check that every manifest declares license: and ships a LICENSE file

extensions_search index is never created if the first attempt fails, until the process restarts

S3/GCS datastore sync: shared sync service lets one sync clear another's dirty marks and pull over unpushed files

swamp-uat: the SIGTERM drain test does not check that the run succeeded

stagecraft: a factory-wide read of work items waiting on a person (no studio, no per-item status loop)

stagecraft: rebind a work item to another tracker (mirrored built-in tickets to Jira)

s3-datastore: first push on a fresh cache prints 'Index merge failed' for a missing local .datastore-index.json

swamp-extensions: use explicit npm:/jsr: specifiers in root extensions/models

stagecraft: studio state_test 'after its event stream reconnects' is timing-flaky under load

Activity projector fails with ZooKeeper "Bad version" error on swamp.score_grants during keeper instability

s3-datastore: model-scoped pullChanged lists and walks the whole namespace

extension safety analyzer does not warn on node:child_process or cluster subprocess spawning

extension push: a swamp-quality-ignore directive after a comment holding an apostrophe is silently ignored

Model types: decide whether names like @/@x/y or a+b/c should be legal (type grammar)

seed:collective cannot produce a superseded personal trial or a returned trial, so the UAT cannot cover them

serve HA: pull a token's definition on an auth miss so a peer accepts a new token at once

extension push: accept attestation-backed adversarial-review evidence (lookup by extension and content hash), generated provenance, and carry the evidence to the registry at confirm (design: Lab #3023)

Decide a stricter naming rule for workflow step and job names

s3-datastore/serve: readResource without a version returns an older version; the latest marker is rewritten backwards

Publish agent-facing projections of the manual: /llms.txt, .md pages, and llms-full.txt

Let agent-runner drive more agent CLIs (Kilo Code and others)

s3-datastore: every fast-path miss re-downloads every _index shard, so a busy serve pulls the whole index every poll (~230 GB/day S3 egress)

Feedback: Swamp as an approval-gated control plane for a small fleet

S3/GCS datastore: a push that fails after uploading drops its recorded deletes, so the retry brings deleted data back

Tracking: test baseline required before the datastore refactor (commit-log design)

serve: define and supply the collective and owner grant condition variables

No first-class workflow primitive for spawning a permission-scoped agent session

Decorative animations outside the Lab still repaint on the main thread

s3/gcs datastore: pullChanged overwrites dirty, unpushed cache files

Codegen/AWS: create-only required fields block list/get/delete in generated AWS models

End a collective's trial when it begins a paid subscription

s3-datastore: SWAMP_S3_REQUEST_TIMEOUT_MS does not appear to apply to ListObjectsV2 during pull

s3-datastore: S3Lock.acquire overshoots maxWaitMs by up to a full backoff interval

s3-datastore: a model-scoped pull still lists, walks and indexes the whole namespace