Skip to main content
← Back to list
01Issue
FeatureShippedSwamp CLIPublic
Assigneeszack

Relationships

#3706 data query: filter on data lifecycle so deleted resources can be excluded

Opened by zack · 10/21/2026· Shipped 10/21/2026

Summary

data query predicates, data search, and CEL data.query() cannot tell a deleted resource from a live one. Data already carries a lifecycle (active / deleted) in each version's metadata.yaml and on the Data entity, but the catalog has no lifecycle column and DataRecord has no lifecycle field, so no predicate can say "only resources that still exist".

Why now

swamp-club#3698 makes the catalog rebuild index deletion markers the same way write-through already does. Before that fix, a rebuilt catalog (multi-checkout S3/GCS datastores rebuild after every pull) hid deleted resources by accident; after it, queries such as modelType == "@swamp/hetzner-cloud/servers" consistently include resources a kind: delete method has deleted. Users who relied on the accidental filtering need a reliable way to exclude them.

Current workaround and its problems

!has(attributes.deletedAt) works because a delete method writes deletedAt into the marker's content, but:

  • it is not a contract — a resource whose real state has its own deletedAt is wrongly excluded;
  • filtering on attributes loads each matched record's content, which on a lazy-hydration datastore downloads bodies just to evaluate the filter;
  • it is undocumented.

Proposal

  • Add a lifecycle column to the catalog, filled from Data.lifecycle on write-through, full and scoped backfill, and in the v3 catalog projection.
  • Expose it on DataRecord as lifecycle ("active" / "deleted") so lifecycle == "active" is a metadata-only predicate.
  • Opt-in: default query results do not change.
  • Document it in the manual (reference/data.md, Lifecycle States, and the data query reference).

Things to handle

  • Catalog migration: existing rows have no value, and defaulting them to active would mislabel existing deletion markers — the migration must invalidate and rebuild the catalog.
  • Foreign-namespace catalog exports written before the column exists will lack it (as with garbageCollection, which uses an empty value when unknown).
  • Rename markers stay out of catalog rows (the rename forward projection owns them), so deleted rows are deletion markers only.

Related: swamp-club#3698.

02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 10 MOREPR_LINKED+ 2 MORESESSION_SUMMARIZED

Shipped

10/21/2026, 4:34:48 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
zack assigned zack10/21/2026, 2:13:04 PM

Sign in to post a ripple.