Relationships
#3706 data query: filter on data lifecycle so deleted resources can be excluded
Opened by zack · 10/21/2026· Shipped 10/21/2026
Summary
data query predicates, data search, and CEL data.query() cannot tell a deleted resource from a live one. Data already carries a lifecycle (active / deleted) in each version's metadata.yaml and on the Data entity, but the catalog has no lifecycle column and DataRecord has no lifecycle field, so no predicate can say "only resources that still exist".
Why now
swamp-club#3698 makes the catalog rebuild index deletion markers the same way write-through already does. Before that fix, a rebuilt catalog (multi-checkout S3/GCS datastores rebuild after every pull) hid deleted resources by accident; after it, queries such as modelType == "@swamp/hetzner-cloud/servers" consistently include resources a kind: delete method has deleted. Users who relied on the accidental filtering need a reliable way to exclude them.
Current workaround and its problems
!has(attributes.deletedAt) works because a delete method writes deletedAt into the marker's content, but:
- it is not a contract — a resource whose real state has its own
deletedAtis wrongly excluded; - filtering on
attributesloads each matched record's content, which on a lazy-hydration datastore downloads bodies just to evaluate the filter; - it is undocumented.
Proposal
- Add a
lifecyclecolumn to the catalog, filled fromData.lifecycleon write-through, full and scoped backfill, and in the v3 catalog projection. - Expose it on
DataRecordaslifecycle("active"/"deleted") solifecycle == "active"is a metadata-only predicate. - Opt-in: default query results do not change.
- Document it in the manual (
reference/data.md, Lifecycle States, and the data query reference).
Things to handle
- Catalog migration: existing rows have no value, and defaulting them to
activewould mislabel existing deletion markers — the migration must invalidate and rebuild the catalog. - Foreign-namespace catalog exports written before the column exists will lack it (as with
garbageCollection, which uses an empty value when unknown). - Rename markers stay out of catalog rows (the rename forward projection owns them), so
deletedrows are deletion markers only.
Related: swamp-club#3698.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.