Skip to main content
← Back to list
01Issue
BugShippedSwamp CLITeam
Assigneesstack72

Relationships

#1398 workflow/job-lifetime data is never reclaimed — expiry check requires an ownerDefinition.workflowId that step execution never sets

Opened by magistr · 7/24/2026· Shipped 7/25/2026

Summary

Data declared with lifetime: "workflow" or lifetime: "job" is never reclaimed. The expiry check requires both workflowId and workflowRunId on the data's ownerDefinition, but data produced by workflow step execution carries workflowRunId and workflowName — never workflowId. The check therefore takes its early-return branch, logs a warning, and reports the data as not expired, forever.

The documented contract is the opposite: /manual/explanation/data-lifetimes states that "a workflow-scoped artifact survives until the entire run ends", that "a job-scoped artifact is garbage-collected when its job completes, even if the workflow continues", and that "Once the workflow run is cleaned up, the data goes with it."

Verified reproduction

Throwaway repo, default (local) datastore. A model with a workflow-lifetime resource:

export const model = {
  type: "@test/producer",
  version: "2026.07.25.1",
  globalArguments: z.object({}),
  resources: {
    wfdata: {
      description: "Data that lives until the workflow completes",
      schema: z.object({ payload: z.string(), timestamp: z.string() }),
      lifetime: "workflow",
      garbageCollection: 50,
    },
  },
  methods: {
    produce: {
      arguments: z.object({}),
      execute: async (_args, context) => ({
        dataHandles: [await context.writeResource("wfdata", "main", {
          payload: "IMPORTANT-WORKFLOW-DATA",
          timestamp: new Date().toISOString(),
        })],
      }),
    },
  },
};

driven by a one-step workflow that calls produce. After swamp workflow run wf1, swamp data get prod1 main --json shows "lifetime": "workflow" and this owner definition — note what is present and what is missing:

"ownerDefinition": {
  "ownerType": "model-method",
  "ownerRef": "f584d5c2-…",
  "workflowRunId": "1e214e44-…",
  "workflowName": "wf1",
  "jobName": "main",
  "stepName": "produce",
  "source": "step-output"
}

Now delete the owning workflow run and try to collect:

swamp run gc --older-than 0d --json
# -> { "workflowRunsDeleted": 1, "outputsDeleted": 1, … }

swamp data gc --json
# -> { "dataEntriesExpired": 0, "versionsDeleted": 0, "bytesReclaimed": 0, "expiredEntries": [] }

swamp data get prod1 main --json
# -> still present, content intact

The run it was scoped to is gone; the data remains. Running with warnings visible names the branch taken:

swamp data gc --log-level warning
# [WRN] swamp·domain·data·lifecycle: Data '"main"' has "workflow" lifetime but missing workflowId or workflowRunId
# Nothing to clean up.

For contrast, the same swamp data gc before the run was deleted also reported zero — so this is not "collected early/late", it is never collected at all.

Cause

src/domain/data/data_lifecycle_service[dot]ts, in isExpired:

if (data.lifetime === "workflow" || data.lifetime === "job") {
  const workflowId = [HOST-1];
  const workflowRunId = [HOST-2];

  if (!workflowId || !workflowRunId) {
    logger.warn("Data '{dataName}' has {lifetime} lifetime but missing workflowId or workflowRunId", …);
    return false;                       // ← always taken for step-output data
  }
  const workflowRun = await [HOST-3](
    createWorkflowId(workflowId), createWorkflowRunId(workflowRunId),
  );
  return workflowRun === null;          // ← the intended logic, unreachable
}

workflowId is optional in OwnerDefinitionSchema (src/domain/data/data_metadata[dot]ts:107), and the workflow step path does not set it. The tag overrides built at src/domain/workflows/execution_service[dot]ts:1006-1013 populate source, workflow (the name), workflowRunId, job and step — there is no workflowId among them, which matches the record above exactly.

So the lookup that would decide expiry is never reached, and the branch that runs unconditionally returns "not expired".

Impact

  • workflow/job lifetimes are silently inert. Every artifact written with them accumulates forever, in every repo, regardless of data gc, retention settings, or how long ago the run finished. Users choosing these lifetimes specifically to avoid unbounded growth get unbounded growth.
  • It is invisible unless you raise the log level. Default output is Nothing to clean up. — indistinguishable from "there was nothing to collect". The warning that explains it is suppressed at the default level.
  • The cost lands on the datastore. For remote datastores this is paid storage and sync time for data the user believes was reclaimed at run end.
  • Worth noting the failure direction is safe — data is retained rather than deleted — so this is a leak and a contract violation, not data loss.

Suggested fix

Either populate workflowId on the owner definition when a step writes data (the execution context has workflow.id available — it is already used at execution_service[dot]ts:2049 and elsewhere), or change the expiry check to resolve the run by workflowRunId alone, or by workflowName + workflowRunId, which is what the record actually carries.

Whichever is chosen, the warning at that branch is a good signal that something is structurally wrong and should probably be an error or a test assertion rather than a debug-level breadcrumb — it currently fires on every data gc in any repo that has ever run a workflow.

For coverage: a test that writes workflow-lifetime data through a real workflow run, deletes the run, and asserts the data is collected would catch this. The existing lifecycle tests construct ownerDefinition by hand and can supply workflowId, so they do not exercise the shape production actually writes.

Environment

  • Source main @ 10943ef, run from source. macOS 14, default local datastore.
  • Found while testing an unrelated hypothesis about run gc making data collectible — that hypothesis was disproven precisely because of this bug.
02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPED+ 1 MOREASSIGNED+ 2 MOREREVIEW+ 4 MOREPR_MERGED+ 1 MORECONTRIBUTOR_NOTIFIED

Shipped

7/25/2026, 12:02:17 AM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack727/24/2026, 11:14:18 PM
Editable. Press Enter to edit.

magistr commented 7/24/2026, 11:09:35 PM

Redaction repair — the redactor read three dotted expressions as hostnames and replaced them. The quoted block from the expiry check should read, in words:

  1. The first two lines read the workflow id and the workflow run id off the data's owner-definition object (the fields named workflowId and workflowRunId on data dot ownerDefinition).
  2. The guard is: if EITHER is falsy, log the warning and return false — not expired. This is the branch that always runs, because step-output data has no workflowId.
  3. The line after the guard awaits a findById call on the workflow-run repository, passing both ids; its result decides expiry (a null run means expired). That call is unreachable for step-output data.

The point stands unchanged: the intended logic is fine, the guard in front of it is unsatisfiable for the shape of record the workflow engine actually writes.

stack72 commented 7/25/2026, 12:02:27 AM

Thanks @magistr for reporting this! The fix has been merged and a release is on its way. We appreciate your contribution to swamp.

Sign in to post a ripple.