Relationships
#1430 writeResource commits per-call: a method that fails midway leaves its earlier outputs persisted, with no marker that the run failed
Opened by vcjdeboer · 7/27/2026
A method's outputs commit one at a time. If a method writes resource first
and then throws before writing second, first stays persisted at version 1
and the method reports failure. There is no rollback and no transaction
boundary around a method execution.
This is a single model, a single execution, and two of its own declared
resource specs — no runModel(), no cross-model access, no extensions beyond
the probe below.
Environment
- swamp
20260727.000815.0-sha.db74fcac - macOS 15 (darwin-aarch64), bundled deno 2.8.3
- Local extension source,
swamp model method runfrom the CLI
Reproduce
A model type with two declared resource specs and two methods:
resources: {
first: { schema: MarkerSchema, lifetime: "infinite", garbageCollection: 20 },
second: { schema: MarkerSchema, lifetime: "infinite", garbageCollection: 20 },
},
methods: {
// control: two writes from one method is ordinary usage
both: {
execute: async (_args, ctx) => {
const a = await ctx.writeResource("first", "a", { marker: "from-both", writtenAt: new Date().toISOString() });
const b = await ctx.writeResource("second", "b", { marker: "from-both", writtenAt: new Date().toISOString() });
return { dataHandles: [a, b] };
},
},
// the case
partial: {
execute: async (_args, ctx) => {
await ctx.writeResource("first", "a", { marker: "from-partial", writtenAt: new Date().toISOString() });
throw new Error("deliberate failure after the first write");
},
},
}swamp model create @me/atomicity-probe atom
swamp model method run atom both # control
swamp model create @me/atomicity-probe atom3
swamp model method run atom3 partial # a fresh definition, no prior data
swamp data list atom3Observed
The control behaves as expected — both succeeds and writes a and b, each
at v1.
partial fails:
wrote first; about to fail before writing second
Error: deliberate failure after the first writeand first survives it:
$ swamp data list atom3
resource (1 item):
a v1 application/json 64B 2026-07-27
$ swamp data get atom3 a --json
{"name":"a","version":1,"content":{"marker":"from-partial","writtenAt":"2026-07-27T15:38:13.791Z"}}
$ swamp data get atom3 b --json
{ "error": "Data not found: \"b\" for model \"atom3\"" }Expected
Either all of a method execution's outputs commit, or none of them do. A method that fails should leave the model's recorded state as it was.
Why this is worse than a partial write
The surviving artifact carries no trace of the failed run. Its full record is indistinguishable from one written by a method that completed:
{
"name": "a",
"version": 1,
"tags": { "type": "resource", "specName": "first", "modelName": "atom3" },
"ownerDefinition": { "ownerType": "model-method", "ownerRef": "a27ee7ab-…" },
"createdAt": "2026-07-27T15:38:13.793Z"
}There is no run-outcome field anywhere on it. @swamp/method-summary does
record {"status": "failed", "method": "partial"}, so the failure is not lost —
but it lives in a separate report, and correlating an artifact with it means
matching by definition and timestamp.
A downstream workflow step reading data.latest("atom3", "a") gets the
half-written value and has no way to know. So the failure is silent at exactly
the layer that consumes the data.
Why I hit it
I am building lab instrument drivers as swamp model types — currently a plate reader over its USB-FTDI bridge.
Instruments produce many readings per operation, and since the write lock scopes
to the model definition (two method run invocations against one definition
serialise), the sensible shape is a single method that emits many outputs rather
than many method runs. A 96-well read is one execution writing 96 resources.
That method failing at well 60 is this bug: 59 readings persist, nothing marks
them as partial, and the next step reading data.latest() cannot tell. The more
faithfully I follow the pattern the lock pushes me toward, the more outputs each
failure can strand.
Closed
No activity in this phase yet.
stack72 commented 7/29/2026, 9:26:20 PM
This issue drove a significant design exploration that landed on three features, all now shipped or filed:
Shipped:
- guard (#1437, shipped in #1993) — CEL expression on workflow steps for idempotent execution. Truthy skips the step, falsy executes. Supports data truthiness checks, value comparisons, and model.method() calls for external state verification.
- resume --from (#1444) — re-enters a failed workflow's DAG at a specified step. Guards decide which steps re-execute. Targets template names for forEach steps.
Filed:
- rollbackOnFailure (#1451) — opt-in method flag for deferred-latest write mode. Writes skip the latest marker during execution, advance on success, delete on failure. No TOCTOU. This is the direct fix for the per-call commit behavior described in this issue.
- Docs (#1446) — document guard and resume --from in the workflows manual reference.
The design journey: we evaluated rollback (rejected — can't undo real-world effects, CloudFormation's mistake), pending operations with marker rewind (rejected — hides real data, shows stale state), and deferred commit for all methods (rejected — breaks read-after-write). The final design separates concerns: the data layer reflects reality, the workflow layer handles safety via guards, and rollbackOnFailure is an opt-in for methods where partial writes are genuinely garbage.
@vcjdeboer — for your plate reader use case, the combination of guard on workflow steps (to prevent re-reading wells that already have data) and rollbackOnFailure on the read method (to discard incomplete well sets on failure) should address both the downstream consumption problem and the partial-write cleanup. The rollbackOnFailure implementation is tracked in #1451.
Sign in to post a ripple.