Skip to main content
← Back to list
01Issue
FeatureOpenSwamp CLIPublic
AssigneesNone

Relationships

#4964 Streaming resource writes from behaviors with no single-string size ceiling

Opened by zack · 11/20/2026

Problem

context.writeResource serializes the whole object with JSON.stringify (or the redaction path) into one string, sometimes re-parses it, and writes it with writeText. The handle also keeps a copy of the data as attributes. Memory scales with the full resource, and a resource larger than the V8 maximum string length (about 512 MiB) cannot be written at all.

Proposal

  • An incremental JSON serializer that writes plain values exactly as JSON.stringify does, writes an AsyncIterable or a non-array Iterable as an array one element at a time, flushes buffered chunks, and respects backpressure.
  • context.writeResourceStream(specName, name, value, overrides?). The tag, lifetime, vary-suffix and name-claim resolution moves out of writeClaimedResource into shared code, so both write paths stamp identical records. Bytes go through DefaultDataWriter.writeStream. The handle carries size but no attributes.
  • Schema validation stays warn-only: the skeleton is checked with streamed arrays empty, and each element against the element schema.
  • The run's redactor is applied per element, with a defined maximum size for one string or windowed redaction.
  • Specs with sensitive fields or sensitiveOutput are refused for now; per-element vaulting is a follow-up.
  • The test context in packages/testing gets writeResourceStream, materialized into the captured resources.

Expected outcome

A behavior can write a resource of any size in bounded memory. Existing writeResource behavior is unchanged.

Context

Split out of swamp-club#4942. Needs swamp-club#4962, so the end of a workflow step does not load the streamed resource whole. Worker support is a separate issue.

02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

11/20/2026, 7:35:29 PM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.