Skip to main content
← Back to list
01Issue
BugOpenSwamp CLIPublic
AssigneesNone

Relationships

#4962 Workflow steps parse every written resource in full, and past the V8 string limit their outputs are silently empty

Opened by zack · 11/20/2026

Problem

After every workflow step, the execution service builds the step's resource records by calling fromResourceHandle (src/domain/data/data_record_mapper.ts, called from src/domain/workflows/execution_service.ts) for each resource the step wrote. It reads the whole content with getContent, decodes it to one string and runs JSON.parse to fill attributes, which liveStepOutputs uses to build the step outputs.

Two consequences:

  • Memory: the orchestrator holds the full parsed size of every resource a step wrote when the step ends, even when the method itself used bounded memory. A 390 MB resource is roughly 1 to 2 GB once parsed.
  • Silent data loss: for content past the V8 maximum string length, the decode throws into a bare catch ("Not valid JSON, skip attributes"). The record gets empty attributes, so the step outputs and the expressions that read them are missing with no error or warning.

Expected

The step-end path has a bounded memory cost. For a resource too large to load as attributes, the run records that clearly (a warning, and a defined shape for the step outputs) instead of silently producing empty outputs. Expressions that need such a resource read it from storage.

Context

Split out of swamp-club#4942 (bounded-memory access to large resources). This blocks streaming resource writes from being useful inside workflows.

02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

11/20/2026, 7:34:45 PM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.