Skip to main content
← Back to list
01Issue
FeatureOpenSwamp CLIPublic
AssigneesNone

Relationships

#4965 Streaming resource writes from workers through the serve data plane

Opened by zack · 11/20/2026

Problem

A behavior placed on a worker writes resources with DataPlaneClient.writeResource, which posts the whole object as one JSON body to POST /data/resource. Serve reads it with req.json(). The worker-side and serve-side memory both scale with the full resource, and the single-string ceiling applies on both ends.

Proposal

  • DataPlaneClient.writeResourceStream(meta, stream) posts to a new route, POST /data/resource-stream. The metadata (specName, name, overrides) goes in a JSON header and the body streams.
  • Serve gives the route the same authorization, dispatch tracking (recordWrite, beginWrite, recordVersion) and spec checks as POST /data/resource. The specName and name it authorizes are exactly the ones written.
  • The body is piped through a JSON stream rewriter: it requires a top-level object, enforces a depth limit, a maximum string token size and the same total-size policy as file-writer content uploads, and redacts with serve's redactor before the serve-side streaming resource writer.
  • A refused upload rolls back its allocated version. Errors returned to the worker are generic; parser offsets are logged at serve only.
  • The remote method context wires context.writeResourceStream.

Expected outcome

Worker-placed behaviors get the same bounded-memory streaming writes as local runs. An integration test against a port-0 serve covers a successful upload and a malformed upload that leaves no version.

Context

Split out of swamp-club#4942. Depends on swamp-club#4964 (the streaming writer and resource write path) and the JSON tokenizer from swamp-club#4942. Adds a serve write endpoint, so it falls under the CI security review.

02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

11/20/2026, 7:35:39 PM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.