Relationships
#4965 Streaming resource writes from workers through the serve data plane
Opened by zack · 11/20/2026
Problem
A behavior placed on a worker writes resources with DataPlaneClient.writeResource, which posts the whole object as one JSON body to POST /data/resource. Serve reads it with req.json(). The worker-side and serve-side memory both scale with the full resource, and the single-string ceiling applies on both ends.
Proposal
DataPlaneClient.writeResourceStream(meta, stream)posts to a new route,POST /data/resource-stream. The metadata (specName, name, overrides) goes in a JSON header and the body streams.- Serve gives the route the same authorization, dispatch tracking (
recordWrite,beginWrite,recordVersion) and spec checks asPOST /data/resource. The specName and name it authorizes are exactly the ones written. - The body is piped through a JSON stream rewriter: it requires a top-level object, enforces a depth limit, a maximum string token size and the same total-size policy as file-writer content uploads, and redacts with serve's redactor before the serve-side streaming resource writer.
- A refused upload rolls back its allocated version. Errors returned to the worker are generic; parser offsets are logged at serve only.
- The remote method context wires
context.writeResourceStream.
Expected outcome
Worker-placed behaviors get the same bounded-memory streaming writes as local runs. An integration test against a port-0 serve covers a successful upload and a malformed upload that leaves no version.
Context
Split out of swamp-club#4942. Depends on swamp-club#4964 (the streaming writer and resource write path) and the JSON tokenizer from swamp-club#4942. Adds a serve write endpoint, so it falls under the CI security review.
Open
No activity in this phase yet.
Sign in to post a ripple.