Relationships
#2503 data.get returns binary file content as a lossily-decoded UTF-8 string, corrupting the bytes
Opened by adam · 9/24/2026· Shipped 9/24/2026
Summary
data.get returns the content field for a file artifact as a lossily-decoded
UTF-8 string. For a binary file (an image, an archive, anything non-text) the
bytes are irrecoverably mangled, so a consumer reading over swamp serve — or
via swamp data get --json — cannot reconstruct the file. The bytes on disk are
correct; only the retrieval path is lossy.
There is no other documented retrieval path for binary content over the data API, so a model that stores a binary file has no way to hand it to a remote reader.
Reproduction
A model declaring a files spec, written with
context.createFileWriter("logo", name, { contentType: "image/png" }).writeAll(bytes),
where bytes is a 67-byte 1x1 PNG.
$ swamp data get competitive-analysis logo-acme-widgets-inc --jsonThe stored file itself is byte-perfect:
$ file .swamp/data/.../logo-acme-widgets-inc/1/raw
... PNG image data, 1 x 1, 8-bit/color RGBA, non-interlaced
$ cmp .swamp/data/.../raw original.png # identicalBut the JSON envelope disagrees with it:
| Field | Value |
|---|---|
size |
67 (correct) |
checksum |
matches the real file (correct) |
contentType |
image/png (correct) |
len(content) |
65 — two bytes lost |
content begins �PNG\r\n\x1a\n... — the 0x89 lead byte has become
U+FFFD, and every other non-UTF-8 byte is likewise replaced. Base64-encoding
the returned content does not recover the original; the loss is upstream.
Expected
One of:
contentis omitted (ornull) for non-text content types, with an explicit field saying so, and a separate documented way to fetch bytes; or- a documented
encodingfield onDataGetPayload(e.g.encoding: "base64") so binary content can be requested losslessly.
Silently returning a corrupted string that has the right size and checksum
sitting next to it is the worst of the three, because the envelope looks valid.
Actual
content is a mangled UTF-8 string, with no indication the value is lossy.
Notes
references/data/references/fields.md documents content as "Empty "" for
binary types. Available for text/*, application/json, application/yaml." The
observed behaviour does not match the documentation — the field is neither
empty nor withheld, it is populated with corrupted data. Whichever way this is
resolved, the docs and the behaviour currently disagree.
Impact
Any consumer over swamp serve that stores a binary artifact (images, PDFs,
archives) cannot read it back. The workaround is to write the file twice — once
as real bytes, and once base64-encoded into a text/plain sidecar file that
survives the round trip — which doubles storage and makes the binary artifact
decorative for remote readers.
Environment
- swamp
20260923.160132.0-sha.a06e8ad6 - repo
swampVersion: 20260901.200207.0 - Linux x86_64
- Read both via
swamp data get --json(local) and thedata.getrequest over theswamp serveWebSocket; both exhibit it.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.