Skip to main content
← Back to list
01Issue
FeatureOpenSwamp CLIPublic
AssigneesNone

Relationships

#4943 Store large values once: content-addressed sharing across resources and versions

Opened by zack · 11/20/2026

Problem

Behaviors that snapshot state naturally repeat the same large values: the same multi-MB string embedded in a state resource, in every history entry, in a handoff copy and in each new version. Swamp stores and reloads every copy independently. In our case about 5 MB of distinct data was stored as several hundred MB per version, because one 3.6 MB text value appeared dozens of times across six resources. Storage, query output and parse memory all scale with the number of copies, not with the distinct data.

Proposal

  • Store string (or subtree) values above a size threshold out of line, keyed by content hash, shared across all resources and versions of a repository's data, with garbage collection through the existing swamp data gc.
  • On read, either return a shared single instance per distinct value within a run, or return a handle that can be resolved or streamed on demand (combining well with partial reads from the feature above).
  • Keep this transparent to existing behaviors: written values round-trip unchanged.

Expected outcome

Storage and memory scale with distinct data. Redundant snapshots stay cheap, so a design mistake like ours degrades gracefully instead of making a lifecycle unrecoverable.

02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

11/20/2026, 3:12:32 PM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.