Skip to main content
← Back to list
01Issue
BugShippedSwamp CLIPublic
AssigneesNone

Relationships

↔ sibling #3179

#3178 Lazy hydration: content readers bypass hydration (stream, stale raw after append); add one ensuring read path and a fitness test

Opened by zack · 10/8/2026· Shipped 10/8/2026

Summary

Under hydrationStrategy: lazy the local cache can hold a data item's metadata without its content. Readers are repaired one at a time (getContent has a hydrate hook, data.latest() materialises the path), and there is no list of them, so some still assume the content is local. Found while working swamp-club#2435, which is blocked on this.

Readers that are wrong today

  • FileSystemUnifiedDataRepository.stream() (src/infrastructure/persistence/unified_data_repository.ts) catches NotFound and yields nothing, without calling the hydrate hook. The serve data plane (src/serve/data_plane.ts) streams artifacts to remote workers through it, so a worker gets an empty or truncated body for content that is not local.
  • A raw file that exists locally is never checked against its metadata. append() rewrites the latest version's raw at the same path and updates size in metadata.yaml. A metadata-only pull brings the new metadata and skips the raw (the S3 and GCS extensions skip every lazily skippable file, whether or not a local copy exists), so the old bytes stay and are served as current.

Proposed change

  1. One content-ensuring step in the repository: the file is present and matches the size in its metadata, or it is downloaded through the hydrate hook. getContent, stream and path materialisation all go through it.
  2. An architecture fitness test in integration/ that pins the list of direct readers of content paths and of getContentSync callers, using the ratchet pattern in arch_fitness_helpers.ts, so a new reader that bypasses the step fails a test instead of waiting for a review to find it.

Notes

  • A fix for stream() with a unit test exists on the issue-2435 branch (commit ac58584a) and can be reused.
  • Not verified: every place a content file path is handed to extension code, which may read it directly.
02Bog Flow
✓OPEN✓TRIAGED✓IN PROGRESS✓SHIPPEDLINKED+ 4 MOREPR_MERGED+ 1 MORESESSION_SUMMARIZED

Shipped

10/8/2026, 8:30:22 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 linked sibling of #317910/8/2026, 6:47:10 PM

Sign in to post a ripple.