Skip to main content
← Back to list
01Issue
BugTriagedExtensionsPublic
Assigneesskunk-ape

Relationships

#4411 S3 and GCS pullFile truncates the local cache file in place, so an aborted or concurrent pull damages a live run record

Opened by skunk-ape · 11/4/2026

Problem

pullFile in the S3 and GCS datastore extensions opens the local cache file with truncate: true and streams the download into it. There is no temp file and no rename. While a download runs, the local file holds only part of its content, and an abort leaves that part behind for good.

The window has been wide since the streaming change of 2026-09-22 (swamp-extensions commit 3571a406d). Before it, the body was buffered first and written in one call.

A pull also replaces a local file whose size differs from the remote index entry. That includes the record of a workflow run still in progress, so a pull can put an older copy over a newer local record.

Evidence

  • Stubbed S3 client, abort mid-download, on versions 2026.09.24.1 and 2026.11.03.1: a 305,599-byte local record was left as the first 262,144 bytes of the remote copy.
  • Real run against MinIO on swamp 20261104.203255.0-sha.3ff9a703 with S3 extension 2026.11.03.1: 40 uncapped forEach iterations, while a second repo pushed to the same bucket and a second process ran swamp datastore sync --pull. inotify recorded an in-place modify of the live run record, and the record went from 184,488 to 7,066 bytes. The next save restored it and the run succeeded.
  • GCS has the same pullFile code. This was confirmed by reading it, not by running it.

Expected

  • A pull writes to a temp file beside the target and renames it over the target, so a reader or an abort never sees a partial file.
  • A pull does not put an older remote copy over a local run record that has newer unpushed writes. The existing guard for marked paths applies only to subdir-scoped pulls.

Context

Found while diagnosing swamp-club#2517, where a truncated run record left a run permanently unreadable. The record damage there is not proven to come from this path. The core handling of a damaged record is fixed under #2517.

02Bog Flow
✓OPEN◉TRIAGED○IN PROGRESS○SHIPPED+ 1 MOREASSIGNED+ 2 MOREREVIEW

Triaged

11/5/2026, 12:00:39 AM

Click a lifecycle step above to view its details.

03Sludge Pulse
skunk-ape assigned skunk-ape11/5/2026, 12:00:06 AM
Editable. Press Enter to edit.

system commented 11/4/2026, 11:30:02 PM

Classified automatically when this issue was filed.

  • Source: Extensions

If you feel this classification is incorrect, add a ripple to tell us so.

Sign in to post a ripple.