flaky: LitestreamReplicator 'a failed launch releases its lock probe, so the next acquire starts a sidecar' fails by timing in verify-build
typed-secrets 4.4: persistence and resume hold references: rehydration stops reading the vault, every persisted copy uses the persisted form, and run events are masked
data_writer_property_test: a secret holding a newline and a literal backslash-n escapes the persisted-content property (seed 1648929202)
typed-secrets: the accessor adapter reads a legacy row stamped with the empty namespace as foreign, so its tagged fields stay reference text on the typed path
typed-secrets query view: route a row's JSON content by whether the adapter adapted the record, not by the service's namespace rule
flaky: for_each_name_property_test lets the text around a sentinel equal the secret's value, so the untyped value search replaces both and the property fails by seed
typed-secrets 4.3: the expression-facing query view registers Secret in DataQueryService
CEL duration arithmetic is off by a second when a duration has negative nanos (cel-js 7.6.1)
typed-secrets 4.2: live and reconstructed step outputs and child outputs return references through the accessor adapter
A coded UserError a model method throws reaches a workflow step without its code, because InProcessExecutor keeps only the message
Dashboard Workflows page takes ~19s on a serve with many extension workflows: workflow.search resolves each workflow's owner with a full extension-workflow scan
codegen: BROKEN_REGEXES override for the @swamp/aws/invoicing InvoiceUnit name lacks the u flag, so it rejects ordinary names
dashboard: run page polling stacks history.get and history.logs requests while serve is slow
serve: add child spans and attributes to workflow.history.get (and run_summary/run_search) to diagnose slow dashboard loads
serve: webhook-triggered workflow runs can't be cancelled — cancel can't find them, so it reports serve's own pid as "the process running it is still alive"
A workflow step fails on a static validation error in a method it does not invoke, where swamp behavior run of the same method succeeds
serve: a run orphaned by a container restart can't be cancelled — the restarted serve reuses the dead owner's pid, so cancel reports the owner still alive
typed-secrets 4.6: turn the typed path on for every entry point, remove the internal option, and delete the regular-expression rewrite, the value search and the plaintext resolution of tagged fields
typed-secrets 4.5: the inline resolutions refuse or mask: placement, task selectors, child references and deferred targets refuse secrets, assert messages mask them, and the deprecated resolvers go
typed-secrets 4.1: workflow passes bind sentinels as Secrets: typed guards and asserts, forEach items and identities, child bindings, webhook inputs, and the synchronous authorization step
typed-secrets 3.5: local entry points, validation diagnostics, context.runModel and model.method delivery
model.method in a workflow guard fails with 'Do not know how to serialize a BigInt' when its inputs hold an int
data query: filter on data lifecycle so deleted resources can be excluded
CEL expression results that hold a timestamp, duration or uint become an empty object
Catalog rebuild restores pre-rename versions of a name renamed away and written again
Remote resolveSecret allowlist is always empty for workflow steps that use vault.get
swamp issue redacts git commit SHAs as secrets and fast-check replay paths as IP addresses
typed-secrets 3.4: destination refusals, identifier stand-ins, shell safety and masking
Remote readResource does not apply the legacy schema fallback for untagged sensitive records
Flaky: datastore_v3_outbox_model_property_test throws MayLandLaterError on a random seed (aged commit, failed discard-and-mark checkpoint, power cut)
v3 outbox model property test fails: a commit dropped after a failed append-discard-and-mark reports MayLandLaterError instead of the expected outcome
Workflow step dataOutputOverrides vaultName is accepted but silently dropped
typed-secrets 3.3: sensitive-field accessors, data-origin sentinels and reference validation
Generated vault keys carry raw data-name and vary characters that strict vault providers reject
typed-secrets 3.2: vault.get as a CEL function, bare-token rewriting, AST extraction and up-front existence checks
typed-secrets 3.1: the production Secret, its refusals, the fault context, the container rule and the boundary conversion, behind the internal option
Flaky test: extension_pull_dependency_conflict_test fails with 'failed to copy esbuild binary' during deno bundle
Extension tarball analyzer ignores webhook entry points
Registry drops contentMetadata.extensions (method grafts) at confirm-push
digitalocean codegen leaves an orphaned security_secret.ts model file after its endpoint left the spec
A --repo-dir or SWAMP_REPO_DIR through a symlink fails with 'unable to open database file'
@swamp/1password: vault delete of a missing item probably fails instead of being a no-op
@swamp/1password: item or vault names containing 'authorization' are misreported as auth failures
typed-secrets 2.2: VaultSecretBag entries hold references, one registry per run, 128-bit sentinels
Show wait keys and pending signals to remote clients and the dashboard
Flaky: LitestreamReplicator.acquire 'concurrent acquires in one process share one sidecar and one hold' fails 'no hold leaked' under full-suite load
VaultService.requireExistenceProvider labels the vault instance name as its type
VaultService capability errors label the vault instance name as its type
typed-secrets 2.1: VaultExistenceProvider, its conformance suite, and has on the built-in providers
MockVaultProvider.get and delete throw synchronously instead of rejecting
typed-secrets 1.2: pin the Secret Proxy, refusal set and container rule on cel-js 7.6.1
issue-lifecycle: pr_merged and ship report success and advance the phase when swamp-club is unreachable, writing nothing upstream
serve check-config crashes with a TypeError when a list-valued auth key is a string
access grant list --server shows user subjects as user:user:<name>
Flaky: process_bundle_prober_test 'stopping a host that has already exited lists no children under its pid' fails under full-suite load
typed-secrets 0.4: characterize worker secret authorization and control-plane secret traffic
issue-lifecycle: an approved plan cannot be sent back to planning
typed-secrets 1.2: minimal opaque Secret, CelVaultNamespace receiver, policy inventory and evaluation frame
typed-secrets 1.1: commit the cel-js 7.6.1 probe harness as pinned behavior tests
typed-secrets 0.1: land the typed-secrets plan as a design enabler doc
swamp update: a slicker, colorful download progress bar that ends with a newline
gcs-datastore: a failed lock read is reported as "unlocked" — #2298 in the GCS backend
A manual worker verify racing an enrollment probe can leave a healthy worker unverified until it reconnects
bundle freshness: a one-off deno info failure leaves a source on the in-directory fallback for good
serve shutdown waits for an unsignalled run index rebuild: the continuation sweep's boot pass and request-led rebuilds cannot be stopped by SIGTERM
serve: handlers that sanitize an error for the client never log the original, so path-bearing failures are undiagnosable
literal() output in a step's inputs logs repeated Refusing to evaluate expression warnings
Read non-string sensitive resource fields back with their type
serve and swamp dashboard shutdown waits seconds for the run index warm-up: the plain index parse is not abortable, and the warm-up waits on request-led loads it joined (regression from #3105)
Run index snapshot cache keeps serving a stale run status: ctime is only second-resolution so in-place same-size rewrites alias the stamp, and snapshots are cached even when repair could not write the index (regression from #3105)
Flaky: serve_v3_extension_lockfile_test 'serve extension.pull commits the new entry' fails under full-suite load with extension_pull_failed
Run index repair-by-stat misses an in-place, same-size record rewrite that restores mtime within the same second as its entry: ctime is reported in whole seconds
Dashboard: an idle, visible tab keeps counting runs that have aged out of a preset time range
Dashboard: Models and a model's page re-send model.output.search every minute on a preset time range
Concurrent run writers leave finished runs stuck as running in the run index: repairIndex returns early once counts agree, so lost updates are never healed (regression from #3105)
Run index updates race across processes: concurrent writer processes can lose each other's index entry updates
Dashboard: a live run page polls a full workflow.run.search every 2.5 s, and time-range presets refetch the run summary every minute
Dashboard Workflows, workflow and run pages are slow with large history: workflow.run.summary and workflow.run.search read every run index of every workflow on each request
README, design docs and agent-constraints examples still teach swamp model commands
Decide and add behavior spellings for --model flags, enum values and SWAMP_MODELS_DIR outside the behavior tree
Telemetry records swamp behavior and swamp model invocations under different command names
swamp behavior help, hints and root help still say model in places
A new or edited extension source is imported at startup, so one that exits, hangs or prints at import breaks every command
Workflow step arguments pass any unknown ${{ }} root through; limit pass-through to known foreign roots
A worker that fails its enrollment probe becomes idle and schedulable in memory
A global argument that evaluates to swamp-namespace expression text is silently dropped, and literal() cannot pass it through
Flaky test: AccessDataPoller "a grant created by a real method run through A stays staged on A and reaches B" fails under full-suite load
WorkerGateway.dispatch leaks a capacity slot when recording the busy status fails
vault type search shows an extension vault type under its type id with no description, and cannot find it by name or description
literal() output is refused in a direct step globalArgs, so foreign ${{ }} text has no escape there
serve: a run whose crashed owner was reaped by a peer stays running on the restarted owner
workflow resume refuses a wait that has an applicable pending signal
Accept early signals through swamp serve
datastore namespace list --json reports a corrupt .namespace.json as an unexpected exception with a stack trace
signal_test fails on main: a signal by key handed over by the sweep is not answered taken
datastore namespace migrate treats the repo-local catalog as user data: it cannot be re-run, resumed, or reversed after a read
A signal by key can be answered pending or taken when that is not what happened to it
swamp serve lists every wait-key record on each continuation pass once any pending signal has lapsed
model method run --timeout exits 0 and prints nothing when the timeout cancels the run
Cross-namespace data.* reads return empty attributes in model method run and workflow run
Hold a signal that arrives before its wait opens
swamp-uat: the SIGTERM drain test does not check that the run succeeded
A running dispatch counts twice against worker capacity, so a --concurrency 2 worker queues a step while it has a free slot
data.latest("otherns:model", name) returns the caller's own record when a local model has that name
Record-typed arguments are not JSON-parsed from plain key=value, unlike array and object arguments
swamp model cancel <model> cancels a run of a different model of the same type and reports it under the named model
serve HA: worker enrollment on a replica that lacks the enrollment token's definition may create a second definition (unverified)
Deliver a signal by key through swamp serve
A garbled nested run record crashes a parent resume with a YAML stack trace
Cancel, reject and supersede of a parent run should settle its suspended nested child runs
workflow recover misjudges whether an unknown step is guarded: forEach iterations read as unguarded, and a same-named step in another job is matched
A log line with a closing brace but no opening brace prints the brace doubled
Signal a wait by workflow and key
manual_approval timeout is never enforced on a forEach-expanded gate
workflow run reports a forEach-expanded approval gate as waiting on a nested run
Let a wait_for_signal step declare a key held by one open wait at a time
Show signal waits to remote clients and the dashboard
extension source add imports src/cli/mod.ts, so every change lists extension as an affected command
Show expired approval gates as expired, with a Cancel action in the dashboard
Lazy hydration: content readers bypass hydration (stream, stale raw after append); add one ensuring read path and a fitness test
Lazy hydration: expressions reading another model's data get empty attributes; hydrate a step's read set before evaluation
Tell timeouts apart from cancels in method-run records, and review the hidden 30 s fallback timer for step-called model methods
Settle expired signal waits from swamp serve
serve never continues a signalled run after a manual resume re-entered its open wait
Resume signalled workflow runs automatically under swamp serve
audit verify is hidden by accident: auditCommand.hidden() in mod.ts hides the last subcommand, not audit
swamp help --json: emit aliases, hidden commands, and whether an option takes a value
GCS datastore: implement fetchContent, a read of one remote file that writes nothing locally
S3 datastore: implement fetchContent, a read of one remote file that writes nothing locally
serve: token secret migration repoints a token even when copying its OAuth access token failed
Extension quality check output carries ANSI codes when FORCE_COLOR is set
datastore setup can overwrite an existing remote config tier when it moves an in-repo tier into an extension datastore
Datastore sync: read one remote file without writing it to the local cache (fetchContent)
A cancelled shell step can fail with a lock timeout instead of reporting as cancelled
Docs: document DatastoreSyncService.fetchContent in the datastore provider reference
datastore lock release gives the wrong reason when the lock file is caught mid-write
A nested structural swamp that outlives the run that started it keeps skipping that run's lock while the holder writes
Flaky test: WalSink pruning the queue under a hung downstream keeps the checkpoint delivered segments wait on
A lock file read during its holder's heartbeat rewrite is treated as no lock
Record the commands affected by a change in the verification attestation
Deliver workflow signals through swamp serve
A nested structural swamp still waits on its run's lock when the lock holder is not a same-host ancestor (cross-host worker, --server into a non-ancestor serve)
Deliver workflow signals through a write-once outcome record instead of writing the run record
Forward held-lock identity over --server so a loopback nested swamp does not wait on its caller's lock
A nested structural swamp on a same-host remote worker waits on its own step's lock held by swamp serve
Fail fast when nested structural commands under parallel runs wait on each other's locks
Docs: remove followUpActions from the MethodResult reference in the extension model manual
signal_test: already-settled test fails by chance when a random UUID spells the sender name
Add a wait_for_signal workflow step that pauses a run for a JSON message
run doctor through serve interrupts a live run of another serve instance when no heartbeats are recorded
Flaky test: WalSink replayed segments are delivered before events written after replay leaves one WAL segment
Remove followUpActions: no model or extension can produce them
Workflow schedule watcher and workflow edit symlink lookup ignore the managed config workflows dir
main is red: repository_dirty_coverage_test fails for UnifiedData.collectGarbage(orphaned deferred write)
isProcessGone always reports alive on Linux under the test task permissions, failing 9 tests on main
Flaky test: WalSink replayed segments are delivered before events written after replay
serve: relative tls cert-file/key-file and --config resolve against the working directory, not the repository
findBySpec/findByTag lose a workflow step's output after a version is deleted, pruned or rolled back
serve HA: pull a token's definition on an auth miss so a peer accepts a new token at once
Run tracker keeps some interrupted workflow rows forever: retention waits for markSettled, which several paths never call
run doctor reads every run record on each run to rebuild the run indexes
Serve audit: two sinks with the same name share one emitter cursor, so the second never receives events
Serve audit: a durable sink write that times out is retried while the first call is still pending, so the store can hold duplicate sequences
Flaky test: ModelResolver data accessors for a renamed model return the earlier id's record from findBySpec
data query --limit returns fewer live records than exist and reports limited: false when stale catalog rows fall inside the limit
data get: help example passes --run latest, which is not a recognised run id
workflow reject leaves sibling gates waiting_approval and dependents pending in a failed, retryable run
workflow cancel cannot cancel a suspended run whose workflow file was deleted, and cancel --all silently skips it
A job alone in its level starts after the abort, and its unstarted step is recorded as a real failure instead of settledByAbort
workflow approve racing workflow cancel loses the cancel: the run returns to suspended after cancel reported cancelled
data query: does not follow data rename forwarding that data get follows
Unit tests share fixed /tmp catalog paths, so concurrent runs corrupt them and fail every later run
data query: renamed data items are not found by their old name
data query: model lookup by id and orphan recovery parity with data get
workflow approve on a run that is not suspended prints a fatal error with a stack trace
data query: no way to target a workflow's latest run
data query: binary content lacks contentEncoding/base64 parity with data get
data query: no fallback from spec name to data instance name
data query: no definitionHash or garbageCollection fields in query results
A nested structural swamp spawned by swamp serve skips every lock serve holds, including other runs' locks
data query: confirm catalog parity with data get for custom datastores
data query: single-result --json mode for scripts migrating from data get
Docs: document data query --single and fix the stale --limit default in reference/data.md
data get --workflow silently returns an arbitrary step's data when several steps share a data name
swamp doctor workflows misses nested and .yml extension workflows that the loader reads
auth whoami says the stored API key is no longer valid when auth.json only holds an env-key identity cache
Docs: swamp skill says cancelling a parent run cancels its nested child runs; the child is left suspended
Namespace advice in the slow-lock warning is wrong for a single repo on a local datastore
Method-run records left running by a dead owner are never reaped
Flaky: usecase_sync_characterization_data_test 'data gc (serve)' asserts markDirty order that varies under parallel load
Worker dispatch rebuilds the definition with Definition.create, so a legacy-named model cannot run on a worker
A cancelled workflow exits before its steps stop, leaving their method-run records at running
Nested swamp structural command in a shell step times out on its parent's per-model lock (SWAMP_LOCK_HOLDER_PID is stripped)
Auth gate blocks nested swamp in workflow shell steps when the credential is in SWAMP_API_KEY
swamp workflow create crashes with an [FTL] ZodError stack trace on an invalid (uppercase) workflow name
swamp model create crashes with an [FTL] ZodError stack trace on an invalid (uppercase) model name
model cancel SIGTERMs a running swamp serve process when the method run belongs to a serve-run workflow
Follow-up action retries ignore the run's abort signal and can outlast a cancel
model cancel --all on a running workflow leaves its method-run records at running, or records them failed instead of cancelled
model cancel SIGKILLs the owner 2 s after SIGTERM, before the step executor's own 3 s kill grace, so the run never records its own cancellation
workflow cancel and supersede of a suspended run leave its jobs running in the cancelled record
A force-exited workflow run stays running with a dead pid: resume, recover and run doctor --fix cannot clear it
A nested workflow that suspends on manual approval forwards its suspended event into the parent run's stream
workflow cancel SIGKILLs the run 2 s after SIGTERM, cutting off always/completed cleanup jobs mid-step
Warn when datastore setup keeps managedConfig but the new datastore has no config tier
Docs: run doctor --fix and workflow recover settle runs left running by a force-exited process
serve: a relative grants-dir resolves against the working directory, not the repository
workflow resume cleanup mode can run an always-gated teardown before a suspended job's approved work
Docs: serve-flags.md should state that relative grants-dir and grants-file resolve against the repository
managedConfig: extension writes update the shared lockfile from a stale cache and overwrite peers' entries
managedConfig: model/vault create and other bulk-push commands re-upload a stale extension lockfile and erase peers' entries
datastore setup extension overwrites the shared config tier with the repo-local copy and drops managedConfig from .swamp.yaml
serve daemon enable writes a unit that crash-loops when serve args are invalid (e.g. missing --admins)
Concurrent embedded deno extraction races: stale temp-file cleanup deletes another process's in-flight .deno.tmp file (chmod ENOENT)
Docs: manual approval gates inside nested workflows (swamp-club#2736)
Telemetry bridge drops a parent step's invocation when a nested workflow step has the same job and step names
serve daemon: unit sets SWAMP_HOME, which relocates the config dir — token/oauth daemons can't find auth.json and crash-loop
Workflow step that times out on the model lock leaves its output and run-tracker row stuck at 'running'
Docs: serve daemon enable pins SWAMP_CONFIG_DIR; document the SWAMP_CONFIG_DIR override
An extension upgrade that hits a type collision is rolled back to no version: roll back to the prior version with a commit/rollback install handle
Telemetry bridge attributes a parent run's later method invocations to its nested child run
sensitive_data_delivery_test fails where /bin/sh is bash: the final ps is exec'd and reports its own argv
build-attestation rejects every run: evaluated workflows now carry sensitiveFormat, which the provenance check treats as an uncommitted field
DuplicateTypeError names the incoming extension as the existing claimant and reports a false ghost row when it sorts first
Install extensions by stage and swap, with a journal and crash recovery
workflow resume ignores SWAMP_MAX_CONCURRENT_STEPS for job-level concurrency
Tidy up tests and helpers from the serve suspended-run cancel
Concurrent extension installs and removals can interleave in one checkout: add a pulled-extensions lock and LockfileRepository.refresh()
deno run audit prints No description available for every advisory
Add a literal() CEL function so one string can mix swamp expressions with another service template syntax
Extension archives have no size cap: cap the compressed and decompressed size on pull and push
withGeneratorSpan does not forward return() to the wrapped generator, so its finally blocks are skipped on early exit
forEach over a direct step with a shared modelName runs every iteration with one iteration's globalArgs on first creation
Split installExtension into an unlocked prepare and a repo-changing apply
workflow cancel --server shows raw JSON errors and drops --reason
Two pulled extensions providing one type + a rebuilt catalog brick every command (I-Repo-1 at startup reconcile, even rm/--repair)
Extension catalog keeps stale rows after install migration and rm; model type describe crashes with ENOENT
Flaky test: CollectiveRefreshService fallback getAccessToken test counts calls after a fixed 200ms sleep
Docs: worker-fleet guides start swamp serve without --admins and mint tokens with a removed command
model validate log output misaligns multi-entry Expression paths errors
Retry refusals print unquoted forEach template names in --from hints, which bash rejects
workflow run log output: a forEach step with a templated name labels its iterations with the raw expression, e.g. deploy-${{ self.env }}[0]
Docs: update the model validate Expression paths example in cel-expressions.md
Docker image runs swamp as PID 1 without an init, so orphaned step processes are never reaped
swamp.cli.bootstrap, configure_extension_loaders and teardown spans each start their own trace
Docs: run swamp under an init in the worker-fleet guides
Extension restore leaves a parent installed and its dependency permanently missing when the dependency install fails
serve: vault edit over --server opens an editor on the server host and ignores the managedConfig vaults dir
A suspended run started by swamp serve cannot be cancelled
command/shell: aborting a step kills only sh, leaving the command's child processes orphaned
Extension restore drops the lockfile entry's channel and leaks the parent's expectedChecksum into dependency installs
managedConfig: serve extension handlers disagree on the pulled-extensions root, and lockfile files[] are repo-relative
swamp.model.method spans are parented to the workflow run span instead of their swamp.workflow.step span
Platform certificate load failure surfaces as an uncaught error on network commands
serve: server-login can't be traced: no request spans, and gcs-datastore push is unparented with an 81s gap in child spans
workflow resume keeps the original run's process identity (run record pid and instanceId)
managedConfig: store pulled extension archives in the config tier and converge each checkout to the shared lockfile
workflow run --json: guarded-skip lines land on stdout or stderr depending on timing
workflow run and model method run reject --ca-cert, but their TLS error tells users to pass it
After a cancel or --timeout, jobs that share a level are abandoned mid-flight: their steps stay running and their own cleanup steps never run
SWAMP_CA_CERT overrides the --ca-cert flag, contrary to the documented flag-wins precedence
workflow resume --timeout records cancel_reason 'The signal has been aborted' instead of a timeout reason
InProcessExecutor mutates the process-wide TRACEPARENT, so concurrent in-process executions clobber and leak trace context
workflow resume has no job-level cleanup mode: after --timeout or a cancel, always/completed cleanup jobs start with the aborted signal and fail at once
workflow cancel on a live local run overwrites the owning process's final run record with a stale snapshot
s3-datastore: SWAMP_S3_REQUEST_TIMEOUT_MS does not appear to apply to ListObjectsV2 during pull
s3-datastore: S3Lock.acquire overshoots maxWaitMs by up to a full backoff interval
s3-datastore: a model-scoped pull still lists, walks and indexes the whole namespace
After a cancel or --timeout, a never-started dependency stays pending, so failed- and completed-gated cleanup steps are skipped
Direct-type forEach fan-out waits about 1s on the auto-definition lock when its definition is first created (missed by swamp-club#2126)
managedConfig: fix startup config-base resolution and extension list (prerequisites for swamp-club#2429)
workflow run log output: a skipped plain step's line names only the job, not the step
Docs: manual examples show the old workflow skip line without the step name
A forEach step's dependsOn condition is never evaluated, so its iterations run when the condition is false
managedConfig: the datastore extension is only found through a lockfile, so the config base is guessed (part 3c of swamp-club#2483)
model output data/logs/search can't find outputs of user extension model types locally
serve: ServerTokenGcService is never started, so expired server tokens accumulate forever
workflow resume --input key=value does not coerce the value to the declared input type (workflow run does)
Direct-execution workflow steps validate evaluated globalArgs, so the CEL concatenation workaround for colliding template names fails
workflow resume of a suspended run crashes mid-run when the workflow was edited during the approval window
Install rollback recursively deletes pre-existing skill dirs that pull merged into
Foreign CEL like GitHub Actions ${{ github.sha }} in a global argument fails validate and throws when the method reads it
model validate blames foreign {{ ... }} text for an unclosed ${{ expression
workflow resume --from: a step moved or renamed since the run failed is skipped and the run reported succeeded, or crashes with 'Step run not found'
managedConfig: extension lockfile writes lose updates, and auto-resolved installs never reach the shared lockfile
workflow resume prints unexpected errors as a fatal stack trace, and --json returns them unclassified with the stack
serve: remove the remaining no-path markDirty() calls that force full-cache datastore pushes
workflow resume: Ctrl-C leaves the run 'running' with nothing driving it, and it can no longer be resumed
workflow recover crashes on every invocation: 'Path must be a string'
workflow resume: retry the failed steps of a failed run without --from
Remote indicator prints the full --server URL, including a ?token= query credential, to stderr
serve: vault annotate --label over --server is rejected (CLI sends labels as an object, protocol expects string[])
workflow recover: a changed definition leaves the interrupted run with no way to recover or resume
serve: the device-token poll that returns the token takes 88s, making server-login to ops.swamp-club.com take ~90s
datastore sync: repositories mark paths dirty before writing, so a concurrent ungated push can drop the write
gcs-datastore: push spends almost all its time in an untraced gap between index read and first upload (81s in the #2408 login, up to 836s)
serve: after lazy hydration, scoped poller pulls never take the fast path, and the login mint waits behind them on the sync gate
datastore extensions: full pushes re-hash every pulled file on every run because index mtimes never match pulled copies
steps.<name>.outputs is never populated for model_method steps
auth server-login hangs after device-flow authorization is approved in browser; never completes or times out
Let a method inside serve start a workflow run: context.runWorkflow
s3-datastore: every scoped push on a fresh datastore warns 'Index merge failed, fast path stays unarmed' (missing .datastore-index.json) and the fast path never arms
Flaky integration tests: cel_data_access and webhook_payload_inputs race on SWAMP_TEST_2172_AUTHORED env var
serve rejects doctor.datastores, cluster.instances and serve.config requests: no zod schema
s3/gcs datastore: no safe recovery when _meta.json v2 lists a shard missing from the bucket
Scheduled runs that suspend on an approval gate are logged as failures
swamp run history reports completed for failed workflow runs
s3/gcs-datastore: clean up data left behind by deletes that silently no-opped before the swamp-club#2249 fix
Every webhook lifecycle line is logged twice by swamp serve
gcs-datastore: verify generation-precondition support at setup and in doctor; an emulator or proxy that ignores ifGenerationMatch makes locking silently unsafe
agent-constraints: planning-conventions CI fan-out checklist points at jobs that no longer live in ci.yml
Property test for deferred bindings fails intermittently on a __proto__ key
serve: poller pull can race a handler's delete-then-push and resurrect the deleted item
Relationships
#2431 workflow recover crashes on every invocation: 'Path must be a string'
Opened by zack · 9/23/2026· Shipped 9/23/2026
Problem
swamp workflow recover crashes on every invocation, before it looks up the workflow or the run:
$ swamp workflow recover <workflow>
[FTL] error: TypeError: Path must be a string, received "{"telemetry":true,"color":true,"acknowledgeUnknown":false,"assessOnly":false}"
at assertPath (https://jsr.io/@std/path/1.1.4/_common/assert_path.ts:6:11)
...--run <id>, --assess-only, and --acknowledge-unknown make no difference, and a workflow name that does not exist crashes the same way. That makes checkpoint recovery (swamp-club#2153) unreachable from the CLI. It is also the command serve's boot reaper tells operators to run: Boot: N workflow run(s) interrupted by crash — recover with 'swamp workflow recover <workflow>'.
Cause
src/cli/commands/workflow_recover.ts calls resolveRepoDir(options) with the whole options object. resolveRepoDir in src/cli/context.ts takes the --repo-dir value (string | undefined), and every other command passes options.repoDir. Type checking does not catch it because the action types options as AnyOptions. The command has no test file, so no test runs the action.
Present on main since the command was added in a3123298 (swamp-club#2153).
Fix approach
Pass options.repoDir as the other commands do, and add a command-level test that runs workflow recover against an initialized temp repo with an interrupted run, covering both --assess-only and a real recovery, so the action is exercised end to end.
Reproduction
- In any initialized repo,
swamp workflow recover <any-workflow-name>. - The command exits 1 with the
Path must be a stringTypeError above.
Shipped
Click a lifecycle step above to view its details.