Lab #3217: workflow run reports a forEach-expanded approval gate as waiting on a nested run

Description

When a workflow suspends on a manual_approval step that was expanded by forEach, swamp workflow run reports the suspension as a wait on a nested run. The step is an approval gate of the run's own: the same output prints its prompt and its approve/reject commands, swamp workflow approvals lists it, and swamp workflow approve accepts it.

A manual_approval step without forEach is reported correctly.

Steps to reproduce

In an initialized repo, add this workflow and run it:

id: 7c1f0b2e-5d7a-4c3b-9e21-3027a0b1c2d3
name: foreach-input-gate
tags: {}
inputs:
  type: object
  properties:
    env:
      type: string
  required:
    - env
jobs:
  - name: main
    steps:
      - name: approve-${{ self.env }}
        forEach:
          item: env
          in: "${{ [inputs.env] }}"
        task:
          type: manual_approval
          prompt: "Deploy?"
        dependsOn: []
        weight: 0
    dependsOn: []
    weight: 0
version: 1

swamp workflow run foreach-input-gate --input env=prod

Actual

system │ Starting workflow foreach-input-gate · 17:25:16 UTC

main │ start 17:25:16 UTC
main │ waiting approval required: "Deploy?"

main │ To approve:  swamp workflow approve foreach-input-gate approve-prod --run <run-id>
main │ To reject:   swamp workflow reject foreach-input-gate approve-prod --run <run-id>

system │ Suspended workflow foreach-input-gate — step approve-prod waits on a nested run · 17:25:16 UTC

system │ Once it finishes:  swamp workflow resume foreach-input-gate --run <run-id>

The closing hint, "Once it finishes: swamp workflow resume …", is misleading: nothing finishes until someone approves the gate.

Expected

What the same step prints without forEach (same binary):

system │ Suspended workflow plain-gate — awaiting approval on step approve-prod · 17:25:17 UTC

system │ To approve:  swamp workflow approve plain-gate approve-prod --run <run-id>
system │ After approval:  swamp workflow resume plain-gate --run <run-id>

Likely cause

waitsOnNestedRun() in src/presentation/renderers/workflow_run.ts, added in 3aa438b6 (swamp-club#3110, PR #2954). It returns true when the suspended event has no nested, no wait, an empty prompt, and the named step's status in the run view is not waiting. The forEach-expanded gate evidently meets that condition, though it is a gate with a non-empty prompt ("Deploy?"). I have not traced which of the two inputs is wrong for an expanded step (the event's prompt, or the step lookup/status by expanded name).

Impact

swamp-uat's Adversarial Tier 6 fails on every run: tests/cli/adversarial/step_name_control_chars_test.ts, "swamp workflow run and approvals print a forEach step name from an item value with its control characters escaped", which asserts awaiting approval on step <name>. First seen on swamp-uat run 37963547703 (PR #605, unrelated change). Reproduces 3/3 locally.

Environment

swamp 20261009.154711.0-sha.8df63c9b
Linux x86_64

flaky: LitestreamReplicator 'a failed launch releases its lock probe, so the next acquire starts a sidecar' fails by timing in verify-build

typed-secrets 4.4: persistence and resume hold references: rehydration stops reading the vault, every persisted copy uses the persisted form, and run events are masked

data_writer_property_test: a secret holding a newline and a literal backslash-n escapes the persisted-content property (seed 1648929202)

typed-secrets: the accessor adapter reads a legacy row stamped with the empty namespace as foreign, so its tagged fields stay reference text on the typed path

typed-secrets query view: route a row's JSON content by whether the adapter adapted the record, not by the service's namespace rule

flaky: for_each_name_property_test lets the text around a sentinel equal the secret's value, so the untyped value search replaces both and the property fails by seed

typed-secrets 4.3: the expression-facing query view registers Secret in DataQueryService

CEL duration arithmetic is off by a second when a duration has negative nanos (cel-js 7.6.1)

typed-secrets 4.2: live and reconstructed step outputs and child outputs return references through the accessor adapter

A coded UserError a model method throws reaches a workflow step without its code, because InProcessExecutor keeps only the message

Dashboard Workflows page takes ~19s on a serve with many extension workflows: workflow.search resolves each workflow's owner with a full extension-workflow scan

codegen: BROKEN_REGEXES override for the @swamp/aws/invoicing InvoiceUnit name lacks the u flag, so it rejects ordinary names

dashboard: run page polling stacks history.get and history.logs requests while serve is slow

serve: add child spans and attributes to workflow.history.get (and run_summary/run_search) to diagnose slow dashboard loads

serve: webhook-triggered workflow runs can't be cancelled — cancel can't find them, so it reports serve's own pid as "the process running it is still alive"

A workflow step fails on a static validation error in a method it does not invoke, where swamp behavior run of the same method succeeds

serve: a run orphaned by a container restart can't be cancelled — the restarted serve reuses the dead owner's pid, so cancel reports the owner still alive

typed-secrets 4.6: turn the typed path on for every entry point, remove the internal option, and delete the regular-expression rewrite, the value search and the plaintext resolution of tagged fields

typed-secrets 4.5: the inline resolutions refuse or mask: placement, task selectors, child references and deferred targets refuse secrets, assert messages mask them, and the deprecated resolvers go

typed-secrets 4.1: workflow passes bind sentinels as Secrets: typed guards and asserts, forEach items and identities, child bindings, webhook inputs, and the synchronous authorization step

typed-secrets 3.5: local entry points, validation diagnostics, context.runModel and model.method delivery

model.method in a workflow guard fails with 'Do not know how to serialize a BigInt' when its inputs hold an int

data query: filter on data lifecycle so deleted resources can be excluded

CEL expression results that hold a timestamp, duration or uint become an empty object

Catalog rebuild restores pre-rename versions of a name renamed away and written again

Remote resolveSecret allowlist is always empty for workflow steps that use vault.get

swamp issue redacts git commit SHAs as secrets and fast-check replay paths as IP addresses

typed-secrets 3.4: destination refusals, identifier stand-ins, shell safety and masking

Remote readResource does not apply the legacy schema fallback for untagged sensitive records

Flaky: datastore_v3_outbox_model_property_test throws MayLandLaterError on a random seed (aged commit, failed discard-and-mark checkpoint, power cut)

v3 outbox model property test fails: a commit dropped after a failed append-discard-and-mark reports MayLandLaterError instead of the expected outcome

Workflow step dataOutputOverrides vaultName is accepted but silently dropped

typed-secrets 3.3: sensitive-field accessors, data-origin sentinels and reference validation

Generated vault keys carry raw data-name and vary characters that strict vault providers reject

typed-secrets 3.2: vault.get as a CEL function, bare-token rewriting, AST extraction and up-front existence checks

typed-secrets 3.1: the production Secret, its refusals, the fault context, the container rule and the boundary conversion, behind the internal option

Flaky test: extension_pull_dependency_conflict_test fails with 'failed to copy esbuild binary' during deno bundle

Extension tarball analyzer ignores webhook entry points

Registry drops contentMetadata.extensions (method grafts) at confirm-push

digitalocean codegen leaves an orphaned security_secret.ts model file after its endpoint left the spec

A --repo-dir or SWAMP_REPO_DIR through a symlink fails with 'unable to open database file'

@swamp/1password: vault delete of a missing item probably fails instead of being a no-op

@swamp/1password: item or vault names containing 'authorization' are misreported as auth failures

typed-secrets 2.2: VaultSecretBag entries hold references, one registry per run, 128-bit sentinels

Show wait keys and pending signals to remote clients and the dashboard

Flaky: LitestreamReplicator.acquire 'concurrent acquires in one process share one sidecar and one hold' fails 'no hold leaked' under full-suite load

VaultService.requireExistenceProvider labels the vault instance name as its type

VaultService capability errors label the vault instance name as its type

typed-secrets 2.1: VaultExistenceProvider, its conformance suite, and has on the built-in providers

MockVaultProvider.get and delete throw synchronously instead of rejecting

typed-secrets 1.2: pin the Secret Proxy, refusal set and container rule on cel-js 7.6.1

issue-lifecycle: pr_merged and ship report success and advance the phase when swamp-club is unreachable, writing nothing upstream

serve check-config crashes with a TypeError when a list-valued auth key is a string

access grant list --server shows user subjects as user:user:<name>

Flaky: process_bundle_prober_test 'stopping a host that has already exited lists no children under its pid' fails under full-suite load

typed-secrets 0.4: characterize worker secret authorization and control-plane secret traffic

issue-lifecycle: an approved plan cannot be sent back to planning

typed-secrets 1.2: minimal opaque Secret, CelVaultNamespace receiver, policy inventory and evaluation frame

typed-secrets 1.1: commit the cel-js 7.6.1 probe harness as pinned behavior tests

typed-secrets 0.1: land the typed-secrets plan as a design enabler doc

swamp update: a slicker, colorful download progress bar that ends with a newline

gcs-datastore: a failed lock read is reported as "unlocked" — #2298 in the GCS backend

A manual worker verify racing an enrollment probe can leave a healthy worker unverified until it reconnects

bundle freshness: a one-off deno info failure leaves a source on the in-directory fallback for good

serve shutdown waits for an unsignalled run index rebuild: the continuation sweep's boot pass and request-led rebuilds cannot be stopped by SIGTERM

serve: handlers that sanitize an error for the client never log the original, so path-bearing failures are undiagnosable

literal() output in a step's inputs logs repeated Refusing to evaluate expression warnings

Read non-string sensitive resource fields back with their type

serve and swamp dashboard shutdown waits seconds for the run index warm-up: the plain index parse is not abortable, and the warm-up waits on request-led loads it joined (regression from #3105)

Run index snapshot cache keeps serving a stale run status: ctime is only second-resolution so in-place same-size rewrites alias the stamp, and snapshots are cached even when repair could not write the index (regression from #3105)

Flaky: serve_v3_extension_lockfile_test 'serve extension.pull commits the new entry' fails under full-suite load with extension_pull_failed

Run index repair-by-stat misses an in-place, same-size record rewrite that restores mtime within the same second as its entry: ctime is reported in whole seconds

Dashboard: an idle, visible tab keeps counting runs that have aged out of a preset time range

Dashboard: Models and a model's page re-send model.output.search every minute on a preset time range

Concurrent run writers leave finished runs stuck as running in the run index: repairIndex returns early once counts agree, so lost updates are never healed (regression from #3105)

Run index updates race across processes: concurrent writer processes can lose each other's index entry updates

Dashboard: a live run page polls a full workflow.run.search every 2.5 s, and time-range presets refetch the run summary every minute

Dashboard Workflows, workflow and run pages are slow with large history: workflow.run.summary and workflow.run.search read every run index of every workflow on each request

README, design docs and agent-constraints examples still teach swamp model commands

Decide and add behavior spellings for --model flags, enum values and SWAMP_MODELS_DIR outside the behavior tree