#2579 Direct-type forEach fan-out waits about 1s on the auto-definition lock when its definition is first created (missed by swamp-club#2126)

Summary

When the iterations of a forEach step that uses a direct-type task (modelType + modelName) start together and the definition does not exist yet, every iteration except one waits about one second on the auto-definition create lock, although the holder releases it within a few milliseconds. Each waiter also logs a Waiting for lock ... .auto-definition-create/<name>.lock warning that names its own process as the holder.

swamp-club#2126 fixed the same one-second initial retry for model locks by passing retryIntervalMs: MODEL_LOCK_RETRY_INTERVAL_MS (25 ms) in createModelLock() (src/cli/repo_context.ts). The auto-definition lock was not included, so it still starts its backoff at the FileLock default of 1,000 ms.

Steps to reproduce

jobs:
  - name: main
    steps:
      - name: fan
        forEach: { item: i, in: '${{ ["0","1","2","3","4"] }}' }
        task: { type: model_method, modelType: command/shell, modelName: fan-fresh-1, methodName: execute, inputs: { run: 'true' } }

Run it once with a modelName that has no definition yet, then again (the definition now exists).

Observed

swamp source at ca3aa618 (main), Linux x86_64, local filesystem datastore. Three runs each, a fresh modelName per run:

With the definition already present the lock is not taken (the fast-path lookup finds it), and the same 5-iteration run takes about 130 ms.

With retryIntervalMs: 25 added to that lock locally, the waits drop to 21-175 ms and the first runs to 111-147 ms (5 iterations) and 259-550 ms (10 iterations).

Cause

resolveOrCreateDefinition() in src/libswamp/models/direct_execution.ts serializes concurrent auto-creation of a name with

new FileLock(lockDir, { lockKey: autoDefinitionLockKey(definitionName), ttlMs: 5_000, maxWaitMs: 10_000 })

without retryIntervalMs. FileLock.acquire() (src/infrastructure/persistence/file_lock.ts) starts at DEFAULT_RETRY_INTERVAL_MS = 1,000 ms with 25% jitter, so a sibling that loses the create race sleeps 750-1,250 ms although the critical section (re-check, create) takes milliseconds.

Impact

Every forEach fan-out over a direct-type step pays about a second the first time its definition is created. .swamp/ is in the managed .gitignore, so a fresh checkout without a shared datastore (a CI job, for example) likely pays it on every run. Workflow steps that call the same direct-type model concurrently are affected the same way.

Suggested fix

Give the auto-definition lock the same short initial retry interval as model locks (25 ms), keeping jitter, exponential backoff, maxWaitMs and stale-lock handling. MODEL_LOCK_RETRY_INTERVAL_MS lives in src/cli/repo_context.ts, so a libswamp import of it may cross a layer boundary. A local constant or one shared from the infrastructure layer avoids that. A unit test can assert the option is passed, as #2126's test does for model locks.

Out of scope: the per-model data lock (.swamp/data/<type>/<id>/.lock) also logs Waiting for lock when iterations of one model write concurrently. Its waits are short since #2126, and the warning is the intended feedback from swamp-club#387.

Found while end-to-end testing swamp-club#2537.