WORKFLOW PLACEMENT

Workflow steps can include placement fields that control where they execute. Steps without placement fields run locally on the orchestrator. Steps with placement fields are dispatched to a matching remote worker.

For the underlying executor model, see Remote Execution. For the full workflow YAML schema, see Workflows.

Placement Fields

Two mutually exclusive fields control step placement. A step may declare target or labels, but not both.

target

Dispatches the step to a specific worker by name or instance UUID.

The value must match the worker's registered name (set at enrollment) or its instance UUID. If the string is a valid UUID, it is matched against instance UUIDs first, then worker names.

labels

Dispatches the step to any connected worker whose labels are a superset of the selector.

Every key-value pair in the selector must be present on the worker's label set for the worker to match. Workers register labels at connect time via the --label flag on swamp worker connect — see Worker Commands.

queueTimeout

Overrides the server's default queue timeout for this step.

When a step is queued waiting for a matching worker, it times out after this many seconds. Set to 0 to disable the timeout. The per-step value overrides the server's --queue-timeout flag.

Example

jobs:
  - name: build
    steps:
      - name: compile
        task:
          type: model_method
          modelIdOrName: builder
          methodName: run
        target: build-node
      - name: test
        task:
          type: model_method
          modelIdOrName: tester
          methodName: run
        labels:
          region: us-east
          gpu: "true"

The compile step is dispatched to the worker named build-node. The test step is dispatched to any connected worker whose labels include both region: us-east and gpu: "true".

Scheduling Behavior

When a worker matches

The orchestrator assigns the step to a matching worker immediately. If multiple workers match a labels selector, the orchestrator selects one from the available pool.

When no worker matches

If no connected worker matches the placement, the step is queued. It remains queued until a matching worker connects or the queue timeout expires.

If no matching worker appears within the timeout, the step fails with an error naming the unmet placement requirement (e.g., "Timed out waiting for a worker matching labels gpu=true to become available").

The timeout is layered: per-step queueTimeout overrides the server --queue-timeout flag, which defaults to 10m. 0 at either layer disables the timeout — the step queues indefinitely until a matching worker enrolls or the workflow is cancelled.

Interaction with Non-Placed Steps

Steps without target or labels run locally on the orchestrator through the local loopback executor. Placed and non-placed steps can coexist in the same job. The orchestrator applies the same dependency resolution and execution ordering regardless of where each step runs.

Output, reports, and follow-up actions behave identically regardless of where the step executes. The orchestrator proxies all data access — a remote worker streams results back through the data plane, and downstream steps see the same output shape as if the step had run locally.