Gcp/dataflow
@swamp/gcp/dataflowv2026.10.23.1
01README
Google Cloud dataflow infrastructure models
02Release Notes
- Updated: configstoresettings, jobs, jobs_messages, jobs_snapshots, snapshots, templates
03Behaviors
configstoresettings.tsv2026.10.23.1
Global Arguments
| Argument | Type | Description |
|---|---|---|
| accessToken? | string | GCP OAuth2 access token; overrides GCP_ACCESS_TOKEN environment variable. Wire with a vault.get(...) expression to source it from a vault. |
| credentialsJson? | string | GCP service account JSON credentials; overrides GOOGLE_APPLICATION_CREDENTIALS_JSON environment variable. Wire with a vault.get(...) expression to source it from a vault. |
| project? | string | GCP project ID; overrides GCP_PROJECT / GOOGLE_CLOUD_PROJECT environment variables. |
| scopes? | string | Comma-separated OAuth scopes to request when minting access tokens via gcloud. Defaults to the API's Discovery Document scopes. |
| quotaProject? | string | GCP project ID for quota and billing attribution; sets the x-goog-user-project header. Overrides GOOGLE_CLOUD_QUOTA_PROJECT environment variable. Required for APIs like Cloud Identity when using user credentials. |
| apiEndpoint? | string | Custom API endpoint for emulators; overrides GCP_API_ENDPOINT environment variable. Defaults to the service's production URL. |
| name? | string | Identifier. The resource name of the setting. |
| value? | object | Represents a boolean value. |
| configStoreSettingId? | string | Required. The ID to use for the setting. |
| parent? | string | The parent resource name (e.g., projects/my-project/locations/us-central1, organizations/123, folders/456) |
fn create()
Create a configStoreSettings
fn get(identifier: string)
Get a configStoreSettings
| Argument | Type | Description |
|---|---|---|
| identifier | string | The name of the configStoreSettings |
fn delete(identifier: string)
Delete the configStoreSettings
| Argument | Type | Description |
|---|---|---|
| identifier | string | The name of the configStoreSettings |
fn sync(identifier?: string)
Sync configStoreSettings state from GCP
| Argument | Type | Description |
|---|---|---|
| identifier? | string | Target a specific configStoreSettings by name (e.g. one discovered by list) |
fn list(pageSize?: number, maxPages?: number)
List configStoreSettings resources
| Argument | Type | Description |
|---|---|---|
| pageSize? | number | Optional. The maximum number of settings to return. |
| maxPages? | number | Maximum number of pages to fetch (default: 10) |
fn resolve()
resolve
Resources
state(infinite)— A ConfigStoreSetting resource.
jobs.tsv2026.10.23.1
Global Arguments
| Argument | Type | Description |
|---|---|---|
| accessToken? | string | GCP OAuth2 access token; overrides GCP_ACCESS_TOKEN environment variable. Wire with a vault.get(...) expression to source it from a vault. |
| credentialsJson? | string | GCP service account JSON credentials; overrides GOOGLE_APPLICATION_CREDENTIALS_JSON environment variable. Wire with a vault.get(...) expression to source it from a vault. |
| project? | string | GCP project ID; overrides GCP_PROJECT / GOOGLE_CLOUD_PROJECT environment variables. |
| scopes? | string | Comma-separated OAuth scopes to request when minting access tokens via gcloud. Defaults to the API's Discovery Document scopes. |
| quotaProject? | string | GCP project ID for quota and billing attribution; sets the x-goog-user-project header. Overrides GOOGLE_CLOUD_QUOTA_PROJECT environment variable. Required for APIs like Cloud Identity when using user credentials. |
| apiEndpoint? | string | Custom API endpoint for emulators; overrides GCP_API_ENDPOINT environment variable. Defaults to the service's production URL. |
| clientRequestId? | string | The client's unique identifier of the job, re-used across retried attempts. If this field is set, the service will ensure its uniqueness. The request to create a job will fail if the service has knowledge of a previously submitted job with the same client's ID and job name. The caller may use this field to ensure idempotence of job creation across retried attempts to create a job. By default, the field is empty and, in that case, the service ignores it. |
| createTime? | string | The timestamp when the job was initially created. Immutable and set by the Cloud Dataflow service. |
| createdFromSnapshotId? | string | If this is specified, the job's initial state is populated from the given snapshot. |
| currentState? | enum | The current state of the job. Jobs are created in the `JOB_STATE_STOPPED` state unless otherwise specified. A job in the `JOB_STATE_RUNNING` state may asynchronously enter a terminal state. After a job has reached a terminal state, no further state updates may be made. This field might be mutated by the Dataflow service; callers cannot mutate it. |
| currentStateTime? | string | The timestamp associated with the current state. |
| environment? | object | The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com". |
| executionInfo? | object | The steps associated with the execution stage. Note that stages may have several steps, and that a given step might be run by more than one stage. |
| id? | string | The unique ID of this job. This field is set by the Dataflow service when the job is created, and is immutable for the life of the job. |
| jobMetadata? | object | InstanceId accessed in the connection. |
| labels? | record | User-defined labels for this job. The labels map can contain no more than 64 entries. Entries of the labels map are UTF8 strings that comply with the following restrictions: * Keys must conform to regexp: \\p{Ll}\\p{Lo}{0,62} * Values must conform to regexp: [\\p{Ll}\\p{Lo}\\p{N}_-]{0,63} * Both keys and values are additionally constrained to be <= 128 bytes in size. |
| location? | string | Optional. The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains this job. |
| name? | string | Optional. The user-specified Dataflow job name. Only one active job with a given name can exist in a project within one region at any given time. Jobs in different regions can have the same name. If a caller attempts to create a job with the same name as an active job that already exists, the attempt returns the existing job. The name must match the regular expression `[a-z]([-a-z0-9]{0,1022}[a-z0-9])?` |
| pipelineDescription? | object | Contains value if the data is of a boolean type. |
| projectId? | string | The ID of the Google Cloud project that the job belongs to. |
| replaceJobId? | string | If this job is an update of an existing job, this field is the job ID of the job it replaced. When sending a `CreateJobRequest`, you can update a job by specifying it here. The job named here is stopped, and its intermediate state is transferred to this job. |
| replacedByJobId? | string | If another job is an update of this job (and thus, this job is in `JOB_STATE_UPDATED`), this field contains the ID of that job. |
| requestedState? | enum | The job's requested state. Applies to `UpdateJob` requests. Set `requested_state` with `UpdateJob` requests to switch between the states `JOB_STATE_STOPPED` and `JOB_STATE_RUNNING`. You can also use `UpdateJob` requests to change a job's state from `JOB_STATE_RUNNING` to `JOB_STATE_CANCELLED`, `JOB_STATE_DONE`, or `JOB_STATE_DRAINED`. These states irrevocably terminate the job if it hasn't already reached a terminal state. This field has no effect on `CreateJob` requests. |
| runtimeUpdatableParams? | object | Optional. Deprecated: Use `latency_tier` instead. The backlog threshold duration in seconds for autoscaling. Value must be non-negative. |
| satisfiesPzs? | boolean | Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests. |
| stageStates? | array | The time at which the stage transitioned to this state. |
| startTime? | string | The timestamp when the job was started (transitioned to JOB_STATE_PENDING). Flexible resource scheduling jobs are started with some delay after job creation, so start_time is unset before start and is updated when the job is started by the Cloud Dataflow service. For other jobs, start_time always equals to create_time and is immutable and set by the Cloud Dataflow service. |
| steps? | array | The kind of step in the Cloud Dataflow job. |
| stepsLocation? | string | The Cloud Storage location where the steps are stored. |
| tempFiles? | array | A set of files the system should be aware of that are used for temporary storage. These temporary files will be removed on job completion. No duplicates are allowed. No file patterns are supported. The supported files are: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object} |
| transformNameMapping? | record | Optional. The map of transform name prefixes of the job to be replaced to the corresponding name prefixes of the new job. |
| type? | enum | Optional. The type of Dataflow job. |
| view? | string | The level of information requested in response. |
fn create()
Create a jobs
fn get(identifier: string)
Get a jobs
| Argument | Type | Description |
|---|---|---|
| identifier | string | The name of the jobs |
fn update(identifier?: string)
Update jobs attributes
| Argument | Type | Description |
|---|---|---|
| identifier? | string | Target a specific jobs by name (e.g. one discovered by list) |
fn sync(identifier?: string)
Sync jobs state from GCP
| Argument | Type | Description |
|---|---|---|
| identifier? | string | Target a specific jobs by name (e.g. one discovered by list) |
fn list(createTimeRange_endTime?: string, createTimeRange_startTime?: string, filter?: string, location?: string, name?: string, pageSize?: number, regionalFanoutRequested?: boolean, terminationTimeRange_endTime?: string, terminationTimeRange_startTime?: string, maxPages?: number)
List jobs resources
| Argument | Type | Description |
|---|---|---|
| createTimeRange_endTime? | string | Optional. Exclusive end of the interval. If specified, a Timestamp matching this interval will have to be before the end. |
| createTimeRange_startTime? | string | Optional. Inclusive start of the interval. If specified, a Timestamp matching this interval will have to be the same or after the start. |
| filter? | string | The kind of filter to use. |
| location? | string | The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains this job. |
| name? | string | Optional. The job name. |
| pageSize? | number | If there are many jobs, limit response to at most this many. The actual number of jobs returned will be the lesser of max_responses and an unspecified server-defined limit. |
| regionalFanoutRequested? | boolean | Optional. |
| terminationTimeRange_endTime? | string | Optional. Exclusive end of the interval. If specified, a Timestamp matching this interval will have to be before the end. |
| terminationTimeRange_startTime? | string | Optional. Inclusive start of the interval. If specified, a Timestamp matching this interval will have to be the same or after the start. |
| maxPages? | number | Maximum number of pages to fetch (default: 10) |
fn aggregated(createTimeRange_endTime?: any, createTimeRange_startTime?: any, filter?: any, location?: any, name?: any, pageSize?: any, pageToken?: any, regionalFanoutRequested?: any, terminationTimeRange_endTime?: any, terminationTimeRange_startTime?: any, view?: any)
aggregated
| Argument | Type | Description |
|---|---|---|
| createTimeRange_endTime? | any | |
| createTimeRange_startTime? | any | |
| filter? | any | |
| location? | any | |
| name? | any | |
| pageSize? | any | |
| pageToken? | any | |
| regionalFanoutRequested? | any | |
| terminationTimeRange_endTime? | any | |
| terminationTimeRange_startTime? | any | |
| view? | any |
fn get_metrics(location?: any, startTime?: any)
get metrics
| Argument | Type | Description |
|---|---|---|
| location? | any | |
| startTime? | any |
fn snapshot(description?: any, location?: any, snapshotSources?: any, ttl?: any)
snapshot
| Argument | Type | Description |
|---|---|---|
| description? | any | |
| location? | any | |
| snapshotSources? | any | |
| ttl? | any |
Resources
state(infinite)— Defines a job to be run by the Cloud Dataflow service. Do not enter confident...
jobs_messages.tsv2026.10.23.1
Global Arguments
| Argument | Type | Description |
|---|---|---|
| name | string | Instance name for this resource (used as the unique identifier in the factory pattern) |
| accessToken? | string | GCP OAuth2 access token; overrides GCP_ACCESS_TOKEN environment variable. Wire with a vault.get(...) expression to source it from a vault. |
| credentialsJson? | string | GCP service account JSON credentials; overrides GOOGLE_APPLICATION_CREDENTIALS_JSON environment variable. Wire with a vault.get(...) expression to source it from a vault. |
| project? | string | GCP project ID; overrides GCP_PROJECT / GOOGLE_CLOUD_PROJECT environment variables. |
| scopes? | string | Comma-separated OAuth scopes to request when minting access tokens via gcloud. Defaults to the API's Discovery Document scopes. |
| quotaProject? | string | GCP project ID for quota and billing attribution; sets the x-goog-user-project header. Overrides GOOGLE_CLOUD_QUOTA_PROJECT environment variable. Required for APIs like Cloud Identity when using user credentials. |
| apiEndpoint? | string | Custom API endpoint for emulators; overrides GCP_API_ENDPOINT environment variable. Defaults to the service's production URL. |
| jobId | string | The job to get messages about. |
fn get(identifier: string)
Get a messages
| Argument | Type | Description |
|---|---|---|
| identifier | string | The name of the messages |
fn sync(identifier?: string)
Sync messages state from GCP
| Argument | Type | Description |
|---|---|---|
| identifier? | string | Target a specific messages by name (e.g. one discovered by list) |
fn list(endTime?: string, location?: string, minimumImportance?: string, pageSize?: number, startTime?: string, maxPages?: number)
List messages resources
| Argument | Type | Description |
|---|---|---|
| endTime? | string | Return only messages with timestamps < end_time. The default is now (i.e. return up to the latest messages available). |
| location? | string | The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the job specified by job_id. |
| minimumImportance? | string | Filter to only get messages with importance >= level |
| pageSize? | number | If specified, determines the maximum number of messages to return. If unspecified, the service may choose an appropriate default, or may return an arbitrarily large number of results. |
| startTime? | string | If specified, return only messages with timestamps >= start_time. The default is the job creation time (i.e. beginning of messages). |
| maxPages? | number | Maximum number of pages to fetch (default: 10) |
Resources
state(infinite)— A structured message reporting an autoscaling decision made by the Dataflow s...
jobs_snapshots.tsv2026.10.23.1
Global Arguments
| Argument | Type | Description |
|---|---|---|
| name | string | Instance name for this resource (used as the unique identifier in the factory pattern) |
| accessToken? | string | GCP OAuth2 access token; overrides GCP_ACCESS_TOKEN environment variable. Wire with a vault.get(...) expression to source it from a vault. |
| credentialsJson? | string | GCP service account JSON credentials; overrides GOOGLE_APPLICATION_CREDENTIALS_JSON environment variable. Wire with a vault.get(...) expression to source it from a vault. |
| project? | string | GCP project ID; overrides GCP_PROJECT / GOOGLE_CLOUD_PROJECT environment variables. |
| scopes? | string | Comma-separated OAuth scopes to request when minting access tokens via gcloud. Defaults to the API's Discovery Document scopes. |
| quotaProject? | string | GCP project ID for quota and billing attribution; sets the x-goog-user-project header. Overrides GOOGLE_CLOUD_QUOTA_PROJECT environment variable. Required for APIs like Cloud Identity when using user credentials. |
| apiEndpoint? | string | Custom API endpoint for emulators; overrides GCP_API_ENDPOINT environment variable. Defaults to the service's production URL. |
| location | string | The location to list snapshots in. |
| jobId | string | If specified, list snapshots created from this job. |
fn get(identifier: string)
Get a snapshots
| Argument | Type | Description |
|---|---|---|
| identifier | string | The name of the snapshots |
fn sync(identifier?: string)
Sync snapshots state from GCP
| Argument | Type | Description |
|---|---|---|
| identifier? | string | Target a specific snapshots by name (e.g. one discovered by list) |
fn list(maxPages?: number)
List snapshots resources
| Argument | Type | Description |
|---|---|---|
| maxPages? | number | Maximum number of pages to fetch (default: 10) |
Resources
state(infinite)— Represents a snapshot of a job.
snapshots.tsv2026.10.23.1
Global Arguments
| Argument | Type | Description |
|---|---|---|
| name | string | Instance name for this resource (used as the unique identifier in the factory pattern) |
| accessToken? | string | GCP OAuth2 access token; overrides GCP_ACCESS_TOKEN environment variable. Wire with a vault.get(...) expression to source it from a vault. |
| credentialsJson? | string | GCP service account JSON credentials; overrides GOOGLE_APPLICATION_CREDENTIALS_JSON environment variable. Wire with a vault.get(...) expression to source it from a vault. |
| project? | string | GCP project ID; overrides GCP_PROJECT / GOOGLE_CLOUD_PROJECT environment variables. |
| scopes? | string | Comma-separated OAuth scopes to request when minting access tokens via gcloud. Defaults to the API's Discovery Document scopes. |
| quotaProject? | string | GCP project ID for quota and billing attribution; sets the x-goog-user-project header. Overrides GOOGLE_CLOUD_QUOTA_PROJECT environment variable. Required for APIs like Cloud Identity when using user credentials. |
| apiEndpoint? | string | Custom API endpoint for emulators; overrides GCP_API_ENDPOINT environment variable. Defaults to the service's production URL. |
| location | string | The location that contains this snapshot. |
fn get(identifier: string)
Get a snapshots
| Argument | Type | Description |
|---|---|---|
| identifier | string | The name of the snapshots |
fn delete(identifier: string)
Delete the snapshots
| Argument | Type | Description |
|---|---|---|
| identifier | string | The name of the snapshots |
fn sync(identifier?: string)
Sync snapshots state from GCP
| Argument | Type | Description |
|---|---|---|
| identifier? | string | Target a specific snapshots by name (e.g. one discovered by list) |
fn list(jobId?: string, maxPages?: number)
List snapshots resources
| Argument | Type | Description |
|---|---|---|
| jobId? | string | If specified, list snapshots created from this job. |
| maxPages? | number | Maximum number of pages to fetch (default: 10) |
Resources
state(infinite)— Represents a snapshot of a job.
templates.tsv2026.10.23.1
Global Arguments
| Argument | Type | Description |
|---|---|---|
| name | string | Instance name for this resource (used as the unique identifier in the factory pattern) |
| accessToken? | string | GCP OAuth2 access token; overrides GCP_ACCESS_TOKEN environment variable. Wire with a vault.get(...) expression to source it from a vault. |
| credentialsJson? | string | GCP service account JSON credentials; overrides GOOGLE_APPLICATION_CREDENTIALS_JSON environment variable. Wire with a vault.get(...) expression to source it from a vault. |
| project? | string | GCP project ID; overrides GCP_PROJECT / GOOGLE_CLOUD_PROJECT environment variables. |
| scopes? | string | Comma-separated OAuth scopes to request when minting access tokens via gcloud. Defaults to the API's Discovery Document scopes. |
| quotaProject? | string | GCP project ID for quota and billing attribution; sets the x-goog-user-project header. Overrides GOOGLE_CLOUD_QUOTA_PROJECT environment variable. Required for APIs like Cloud Identity when using user credentials. |
| apiEndpoint? | string | Custom API endpoint for emulators; overrides GCP_API_ENDPOINT environment variable. Defaults to the service's production URL. |
| environment? | object | Optional. Additional experiment flags for the job, specified with the `--experiments` option. |
| gcsPath? | string | Required. A Cloud Storage path to the template from which to create the job. Must be a valid Cloud Storage URL, beginning with `gs://`. |
| jobName? | string | Required. The job name to use for the created job. |
| location? | string | The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) to which to direct the request. |
| parameters? | record | The runtime parameters to pass to the job. |
fn create()
Create a templates
fn get(identifier: string)
Get a templates
| Argument | Type | Description |
|---|---|---|
| identifier | string | The name of the templates |
fn sync(identifier?: string)
Sync templates state from GCP
| Argument | Type | Description |
|---|---|---|
| identifier? | string | Target a specific templates by name (e.g. one discovered by list) |
fn launch(environment?: any, jobName?: any, parameters?: any, transformNameMapping?: any, update?: any, dynamicTemplate_gcsPath?: any, dynamicTemplate_stagingLocation?: any, gcsPath?: any, validateOnly?: any)
launch
| Argument | Type | Description |
|---|---|---|
| environment? | any | |
| jobName? | any | |
| parameters? | any | |
| transformNameMapping? | any | |
| update? | any | |
| dynamicTemplate_gcsPath? | any | |
| dynamicTemplate_stagingLocation? | any | |
| gcsPath? | any | |
| validateOnly? | any |
Resources
state(infinite)— The response to a GetTemplate request.
04Previous Versions
2026.10.22.1AGPL-3.0-or-later
- Updated: jobs
2026.10.17.1AGPL-3.0-or-later(was none)
- Updated: jobs
2026.10.13.1
- Updated: configstoresettings, jobs, jobs_messages, jobs_snapshots, snapshots, templates
2026.10.06.1
- Updated: jobs, snapshots
Modified 1 behaviors
2026.09.07.2
- Updated: templates
2026.09.05.1
- Updated: jobs
2026.08.27.1
- Added: configstoresettings
2026.08.25.1
- Updated: jobs, jobs_messages, jobs_snapshots, snapshots, templates
2026.08.14.1
- Updated: jobs
2026.08.12.2
- Updated: jobs, jobs_messages, jobs_snapshots, snapshots, templates
2026.07.29.1
- Updated: jobs, jobs_messages, jobs_snapshots, snapshots, templates
2026.07.21.4
- Updated: jobs
2026.07.21.1
- Updated: jobs, templates
2026.07.20.2
- Updated: jobs
2026.07.20.1
- Updated: jobs, jobs_messages, jobs_snapshots, snapshots, templates
2026.07.19.3
- Updated: jobs_messages, jobs_snapshots, snapshots
2026.07.19.2
- Updated: jobs, jobs_messages, jobs_snapshots, snapshots, templates
2026.07.19.1
- Updated: jobs
2026.07.18.2
- Updated: jobs, jobs_messages, jobs_snapshots, snapshots, templates
05Stats
A
100 / 100
Downloads
35
Archive size
92.1 KB
Verified by Swamp
- Has README or module doc2/2earned
- README has a code example1/1earned
- README is substantive1/1earned
- Most symbols documented1/1earned
- No slow types (deprecated)1/1earned
- Dependencies pass trust audit2/2earned
- Docs keep secrets in a vault1/1earned
- Has description1/1earned
- Platform support declared (or universal)2/2earned
- License2/2earned
- Verified public repository2/2earned
06Platforms
07Labels