Gcs Datastore

Store data in a Google Cloud Storage bucket with local cache synchronization. Provides distributed locking via GCS generation-based preconditions and bidirectional sync between a local cache directory and GCS. Emits OpenTelemetry spans for GCS operations, lock acquisition/release, and push/pull sync so datastore activity is visible in traces when a TracerProvider is configured. Features shard-first indexing with per-model partition shards under _index/ as the source of truth (v2 repos skip the monolithic index upload entirely), commitSeq-based zero-diff fast path, two-phase sync to narrow the global-lock critical section, SHA-256 content hashing, per-path dirty tracking, namespace-scoped sync for multi-repo shared datastores, configurable transfer concurrency, and configurable per-request timeout.

Authentication

Uses Google Cloud Application Default Credentials (ADC) — no credentials in config. Provide credentials via one of:

• Environment variable: GOOGLE_APPLICATION_CREDENTIALS pointing to a service account key JSON file
• User credentials: gcloud auth application-default login
• Attached service account on GCE, Cloud Run, or GKE

Required IAM Permissions

• storage.buckets.get
• storage.objects.create
• storage.objects.get
• storage.objects.delete
• storage.objects.list

The predefined role roles/storage.objectAdmin covers all of these.

Usage

swamp datastore setup @swamp/gcs-datastore \
  --config '{"bucket": "my-bucket", "prefix": "swamp"}' --json