Gcs Datastore
Store data in a Google Cloud Storage bucket with local cache synchronization. Provides distributed locking via GCS generation-based preconditions and bidirectional sync between a local cache directory and GCS. Features shard-first indexing with per-model partition shards under _index/ as the source of truth (v2 repos skip the monolithic index upload entirely), commitSeq-based zero-diff fast path, two-phase sync to narrow the global-lock critical section, SHA-256 content hashing, per-path dirty tracking, namespace-scoped sync for multi-repo shared datastores, configurable transfer concurrency, and configurable per-request timeout.
Authentication
Uses Google Cloud Application Default Credentials (ADC) — no credentials in config. Provide credentials via one of:
- Environment variable:
GOOGLE_APPLICATION_CREDENTIALSpointing to a service account key JSON file - User credentials:
gcloud auth application-default login - Attached service account on GCE, Cloud Run, or GKE
Required IAM Permissions
storage.buckets.getstorage.objects.createstorage.objects.getstorage.objects.deletestorage.objects.list
The predefined role roles/storage.objectAdmin covers all of these.
Usage
swamp datastore setup @swamp/gcs-datastore \
--config '{"bucket": "my-bucket", "prefix": "swamp"}' --jsonStore data in a Google Cloud Storage bucket with local cache synchronization.
Config Fields
| Field | Type | Description |
|---|---|---|
| bucket | string | GCS bucket name |
| prefix? | string | Object name prefix within the bucket |
| projectId? | string | GCP project ID (defaults to ADC project) |
| apiEndpoint? | string | Custom API endpoint URL (for emulators like fake-gcs-server) |
| pullConcurrency? | number | Maximum concurrent GCS downloads during pull. Default: 50 |
| pushConcurrency? | number | Maximum concurrent GCS uploads during push. Default: 25 |
| requestTimeoutMs? | number | Per-request timeout in milliseconds. Default: 30000. |
Modified 1 datastores
Modified 1 datastores
- Has README or module doc2/2earned
- README has a code example1/1earned
- README is substantive1/1earned
- Most symbols documented1/1earned
- No slow types (deprecated)1/1earned
- Dependencies pass trust audit2/2earned
- Has description1/1earned
- Platform support declared (or universal)2/2earned
- License declared1/1earned
- Verified public repository2/2earned