Relationships
#3300 Datastore rework: on v3, turning managedConfig on hides existing config (the default vault); give v3 its own config migrate
Opened by stack72 · 10/12/2026· Shipped 10/12/2026
Background (read this first)
swamp is reworking its datastore layer (tracking: swamp-club#2865). The new format, v3, records every change as a commit in a single-writer SQLite commits.db. Content is stored as chunks, and each checkout's local files and _catalog_v3.db are rebuilt from the commits by a read model that catches up when a context opens.
swamp-club#3279 made v3 usable for new, empty filesystem datastores with a single writer, through swamp datastore setup filesystem --path <dir> --format 3. It's released from 20261012.143252.0-sha.cc63f3a8. The design is in design/enablers/datastores.md, under the "v3 …" sections and the opt-in section.
Facts about v3 that this issue depends on:
- Repositories still write local files, and also stage typed changes into a commit-log unit. A commit lands when the unit ends.
withUnitOfWork(src/libswamp/unit_of_work.ts) ends a v3 unit by decision 5:- it commits on
completed,suspended,cancelledand a reportederror; - it abandons on a throw or an early stop.
- it commits on
- A second checkout joins as a git clone does: the same
.swamp.yamland repo id, and a fresh.swamp/. Its first command catches up. - v2 must not change. Any shared code path needs the v2 gate that #3240, #3250 and #3279 used: v2 suites unedited, and a whole-suite route recorder that's identical before and after.
Found by swamp-uat#619's filesystem-v3 backend (PR swamp-uat#623). That suite fails on this bug today, on purpose.
The bug
Turning managedConfig on in a v3 repo hides repo init's default vault, in the original checkout too.
swamp-uat#623's failing test is "swamp datastore managedConfig: models, workflows and vaults created on clone A appear on clone B with identical content [filesystem-v3]".
Why.
- Since swamp-club#3259,
repo initwrites a defaultlocal_encryptionvault config into the repo. - With managedConfig on, vault configs (and model and workflow definitions) are read from the datastore's config tier.
- On v2,
swamp datastore config migrateturns managedConfig on and copies the repo's existing definitions, workflows and vault configs into the config tier. - On v3,
config migrateis refused (#3279). The UAT turns managedConfig on by settingdatastore.managedConfig: truein.swamp.yaml, as #619's helper does on v3. Nothing copies the existing config, so the default vault, and any existing definitions, workflows or vault configs, disappear from view.
Goal
A v3 repo can turn managedConfig on and keep everything it already has. Its existing model and workflow definitions and its vault configs (the init default included) are committed into the config tier and appear on every clone. That matches v2's config migrate.
Design
Put the fix in the
config migratepath. Give v3 its owndatastore config migrate, instead of refusing it:- it commits the repo's existing definitions, workflows and vault configs into the config tier, through the commit-log unit and #3256/#3257's
definitionSaveBatches, as managed-config definition partitions (config--{kind}--{name}); - it then sets managedConfig;
- don't change
datastore setup: the parallel namespace issue changessetup.ts, and keeping this fix out of it lets both run at once.
- it commits the repo's existing definitions, workflows and vault configs into the config tier, through the commit-log unit and #3256/#3257's
Turning managedConfig on by hand. Decide what happens when a user sets
datastore.managedConfig: truein.swamp.yamlthemselves on v3, as the UAT does. Either:- refuse it, with a message pointing at
config migrate; or - adopt the existing repo config on the first writing command.
Record the choice and why. Don't leave it silently hiding config.
- refuse it, with a message pointing at
Idempotent and safe. Running it twice changes nothing. A clone that turns it on after another clone already did doesn't overwrite the config tier: the precondition on edits refuses a stale write.
Secrets are never committed; only vault configs are, as #3257 established.
v2's
config migratedoesn't change. Run the v2 gate.
Tests
- On a v3 filesystem datastore, in-process:
- a repo with the init default vault, a model, a workflow and a second vault turns managedConfig on;
- all of them stay visible in that checkout, and a second checkout sees identical content after catch-up;
- a re-run is a no-op;
- a concurrent second clone's stale write is refused.
- The by-hand case, per your decision in design step 2.
- The v2 gate.
- When this ships, swamp-uat#623's managedConfig test must pass against the release.
Out of scope
datastore setup(the namespace issue changes it).- S3 and GCS v3.
Blocked by: nothing. It runs in parallel with the v3 SIGTERM and v3 namespace issues.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.