#3256 Datastore rework Phase 3: definition, workflow and evaluated repositories stage typed changes on v3 (no v2 behaviour change)

Background (read this first)

swamp is reworking its datastore layer (tracking: swamp-club#2865). In the target design, v3, every change is an explicit commit in commits.db, made through a commit-log unit of work from typed changes: an intent that says what the operation is, plus its content. Phase 3 builds this dormant: nothing reaches it until a user opts a new, empty datastore into v3.

What's on main (897b64a0), documented in design/enablers/datastores.md:

• The shared typed staging route (swamp-club#3250, src/infrastructure/persistence/unit_of_work_scope.ts):

  • signalChange(markDirty, change, { typed: true }). It keeps today's path change, which is skipped only when a typed stager is ambient.
  • stageTyped(markDirty, build). Called after the write, it runs build only under a commit-log unit bound to the hook, and stages the result as one batch, which lands in one commit. On v2 it returns after one lookup and never runs build. On a v3 hook with no bound unit, it throws UnscopedWriteRefusedError.
  • typedStagingActive(markDirty). It gates any read a typed change needs before the write destroys it, for example the digest of a file being replaced, so v2 reads nothing extra.
  • The model. unified_data_repository.ts and yaml_output_repository.ts already stage this way, through ModelChanges (src/domain/datastore_v3/repository_changes.ts). Follow that pattern.

• The domain model (swamp-club#3233, src/domain/datastore_v3/): partitionOf(), createIntent, intentPartition, the ops and MAX_COMMIT_OPS (10,000 changes per partition per batch). Intents include:

  • run.create, run.update and run.delete, plus run-snapshot.save and run-snapshot.delete, all on run--{workflowId}--{runId};
  • definition.create, definition.edit and definition.delete, plus definition.rename-to and definition.rename-from, and evaluated-workflow.save and evaluated-workflow.delete, on config--{model|workflow|vault}--{name};
  • evaluated-definition.save and evaluated-definition.delete, on the model partition;
  • auto-definition.save, on definition--{id}.

  A definition rename spans two partitions, committed in order: a definition.rename-to put with expect: "absent" in the new name's partition, then a definition.rename-from tombstone with expect: "present" in the old one.

• The commit-log unit (swamp-club#3240, #3249, #3250):

  • one commit per partition per batch, with run partitions last;
  • retries keep the txnId;
  • a post-commit chunk check re-stores missing chunks from the content source;
  • staged() is [] on v3, so read stagedOps().

• The read model (swamp-club#3239): applies commits to a second directory and to _catalog_v3.db. #3250's end-to-end test compares the files it rebuilds with what the repositories wrote. Reuse it: integration/datastore_v3_repository_staging_test.ts.

• The Phase 3 decision: on v3, repositories keep writing local files exactly as today and also stage typed changes.

This issue is one of two repository staging groups that build on #3250's route. The other is run and vault config, filed in parallel. They touch different repositories.

Goal

Make the definition, workflow and evaluated repositories stage typed changes on v3 for every write they make:

• yaml_definition_repository.ts;
• yaml_workflow_repository.ts;
• yaml_evaluated_definition_repository.ts;
• yaml_evaluated_workflow_repository.ts.

v2 doesn't change at all.

Design

1. Which writes are datastore writes. Definitions and workflows are datastore-tier only under managed config. Without it they live in git and their repositories have no hook. Auto-definitions and evaluated definitions and workflows are datastore-tier. Confirm from createRepositoryContext which hooks each repository gets, and stage only where today's code calls signalChange.
2. Mapping. Use these intents:
   • create, edit and delete of a managed-config definition: definition.* in config--{kind}--{name}. An edit carries expect: { sha256 } of the version the write replaced, read through typedStagingActive before the write;
   • auto-definitions: auto-definition.save in definition--{id};
   • evaluated definitions: evaluated-definition.* in the model partition;
   • evaluated workflows: evaluated-workflow.* in the definition's partition.
   • Renames (vault edit can rename, and so can YAML cleanups like cleanupOldPaths and pathsToTry) are two batches, in order, as the Background describes. Each of today's bulk cleanups becomes explicit per-key tombstones. Record every mapping you add in datastores.md.
3. A crash between the two rename commits leaves both names until a repair. Readers resolve the definition by its id. Test that state: the new name exists, the old one still does, and nothing is lost.
4. Concurrent edits. A stale edit, whose expect sha differs, is refused by the partition (expect-failed). The repository surfaces that as an error the caller can show, not a silent overwrite. On v2 nothing changes.

The gate

v2 doesn't change:

• every v2 suite (usecase characterization, root-unit, dirty-coverage, peer-propagation, the unscoped-write guard, the routing test) passes with no expectation edits;
• a whole-suite route recorder shows identical route-1 and route-2 writes before and after (same caller, change kind and test). #3240 and #3250 did this, and their PRs describe the approach;
• the builders never run on v2.

v3 is right end to end. For every write path:

1. run the repository on v3 in a commit-log root;
2. commit to a real commits.db and filesystem storage/1;
3. catch up a read model into a second directory;
4. assert that its files (bytes) and _catalog_v3.db rows equal what the repository wrote.

Cover namespaced and unnamespaced datastores. Add mutation or "red" checks for the rules you add, as #3250 did: drop a staged change and show a test fails.

Out of scope

• The run and vault config repositories (the parallel issue).
• Making repositories stop writing files.
• The opt-in, and managed-config serve wiring.

Done when

• These four repositories stage typed changes on v3, with the gate's evidence in the PR.
• Mappings, including renames and cleanups, are recorded in datastores.md.
• All v2 suites pass unchanged.

Blocked by: nothing (#3250 has merged). It runs in parallel with the run and vault config staging issue.