Skip to main content
← Back to list
01Issue
FeatureShippedSwamp CLI
Assigneesstack72

#223 Implement W1b: Repository and RowState (extension catalog rearchitecture)

Opened by stack72 · 5/4/2026· Shipped 5/4/2026

Problem

Second of two PRs for the extension catalog rearchitecture (parent issue #211). W1a shipped in systeminit/swamp#1292 — it landed the schema migration, the state column, and migrated all writers + readers from validation_failed to state. W1b finishes the rearchitecture by introducing the domain aggregate, the repository abstraction, the cold-start guard unification, and dropping the now-vestigial validation_failed column.

W1a deliberately shipped without the domain VOs / ExtensionRepository / loader-guard unification — those are W1b. Until W1b lands, the schema sits with a dead validation_failed column and no production code reads or writes it.

Full architectural plan: see the v6 working spec produced during W1a's implementation (issue spec → v5 (approved) → discovered constraints during W1a → v6). Authoritative source for W1b's scope is the W1b section of /tmp/plan-issue-211-v6.yaml; the broader design lives in design/extension-rearchitecture.md (currently in the rearchitect-extension-loaders worktree).

Scope

Single PR. Implements the deferred-from-W1a items.

Domain layer (src/domain/extensions/)

  • New helper: findRepoRoot(start) — lexical-only ancestor walk (no realpath), innermost .swamp/ wins for nested worktree-in-repo cases, throws typed RepoRootNotFoundError if no match. Four test fixtures: standard, root-termination with idempotence break, nested-.swamp/ innermost-wins, symlink no-realpath.
  • Value objects: SourceLocation (canonicalPath + extensionRoot + relativePath; equality by canonicalPath), BundleLocation (canonicalPath + fingerprint), RowState (hand-rolled discriminated union, 7 tags: Indexed/Bundled/BundleBuildFailed/ValidationFailed/EntryPointUnreadable/OrphanedBundleOnly/Tombstoned). RowState requires a literal markdown state-machine table in the source-file's module-level comment per architect direction. SourceFingerprint as type alias (sha-256 hex or MISSING:<hash>).
  • Source entity (child of Extension, identity = SourceLocation) and Extension aggregate root (keyed (name, version)). Aggregate enforces invariants I1–I5 (extensionRoot match, intra-extension (kind, type) uniqueness with local > source-mounted > pulled precedence, ValidationFailed retains fingerprint+bundle, Tombstoned excluded from registration, sources map matches disk walk). Transitions: observeFreshSource, recordBundled, recordBundleBuildFailed, recordValidationFailed, recordEntryPointUnreadable, markSourceMissing, recordSourceMissing, tombstoneAll(): Extension (returns NEW immutable instance).
  • Local-extension synthetic aggregate: ONE @local/<basename(repoRoot)> per repo, version 0.0.0, owns every Source under every extensions/<kind>/ tree (including source-mounted external dirs — the W1a heuristic already rolls those into the same aggregate).

Infrastructure layer (src/infrastructure/persistence/)

  • ExtensionRepository — sole gateway to bundle_types after this PR. Constructor takes injected getLockedVersion(name): string | null dependency for the lockfile-backed empty-version fallback. Composition over the existing ExtensionCatalogStore.
  • API: loadAll(), loadByName(name), save(extension), saveAll(extensions[]), invalidationGuards(kind), invalidateAll(). save(ext) is sugar for saveAll([ext]); both share I-Repo-1 evaluation. I-Repo-1 (global (kind, typeNormalized) uniqueness across non-Tombstoned Sources) fires on EVERY commit, not just saveAll commits. Diff-based persistence inside SQLite transactions. Violation → ROLLBACK + throw DuplicateType naming both source paths.
  • Fallback derivation for empty-identity rows (W1a deliberately leaves identity columns empty for pulled rows + leaves identity for new-rows-from-loaders empty). Two cases:
    1. Pulled rows: extension_name populated, extension_version empty → consult getLockedVersion(name). If lockfile entry exists, write back. If null (orphaned), log warning + skip + DELETE.
    2. New rows from loaders post-W1a: both empty → call deriveExtensionIdentity (already in W1a). On null, skip + DELETE. On success with empty version (pulled), apply case 1's lockfile fallback.
  • Self-healing write-back (UPDATE bundle_types SET extension_name=?, extension_version=? WHERE source_path=?) on successful derivation. Idempotent under concurrent invocations (SQLite busy_timeout serializes).

Catalog cleanup

  • Drop validation_failed column via SQLite recreate-table pattern (architect-required, not raw ALTER TABLE DROP COLUMN):
    1. CREATE TABLE bundle_types_new without validation_failed
    2. INSERT INTO bundle_types_new SELECT (all other columns) FROM bundle_types
    3. DROP TABLE bundle_types; ALTER TABLE bundle_types_new RENAME TO bundle_types
    4. CREATE INDEX statements for idx_bundle_types_kind, idx_bundle_types_extends, idx_bundle_types_type (must be explicitly recreated; verify via sqlite_master)
  • All wrapped in one transaction with ROLLBACK on any error.

Loader changes

  • Migrate model loader's legacy global source_dirs_fingerprint key to per-kind model (currently user_model_loader.ts:551/634); the four sibling loaders already use per-kind. One-time rescan on first run after deploy.
  • Replace all 5 loaders' cold-start invalidation guards with a single repository.invalidationGuards(kind) call. Currently model loader has 3 guards (layout-version, datastore-base-path, source-dirs fingerprint); siblings have only source-dirs fingerprint. After W1b ALL FIVE have the same coverage — closes the audit's "model loader has four guards, four siblings have one" gap.

Callsite migration

  • Migrate forceCatalogRescan call sites: src/cli/commands/open.ts:109 and src/cli/commands/doctor_extensions.ts:105 to repository.invalidateAll().
  • DELETE the standalone forceCatalogRescan function from extension_catalog_store.ts:494-510. Preserve best-effort error semantics in invalidateAll() (swallow on missing/corrupt DB so open/doctor don't crash).

Pre-work decisions (already pinned in W1a, restated for the W1b PR description)

  • Path canonicalization rule, repo-root identification rule, unmatched-row backfill behaviour — see W1a PR description.
  • W1b is forward-only on revert: dropping validation_failed leaves the schema without the column; reverting the W1b binary leaves old loaders reading raw.validation_failed === 1 against undefined → false → broken rows leak through. Revert path requires deleting _extension_catalog.db (manual ops).

Out of scope (deferred to later workstreams)

  • extension rm row pruning / lifecycle services owning catalog writes → W2 (this is what closes #201)
  • Cross-extension DuplicateType errors at lifecycle save time → W2
  • ReconcileFromDisk service / freshness-as-aggregate-query / removing UNREADABLE_DEP_SENTINEL → W3
  • Loader unification / KindAdapter → W4
  • Per-fingerprint import URLs and subprocess test harness → W5
  • swamp doctor extensions reading aggregate state → W6

Specific edge cases for W3, NOT W1b

  • Two pulled versions of the same extension coexisting on disk (interrupted upgrade): both rows backfill to the same extension_name; lockfile fallback assigns the same extension_version to both rows; I-Repo-1 then fires when the repository loads. That is the correct error in a corrupt state. Repair (drop the stale subtree, re-derive from lockfile) belongs to W3's ReconcileFromDisk. W1b's fallback MUST NOT try to handle this — let the I-Repo-1 violation surface.

Success criteria

  • Extension, Source, RowState, SourceLocation, BundleLocation exported from src/domain/extensions/.
  • ExtensionRepository round-trips aggregates with diff-based saves.
  • saveAll([vN.tombstoneAll(), vN+1]) succeeds when both versions ship the same (kind, type) (the upgrade-as-atomic-transition canary).
  • Two extensions with overlapping (kind, type) rejected with a DuplicateType event naming both source paths and ROLLBACK.
  • All four sibling loaders' cold-start invalidation matches the model loader's coverage.
  • pragma_table_info('bundle_types') shows no validation_failed column post-PR.
  • sqlite_master shows all three indexes after the recreate-table drop.
  • All existing tests pass on Linux and macOS. Windows is NOT a W1b merge gate (parallel workstream).

Suggested test additions

  • Repository diff-save: add Source → INSERT; drop Source → DELETE (label this test as the swamp-club#201 reproducer at the repository layer); transition Source state → UPDATE.
  • saveAll atomicity: tombstone-and-replace + DuplicateType rollback paths.
  • invalidationGuards triggers on each of layout-version / datastore-base-path / source-dirs-fingerprint changes for ALL FIVE kinds.
  • invalidateAll on missing / corrupt DB does not throw.
  • Recreate-table drop preserves indexes (sqlite_master post-condition).
  • Empty-identity row fallback (success + null-return paths).
  • Cross-platform SourceLocation equality with explicit fixture pair like EXTENSIONS/Models/A.tsextensions/models/a.ts.

Pointers

  • Parent issue: #211
  • W1a PR: systeminit/swamp#1292
  • v6 plan working spec (W1b section): /tmp/plan-issue-211-v6.yaml
  • Design doc: design/extension-rearchitecture.md (untracked in .claude/worktrees/rearchitect-extension-loaders/)
  • Files W1a left for W1b: validation_failed column in extension_catalog_store.ts, forceCatalogRescan standalone helper, per-loader cold-start guards (model loader has 3, siblings have 1)

Release-notes framing

"Extension catalog rearchitecture: W1b — domain aggregate + repository abstraction." Pure plumbing; closes the audit's 4-vs-1 cold-start guard gap as the only user-visible improvement this workstream ships.

02Bog Flow
OPENTRIAGEDIN PROGRESSSHIPPED+ 1 MOREASSIGNED+ 7 MOREREVIEW+ 3 MOREPR_MERGEDSHIPPED

Shipped

5/4/2026, 9:05:07 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
stack72 assigned stack725/4/2026, 4:39:11 PM

Sign in to post a ripple.