#2000 Datastore-backed model catalog: register model definitions via the datastore, not only on-disk yaml

Opened by bixu · 9/4/2026

The gap

Model DEFINITIONS live in the top-level models/**/*.yaml on disk. The datastore already covers model DATA (versions, workflow runs, outputs). But model definitions are boot-time-static. A running swamp serve reads them at startup. It does not pick up new ones written to /repo afterward.

Impact on our deployment (org agent-harness)

The k3s pod for our harness writes new factory instance yamls at runtime, one per Linear ticket, into /repo/models/@swamp/software-factory/platform-team-plt-XXXX.yaml. The running serve never sees the new file until the next restart. This forces:

emptyDir for /repo → files vanish on pod restart (durability bug).
Fix: PVC-back /repo (durability) + init container re-registers all yaml on boot.
PVC is ReadWriteOnce → forces replicas=1 (no HA).
ReadWriteOnce + RollingUpdate incompatible → forces strategy: Recreate (~30-90s window on every helm upgrade).

Every downstream constraint traces back to disk-only model definitions.

Ask

Extend the datastore extension interface (currently @swamp/s3-datastore, @swamp/filesystem-datastore) with a model-catalog surface:

registerModel(type, name, definition)
readModelDefinition(type, name)
listModelDefinitions(type)
unregisterModel(type, name)

swamp serve reads its model catalog from the datastore. swamp model edit/create writes there. A change on one replica reaches others.

Benefits

After a pod restart, runtime-provisioned models remain reachable.
Running many replicas of serve becomes viable. Each replica reads the same catalog from S3.
strategy: RollingUpdate maxSurge=1 maxUnavailable=0 becomes valid. Zero-downtime helm upgrades follow.
Removes the emptyDir/PVC/Recreate cascade entirely.

Backward compat

Yaml on disk stays supported. The datastore backs it, does not replace it. Small repos with filesystem datastore keep today's behavior.

02Bog Flow

Closed

9/4/2026, 6:05:52 PM

No activity in this phase yet.

03Sludge Pulse

bixu moved this from the private collective hivemq to public9/4/2026, 3:28:45 PM

stack72 assigned stack729/4/2026, 5:32:47 PM

stack72 commented 9/4/2026, 6:05:51 PM

@bixu Two concerns with the proposed approach:

Auto-reading files from the filesystem is a risk. If serve watches models/ and automatically loads any YAML that appears, anyone with write access to that volume can inject model definitions into a running serve instance — or crash it with malformed YAML. Definitions reference model types (extension code that executes), CEL expressions, and vault secrets — that's a significant trust boundary to leave open to filesystem writes that we don't want to open at this time.

The surface area for runtime definition registration already exists. swamp model create --server creates definitions through serve's authenticated WebSocket API with full access-control evaluation. Definitions created this way are immediately visible to the running serve — no restart needed. Combined with managedConfig: true (via swamp datastore config migrate), definitions persist to the datastore tier (S3), giving you durability across pod restarts and shared state across replicas.

For the k3s deployment described in the issue, we suggest the harness call swamp model create --server instead of writing YAML files to /repo/models/. With managedConfig and an S3-backed datastore, definitions persist across pod restarts and are shared across replicas — removing the emptyDir durability issue and the replicas=1 constraint.

bixu commented 9/4/2026, 7:29:41 PM

I’ll work on it from the suggested angle. I’m not sure why my agents didn’t pick up on this pattern from skills or docs — will try to figure that out.