Relationships
#918 model evaluate is slow resolving data.latest expressions in large datastores
Opened by evrardjp · 7/2/2026· Shipped 7/2/2026
Summary
swamp model evaluate can spend ~30-35s at expression evaluation for a single model that contains only a few data.latest(...) expressions, even when reading the referenced data directly takes ~1s.
This appears to be a performance bug in model expression evaluation: resolving a targeted data.latest(modelName, dataName) expression seems to pay a broad datastore/catalog scan cost, possibly repeated per expression, rather than using the direct data lookup path.
Reproduction
A model has global arguments like:
globalArguments:
host: ${{ data.latest("lab-vm-pool", "vaultwarden").attributes.ipAddress }}
user: ${{ data.latest("lab-vm-pool", "vaultwarden").attributes.bootstrapSSHUser.username }}
hostKeyPath: ${{ data.latest("lab-vm-pool", "vaultwarden").attributes.sshHostCertificate.hostKeyPath }}Timing the evaluation:
time swamp model evaluate vaultwarden-host-key --jsonObserved: about 30-35 seconds, with high memory usage observed around 2GB RSS in one run.
But the directly referenced data is fast to retrieve:
time swamp data get lab-vm-pool vaultwarden --json >/tmp/vw.jsonObserved: about 1 second.
The repository has a large .swamp/data tree, including multi-GB data for unrelated models. The referenced lab-vm-pool/vaultwarden data itself is small.
Expected behavior
data.latest("lab-vm-pool", "vaultwarden") during model evaluation should use an indexed/direct lookup comparable to swamp data get lab-vm-pool vaultwarden, and should not scan or load unrelated data. Multiple references to the same data item in one model evaluation should also ideally be cached within the evaluation.
Actual behavior
swamp model evaluate takes ~30-35s at expression evaluation, while the target data lookup takes ~1s.
Environment
Current swamp version:
20260630.112427.0-sha.0e5078e9swamp update --check --json reports a newer version is available (20260702.171037.0-sha.ca29630a), but source-version comparison was inconclusive because swamp source fetch --version 20260702.171037.0-sha.ca29630a failed in this environment with Read-only file system (os error 30): mkdir '/var/home/<USERNAME>/.swamp/source'.
Shipped
Click a lifecycle step above to view its details.