Relationships
#1020 swamp serve startup/shutdown is very slow on large repo and lacks diagnostics
Opened by evrardjp · 7/7/2026· Shipped 7/8/2026
Summary
swamp serve is extremely slow to start on my repository/data, consumes very high memory/CPU, and there does not appear to be a way to see what it is doing during startup.
I tried running with swamp serve -v, but it did not reveal the slow phase or explain where the time/memory goes.
Environment
- swamp version:
20260706.220403.0-sha.72b7ed98 - repository:
/var/home/evrardjp/git/evrardjp/local-infra .swampsize: ~9.1G- model count from
swamp model search --json: 47 results
Observed behavior
Startup takes minutes before useful output appears. Example run:
❯ swamp serve
19:51:43.232 warning serve auth-mode is 'none' — this mode is deprecated and will be removed in a future release. Use --auth-mode token for authenticated access. See https://swamp-club.com/manual/how-to/swamp-serve/set-up-token-auth
19:51:43.233 info serve Initializing repository at "/var/home/evrardjp/git/evrardjp/local-infra"
19:51:43.317 info swamp·domain·access·policy-snapshot Loaded policy snapshot: 0 active grant(s), 0 group(s)
19:51:43.317 info serve Policy snapshot loaded (reload mode: "manual")
19:52:08.605 warning serve Reaping orphaned workflow run "fc497990-5ed4-4066-b44f-10dd333c8f68" (workflow: "github-project-stale-classification-daily")
19:55:32.427 info serve Scheduled workflow "cluster-pool-sync" ("*/5 * * * *")It takes more than 3 minutes to output the first scheduled-workflow line after the orphan reaping log. Regardless of the schedule, here it is 5 minutes, but it would happen if it is 1 minute.
Side info: My machine is starting to struggle, see CPU spiking:
4190633 evrardjp 20 0 22.4g 2.9g 4960 D 74.7 6.1 1:38.25 swamp .Notes:
- CPU was ~74% in this sample.
- starting is very slow.
- closing is also very slow.
Expected behavior
swamp serveshould either start in a reasonable time on a large existing repo or clearly identify the slow startup phase.- Verbose logging should make it possible to diagnose what is happening: which startup step is running, how many records/files are being scanned, per-phase timing, maybe memory/heap metrics.
- If this is expected for large datasets, there should be a documented way to profile/trace startup or identify the dataset/model/workflow causing the slowdown.
Request
Is there a way to track what swamp serve is doing during startup/shutdown today? If not, please add diagnostics/progress logging or a tracing/profiling mode for swamp serve startup/shutdown so users can understand and report actionable bottlenecks.
Shipped
Click a lifecycle step above to view its details.
evrardjp commented 7/7/2026, 8:49:53 PM(edited)
Interesting that I used my agent to generate this issue, but the format of this does NOT seem to match the usual issues expectations (swamp version, etc.).
Feel free to close if it's not correct.
evrardjp commented 7/7/2026, 8:51:57 PM
Oh, well, I guess that's it:
<--- Last few GCs --->
[10545:0x565482d8b000] 261167 ms: Mark-Compact (reduce) 4092.1 (4100.4) -> 4092.1 (4100.4) MB, pooled: 0.0 MB, 415.14 / 0.00 ms (average mu = 0.005, current mu = 0.000) last resort; GC in old space requested
[10545:0x565482d8b000] 261585 ms: Mark-Compact (reduce) 4092.1 (4100.4) -> 4092.1 (4100.4) MB, pooled: 0.0 MB, 417.26 / 0.00 ms (average mu = 0.002, current mu = 0.000) last resort; GC in old space requested
#
# Fatal JavaScript out of memory: Reached heap limit
#
==== C stack trace ===============================
swamp(+0x23efc5e) [0x565448c54c5e]
swamp(+0x23eec7b) [0x565448c53c7b]
swamp(+0x23f0ec8) [0x565448c55ec8]
swamp(+0x249fb75) [0x565448d04b75]
swamp(+0x2656487) [0x565448ebb487]
swamp(+0x2655bb4) [0x565448ebabb4]
swamp(+0x265537d) [0x565448eba37d]
swamp(+0x2648d9b) [0x565448eadd9b]
swamp(+0x264873c) [0x565448ead73c]
swamp(+0x2626f6c) [0x565448e8bf6c]
swamp(+0x261d471) [0x565448e82471]
swamp(+0x27876d3) [0x565448fec6d3]
swamp(+0x278ad7f) [0x565448fefd7f]
swamp(+0x278ad9b) [0x565448fefd9b]
swamp(+0x278a906) [0x565448fef906]
swamp(+0x278d580) [0x565448ff2580]
swamp(+0x278ad8c) [0x565448fefd8c]
swamp(+0x27872ff) [0x565448fec2ff]
swamp(+0x2786fb0) [0x565448febfb0]
swamp(+0x24ec505) [0x565448d51505]
[0x7f075be81ef6]
Trace/breakpoint trap (core dumped) swamp serveevrardjp commented 7/8/2026, 12:09:02 PM
I confirm the start/stop is faster now.
Sign in to post a ripple.