Relationships
#4590 Lazy loading L0: startup benchmark and CLI parity scripts
Opened by zack · 11/10/2026· Shipped 11/10/2026
First step of the lazy command loading plan. Every later step has to show two things the same way: what it did to startup cost, and that it changed no CLI output. Doing either by hand per PR is slow and easy to get wrong.
Context
Measured on a prototype (compiled, warm), routing argv to the invoked command plus build-time bundling takes swamp version from 384 ms / 219 MB to ~111 ms / 109 MB, data get from 416 to ~132 ms, workflow run from 498 to ~358 ms and --help from 391 to ~377 ms. Every PR in that plan quotes this benchmark before and after, and runs this parity check against a binary built from its base commit.
Scope
scripts/bench_startup.ts, with adeno run bench-startuptask. Given a binary path, it runs a fixed command set in a scratch repo it creates and removes:version--helphelp data getdata getbehavior method runworkflow run(a one-stepcommand/shellworkflow)- the telemetry sender entry
- It reports cold and warm medians of at least 5 runs: wall time and peak RSS. Cold means deleting
$TMPDIR/deno-compile-<binary-name>.cachebefore each run. Peak RSS comes fromwait4rusage: spawn withposix_spawnand readru_maxrssfromos.wait4. The script can callpython3 -I, or use a Deno FFI shim if a Deno-only tool is preferred. scripts/cli_parity.ts, with adeno run cli-paritytask. Given two binaries, it compares stdout, stderr and exit code for:--help,help,help --include-hidden<path> --helpandhelp <path>for every command path in the first binary'shelp --include-hiddenoutputcompletions bash|zsh|fish- a mistyped word at the root and in every group (the suggestion list)
version --json
SWAMP_NO_TELEMETRY=1andSWAMP_NO_UPDATE_CHECK=1are set for every run of either script.- Document both in
contributing/, including when to rerun them: every PR in the lazy-loading plan, and any Deno version bump once the bundled build ships.
Not in scope
CI gating on the numbers.
Acceptance
- The benchmark prints a table for a given binary.
- Two binaries with different file names do not share a code cache.
- The parity script exits non-zero on a difference and prints the first differing invocation. Run against two builds of the same commit, it passes.
- Unit tests cover argument parsing, median calculation and the diff report, with no subprocesses.
Release impact
Only scripts/ and contributing/ change, so affectedCommands is empty and nothing ships in the binary. Risk: none.
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.