Music Library
Multidimensional music library catalog built WITHOUT touching the
filesystem: the inventory and raw tags come from an existing gonic scan
index (gonic.db, read over SSH with sqlite3 -json -readonly), so no
array disk is ever woken by a directory traversal.
On top of the raw rows the scan method builds a star-schema cube:
- facts — one
albumresource per album directory (album → disc → track nesting, disc subdirs likeCD1/collapsed into their parent album) - dimensions — one
artistresource per artist (name variants, album refs, genre/format/year span) plus rollupdimensionresources for genres, years/decades, formats, and quality buckets (lossless / lossy-high / lossy-mid / lossy-low) - cross-cuts — an
issuesresource (untagged tracks, dirname-only albums, encoding fixes with before/after, DOS-mangled 8.3 names) and alibrarysummary with totals and source breakdown
Every tag string passes through encoding recovery: legacy single-byte tags decoded as latin1 (cp1251 / koi8-r / cp866 / windows-125x / CJK mojibake such as "Êëàóäèî Ìîíòåâåðäè") are detected with jschardet and re-decoded; double-encoded UTF-8 is unwrapped. Tracks with missing tags fall back to directory/filename naming patterns ("1983. Artist - Album", "Artist - Album (Year) [FLAC]", "NN - Title", disc subdirs, bracketed artists, catalog-number prefixes).
The probe method deep-probes one file with ffprobe inside a docker
container that has the music share mounted (all formats ffmpeg knows:
mp3, flac, opus, m4a, ogg, ape, wma, wav, …), merging format-level and
stream-level tags and reporting the encoding-fix trace — for debugging
individual files.
The bpm method analyzes tempo per track with essentia
(RhythmExtractor2013 multifeature) in a throwaway container per worker,
reporting bpm, beat-detection confidence, beat-interval spread, key/scale
and danceability. The confidence matters more than the bpm: the beat
tracker imposes an even grid even on rubato or ambient material, so a
track can look metronomic while having no pulse you could follow. Results
carry over between runs, so a whole-library pass can be resumed.
Workflows: ships @magistr/music-wanted-sequence, the gated resolve ->
sync -> derive sequence behind the "wanted" derivation. Installing this
extension does NOT make it runnable: swamp extension pull /
extension source add do not register a workflow file, so nothing under
this name exists until you create it yourself with swamp workflow create and paste the body in from
extensions/workflows/music-wanted.yaml. See that file's own
description for the eight gates, the required dry run, and the portable
three-command form.
2026.08.07.1
Fixes music-wanted-workflow-packaging, then hardens it against four
code-review findings. model.version/manifest.yaml move 2026.08.05.2 ->
2026.08.07.1.
Ship the music-wanted workflow body as part of this package
Adds extensions/workflows/music-wanted.yaml (@magistr/music-wanted-sequence)
under a new manifest workflows: key, so a 464-line artefact that previously
existed in one copy only — in a homelab-repo tree with neither .git nor .jj
— gets version control and diffability. Installing this does NOT make the
workflow runnable: neither swamp extension pull nor extension source add
registers a workflow file, so nothing exists under this name until you create it
with swamp workflow create and paste the body in — stated in the manifest's
Workflows section, the file's own first paragraph, and this README.
The packaged copy also corrects claims the repo-local body carried, fixed in the
homelab copy too (outside this package): gate 6 (catalog-completeness) falsely
claimed "fails only on a cold catalog, passes once warm" — the only recorded dry
run (dc413fca, 2026-08-06) failed gates 4 AND 6, since resolve has no dryRun
guard and resolves for real while batchSize: 0 caches nothing; the stale
~775 / 775 of 2258 figures understated the last real run's cost (1165
resolved) by ~50%; and the unconditional ext-canary-nightly 02:30-03:15 rule
is replaced by the mechanism it was a special case of — a cold sync holds the
musicbrainz lock for the whole pass, and swamp model method run waits only
DEFAULT_LOCK_TIMEOUT_MS (60s) — dropped WITH THIS NOTE: ext-canary-nightly
hasn't run since 2026-07-27 (absent from five days of scheduler logs), so the
window guards nothing today, though the lock-contention hazard remains real. See
the musicbrainz package's own CHANGELOG for the dry-run control's original
documentation; it's now identical on both copies, not "repo-local" only.
wanted's missing-browse-cache throw no longer says "Repo-local"
The throw's last line named a "private homelab repo" that no longer holds the
only copy. It now names the shipped file and create-and-paste procedure, keeping
the substring swamp workflow run music-wanted (so
music_library_methods_test.ts:1982 keeps passing) plus a new
!includes("Repo-local") negative.
README and structural test
README: the "lives in that private repo" claim is replaced with the
shipped-but-not-runnable statement, the eight gates, the dry run's true expected
outcome (an INVARIANT, not a fixed tally — depends on how warm the catalog
already is), the two literal instance names and how to retarget them, and the
two-copy drift comparison. Gains
extensions/workflows/music_wanted_workflow_test.ts — the only automated
coverage for the packaged file (swamp workflow validate can't target a
workflow with no id:; extension push --dry-run validates no workflow at
all). Asserts the name, absence of a top-level id, all ten assert-step names
and gate-leading messages, allowFailure pinned across EVERY step not just
asserts (twelve false, one true), the job dependency edges
(derive<-sync<-resolve<-preflight, each {type: succeeded}), the three
model_method steps' literal targets, the corrected gate-6 sentence, and the
absence of 775 / 2026-08-06 / ext-canary-nightly.
Test-suite hardening (two mutation-testing rounds)
A stray trailing quote in the browse-throw regression needle never matched the
defect it names — fixed, verified against the mutation it exists to catch. A
code-review pass then found the structural test's allowFailure check filtered
to asserts only and never read job dependsOn, so flipping
sync-artist-discographies to allowFailure: true or severing derive's
dependency on sync both passed 9/0 uncaught (and both pass
swamp workflow validate, DAG-shape-only). Fixed per the assertions above; both
mutations verified RED, then reverted.
Deferred
Two follow-ups, both filed: music-wanted-stale-shared-strings (the stale
~775 figure at its remaining SIX sites — music-library/README.md:131 (this
package's own README), musicbrainz/README.md:134,
musicbrainz.ts:1761/:1773/:1819, music_library.ts:3894 — plus
musicbrainz/README.md:139's own stale "private homelab repo" pointer) and
music-wanted-headphones-instance-implicit (resolve passes only refresh to
resolve-artists, relying on its headphonesInstance default).
Recorded but NOT filed: musicbrainz/README.md:138-140 still says "do not start
a cold pass between 02:30 and 03:15" — the ext-canary-nightly window this
package's docs replaced with the lock-contention mechanism. A sibling package's
README outside this diff; neither issue tracks it, noted here so it isn't lost.
Global Arguments
| Argument | Type | Description |
|---|---|---|
| host | string | Host with the music share (unraid) |
| sshUser | string | SSH user |
Resources
Not runnable on install: neither `swamp extension pull` nor `swamp extension source add` registers a workflow file, so nothing under this name exists until you create it yourself. Create it with `swamp workflow create`, paste this file's body in, and invoke it under whatever name you gave it. Wires the three-step wanted sequence (resolve-artists -> sync-artist- discographies -> wanted) into one gated run, deliberately with NO `--fail-on`. The default fails the run on ANY assert, including the tw
Cadence-matched running playlist from the latest tempo analysis: tracks whose beat essentia actually locked onto, matched to a 150-190 spm cadence at 1×, 2× or ½×, bucketed by cadence and ranked by beat confidence, plus the most metronomic albums and what was excluded and why.
Triage of the latest playback-verification run: unreadable files, truncation split by cause, systematically damaged albums (with healthy-duplicate hints from the dupes resource), lossless corruption, and isolated glitches.
Gap report from the latest want-set: missing releases grouped by
2026.08.05.2
Fixes music-wanted-sequence-not-wired. model.version and manifest.yaml
move 2026.08.05.1 -> 2026.08.05.2.
wanted: the missing-browse-cache throw named a nonexistent method
wanted's missing-MusicBrainz-browse-cache error told the operator to run
swamp model method run <mbInstance> browse — there is no browse method
(browse-release-groups / browse-releases / browse-recordings exist;
browse is a resource spec name, not a method), so an operator who skipped the
discography sync got unknown_method instead of an actionable fix. The throw
now names the real runnable command —
swamp model method run <mbInstance> sync-artist-discographies --input 'artistMbids:json=[...]'
— plus the swamp data query extraction command (with its envelope shape,
{"results": [[...]], "total": 1}) to build that artist list from this
instance's own artistMap, and the repo-local music-wanted workflow line. No
schema or resource shape change.
2026.08.05.1
Fixes musicbrainz-ratelimit-runmodel-fanout, measured live:
swamp model method run <instance> resolve-artists completed in 581s having
fired ~1483 MusicBrainz searches over a library of 2258 artists — ~2.5 req/sec
against MusicBrainz's documented 1 req/sec limit. The root cause: mbFetch's
rate limiter (in @magistr/musicbrainz) is correct WITHIN one method
invocation, but resolve-artists reached MusicBrainz through ~1483 SEPARATE
context.runModel invocations of search-artist, each starting with no
rate-limit memory. model.version and manifest.yaml move 2026.08.04.1 ->
2026.08.05.1.
resolve-artists: at most ONE runModel call per run, persisted as a reusable cache
searchMusicBrainzArtists and its isLatest selector are DELETED outright — no
dual path. resolve-artists now collects every DISTINCT name needing a
MusicBrainz verdict (seed-unresolved, and either uncached or past its TTL),
builds one Lucene query per name via the existing escapeLuceneQuery, and
issues EXACTLY ONE call to @magistr/musicbrainz's new search-artists-batch
method — deterministically selecting its own result row by a generated batchId
correlation identity (never isLatest/array order, which nothing in this
codebase ever set). Zero names needing a search means zero runModel calls.
That call passes limit: 25 explicitly — the candidate window is the SAME 25
the deleted per-artist search-artist path implicitly got from MusicBrainz's
own /ws/2 default, not search-artists-batch's own narrower default of 10 (a
deliberate choice for that method's other callers, left unchanged). Without the
explicit override, a duplicate MBID ranked 11-25 would fall outside the window
and matchArtist would auto-pick a single resolved MBID instead of correctly
parking the artist as ambiguous.
resolve-artists also now LOADS its own prior output (context.readResource, a
NEW capability requirement for this method, optional-chained so an absent prior
— including the very first post-merge run — degrades to empty rather than
throwing) and treats it as a persistent, load-modify-write aggregate: a
seed-unresolved artist whose last search verdict (checkedAt) is younger than
ttlMs (new arg, default 30 days) is reused VERBATIM (mbid/status/source/
candidates/checkedAt) without a fresh search — except the LIBRARY's current name
always wins, so a renamed artist is never stuck with a stale display name. A
converged re-run therefore costs ZERO MusicBrainz requests. Two new method
arguments force a re-check: refresh (everything) and refreshKeys (specific
artists, ordered FIRST in the batch so an explicit re-check can never be
silently discarded by the maxQueries ceiling). An artist present in the
library with no prior entry is written; an entry in the prior map whose artist
is no longer in the library is dropped — only artists actually in the library
are ever written.
checkedAt means exactly "the timestamp of the last MusicBrainz SEARCH that
produced a verdict" — it is NEVER set on a seed match (seed coverage is free and
re-derived every run, not a search verdict) and NEVER set when a query's fetch
errored, was deferred, or was never reached this run (the prior value, still
stale, is preserved so the next run retries). A genuine no-match (MusicBrainz
returned zero candidates, no error) still gets a fresh checkedAt — it got a
real verdict this run, just a negative one.
Completeness: pendingSearch / truncated / stopReason
search-artists-batch bounds one run by maxQueries (default 400, passed
through) and a derived maxDurationMs backstop; a run that hits either ceiling
— or an aborted signal, or a Retry-After backoff — defers the remainder rather
than searching it. Without visibility into that, unresolved: 1083 on a first
post-merge run would be indistinguishable from "MusicBrainz doesn't know 1083
artists". Three new OPTIONAL top-level fields on the written artistMap —
pendingSearch (distinct artists that needed a verdict this run and didn't get
one), truncated, stopReason — make that distinction observable: re-run
resolve-artists until pendingSearch is 0 to confirm convergence (about 4
runs for a cold ~1459-1483-artist library at the default maxQueries). All
three are optional in the schema (the live 2258-entry map predates them) and
always set on write.
Schema: five mirrors move together
ArtistMapEntrySchema gains optional checkedAt; ArtistMapSchema gains
optional pendingSearch/truncated/stopReason; wanted.ts's report-side
ArtistMapEntry/ArtistMapContent interfaces mirror both; and — the mirror
missed once already in an earlier draft of this fix — the LOCAL TypeScript
type ArtistMapEntry that resolve-artists actually constructs against also
gains checkedAt, or the object literal it builds trips a TS2353
Modified 1 models. Added 1 reports
2026.08.02.1
Real-fixes all 6 latent bugs characterized by the test-only backfill below
(tracked locally as music-library-latent-bugs, never filed to the Lab).
music_library.ts is no longer byte-frozen; the model version and
manifest.yaml both bump to 2026.08.02.1, with an identity upgrades[] entry
(upgradeAttributes: (old) => old — no stored resource is reshaped, since every
change is either a defaulted global/method argument or an additive BpmSchema
field).
- LB1 (MEDIUM) fixed —
verify's remote ffmpeg decode loop now wraps every file in the shelltimeoutcommand (sized from the new defaultedffmpegDecodeTimeoutSecglobal arg, default 600s; 0 = no timeout), detected once per worker viacommand -v timeoutso it degrades gracefully if the container lacks it — mirrors bpm's ANALYZE_PYsignal.alarm(timeout).sshRunalso gained an optional client-sideAbortControllertransport ceiling (timer always cleared infinally, neverAbortSignal.timeout()) as a belt-and-suspenders guard sized generously per worker so it only fires if the remotetimeoutitself failed or ssh/network wedged. - LB2 (MEDIUM) fixed —
verify/bpm/probe's single-pathargument now resolves through two new shared, exported helpers —normalizeSegments(split on//\, drop./empty, pop on.., throw when..would escape the root — ported in spirit from obsidian-vault's segment guard) andconfineContainerPath— instead of the oldreplace(/^\/+/, "")that stripped only leading slashes. A../traversal now throwsError("Path escapes music root: …")instead of resolving outsidecontainerMusicRoot; a clean relative/absolute path still resolves identically to before. - LB3 (LOW) fixed —
probenow guardsJSON.parseon the ffprobe output: empty stdout throws a cleanErrornaming the file instead of a rawJSON.parse("")SyntaxError, and unparseable non-empty stdout is caught and rethrown as a typedErrortoo (mirrors juick'sJSON.parse→ typed-error pattern). - LB4 (LOW) fixed —
verify's US(0x1f)/RS(0x1e) record framing is now RS-safe: before splitting on\x1f, any\x1e-delimited fragment whose leading field is not a KNOWN cpath is treated as the tail of the previous record's ffmpeg output (which happened to contain a stray RS byte) and re-folded back onto it, rather than being parsed as an unmatchable orphan record. A corrupt file whose ffmpeg output embeds an RS byte is now correctly classified aserrorsinstead of silently passing asok. - LB5 (LOW) fixed —
bpmMediannow uses a proper median (new exportedmedian()helper): an even-length bpm array averages the two middle sorted values instead of returning the upper one (bpms[Math.floor(n / 2)]). - LB6 (LOW) fixed —
bpmgained amaxTracksmethod arg (default 50000, deliberately far higher than verify's 2000 — see below) plustracksTruncated/failuresTruncatedonBpmSchema. All stats (bpmMedian,confidenceBands,bpmHistogram) are computed over the FULL carried-over + newly-analyzed set BEFORE any truncation; only the STOREDtracks/failuresarrays are capped, mirroring verify'sproblems: problemsTruncated ? problems.slice(0, 2000) : problems. The default is high (not verify's 2000) because capping the stored array degrades bpm's resume carry-over (a library bigger than the cap would re-analyze the overflow every run) and therunningmethod's input; passmaxTracks: 0for full resume fidelity on a very large library. - Pin flips in
music_library_adversarial_test.ts: all 6pin: KNOWN BUGtests are nowfixed (music-library-latent-bugs LBn)assertions of the corrected behavior (LB1 also gained a quick-mode variant, an AbortSignal-transport-ceiling check, and affmpegDecodeTimeoutSec=0disable check; LB2 gained a clean-relative-path anti-over-rejection positive; LB6 gained a default-does-not-truncate companion test). The 2 regression-pinned positives (P1 static-SQL, P2 shQuote + control-byte filename filtering), the hostile host/sshUser safety test, and both fixtures-secret-scan tests are UNCHANGED (byte-identical). New direct unit tests fornormalizeSegments/confineContainerPath/medianwere added tomusic_library_coverage_test.ts.
Merge pull request #58 from umag/feat/music-library-bpm-running
feat(music-library): bpm analysis + cadence-matched running playlists (2026.07.17.1)
Modified 1 models. Added 1 reports
Release 2026.07.16.2 — align model versions with manifests
Maintenance release across the @magistr extensions. For most packages this
carries no functional change: the only edit is the model's version: field,
brought back in line with its manifest version so the published model type
version and the package version no longer drift.
Functional changes in this release are limited to:
anime-cron: normalizeTitle now strips a ": subtitle" suffix and a trailing parenthesized year before comparison, fixing dedup false-misses where the torrent title carries a subtitle or year that the AniList romaji does not.
arckit: first publish. Standalone ArcKit port — a 12-phase architecture governance state machine with 65 bundled templates, driven by a bundled skill.
Also tracks three extensions (kaiten, observability-agent, music-library) that previously existed only as untracked working-tree directories, recovered from stashes.
- Has README or module doc2/2earned
- README has a code example1/1earned
- README is substantive1/1earned
- Most symbols documented1/1earned
- No slow types (deprecated)1/1earned
- Dependencies pass trust audit2/2earned
- Has description1/1earned
- Platform support declared (or universal)2/2earned
- License declared1/1earned
- Verified public repository2/2earned