Relationships
#3977 Score extensions with @swamp-club/extension-scoring (rubric v5) instead of the registry's own copy
Opened by skunk-ape · 10/26/2026
Problem
swamp-club#3781 (merged in https://github.com/swamp-club/swamp/pull/3385, first published as @swamp-club/extension-scoring@0.20261026.1) moved the extension quality rubric and every rule it rests on
into the JSR package @swamp-club/extension-scoring, published from the swamp
repo. swamp's extension quality and push score with it. The registry still
runs its own hand-kept copy, which has drifted from swamp's in about 25 places,
so the registry and the CLI disagree until the registry switches.
What to do
- Import
jsr:@swamp-club/extension-scoring@<first published version>(alongside@swamp/client; optionally move that import off the old@systeminit/swamp-libscope while there). - Replace with package imports, and delete:
lib/domain/scorecard/analysis-factors.ts, the factor composition inscore.ts(keepExtensionVersionMeta,ExtensionSettingsand the badge; map them ontoRubricInputswithrepository: { kind: "confirmed", … }andscoredWith: versionMeta?.rubricVersion ?? 2),secret-docs.ts,secret-names.ts,doc-failure.ts,redact-paths.ts,lib/domain/spdx/,dependency-trust.ts,deno-lockfile.ts,dependency-audit-record.ts;- the pure parts of
lib/infrastructure/http-dependency-trust-client.ts(packument, OSV and JSR parsing, severity, CVSS) and oflib/infrastructure/extension-tarball-analyzer.ts(README/LICENSE names, stripped config, controlled deno.json, entrypoint fields, manifest-root search,deno docarguments). The client keeps fetching; it buildsNpmLookup/JsrLookup/OSVLookups, callsplanNpmPackage→ OSV →evaluateNpmPackage/evaluateJsrPackage,groupForAuditfor the 50-dependency cap, and storesscorecardResult(...).
- Feed the analyzer's archive listing to the shared selectors:
secretDocsTargets+markdownImports(every non-test code file and every Markdown file that is not a binary, include file or imported data; files over 1 MiB, or past 10,000, are unread and fail the factor) andisAuditedCodeFile+writtenSpecifiersfor the dependency audit (every non-test code file, tests now excluded; commented-out imports and specifier-shaped strings no longer audited). - Store the
extensionscontent-metadata kind (method argument names and theirsensitiveflag):argumentSchemascounts them, andKNOWN_CONTENT_METADATA_KEYScurrently drops it on purpose. Like the webhooks kind in swamp-club#3592, minus any listing display. - Run the package's vectors (
vectors/*.json, imported from the pinned version) in a registry test, through the registry's own client and analyzer wrappers; droptests/fixtures/dependency_audit_vectors.json. - Bump to
RUBRIC_VERSION5 (the package's) and rescore stored versions.
Behavior changes on the registry (accepted on swamp-club#3781)
- Rubric: a description of only whitespace no longer earns
description; a version with no README showsrich-readmemissing, not pending. - Archive: README/LICENSE names match exactly from a listing (unchanged on Linux); manifest-root search order is sorted.
- Secret-safe-docs: include files and code outside the kind directories are
scanned;
binaries:Markdown is skipped; Markdown any archived module imports is data;extensions:method arguments count as declared/sensitive. - Dependency trust: test files are not audited; commented-out imports and
string literals are not dependencies; an OSV answer whose
vulnsis not a list leaves the dependency not audited; a version npm lists without a manifest is not audited; an OSV hit on an exact pin is reported even when npm is down; an invalid npm name is not fetched; a dependency license is checked as an SPDX expression (MIT OR Apache-2.0passes) and no license warns; a missing maintainers list is no warning (unchanged); age uses days/30.44 months; an unknown OSVdatabase_specific.severityfalls back to CVSS, then UNKNOWN (unchanged).
Manual pages to update with it
reference/extensions/scorecard-rubric.md(rubric v5: factors, statuses incl.partial, /16 vs /14, what each factor reads)how-to/extensions/improve-score.mdexplanation/extension-scorecard.md,explanation/extension-trust.md- the
extension qualityreference: the new--isolatedflag (runsdeno docwith an empty cache and no inherited registry credentials, as the registry does); this one can land as soon as swamp releases.
Open question carried from swamp-club#3781
Push sends an extension-wide secret-unsafe-docs acceptance without a file,
and the registry reads a fileless acceptance as covering only the description,
while the CLI applies it to every finding. Decide the wire form (e.g. a
scope: "extension" field) before or with this switch.
In Progress
Click a lifecycle step above to view its details.
system commented 10/26/2026, 7:32:04 PM
Classified automatically when this issue was filed.
- Source: Swamp Club
If you feel this classification is incorrect, add a ripple to tell us so.
skunk-ape commented 10/26/2026, 8:15:17 PM
Open question on the fileless secret-unsafe-docs acceptance: resolved, no wire change needed. On swamp main the rule is site-scoped and quality.yaml refuses a fileless entry for it. The only fileless acceptance push sends is for a finding on the manifest description line, because manifest.yaml has no archive path. Both the CLI and the package's secretDocsAcceptanceMatcher read that entry as covering the description findings, and the package vector 'a fileless acceptance covers every description finding' pins it. No scope field, no swamp change.
The rescore will cover only the current scorecard version of each extension (resolveScorecardVersion). The scoring queue's claim race that a bulk enqueue would expose is filed as swamp-club#3979.
skunk-ape commented 10/26/2026, 8:17:20 PM
Scope change: the RUBRIC_VERSION 5 rescore of stored versions is split out to swamp-club#3980, which depends on swamp-club#3979. This issue still bumps RUBRIC_VERSION to 5, so newly scored versions are v5; versions scored earlier keep their v4 rows until #3980 sweeps them or an owner presses Rescore.
Sign in to post a ripple.