Skip to main content
← Back to list
01Issue
FeatureIn ProgressSwamp ClubPublic
Assigneesskunk-ape

Relationships

#3977 Score extensions with @swamp-club/extension-scoring (rubric v5) instead of the registry's own copy

Opened by skunk-ape · 10/26/2026

Problem

swamp-club#3781 (merged in https://github.com/swamp-club/swamp/pull/3385, first published as @swamp-club/extension-scoring@0.20261026.1) moved the extension quality rubric and every rule it rests on into the JSR package @swamp-club/extension-scoring, published from the swamp repo. swamp's extension quality and push score with it. The registry still runs its own hand-kept copy, which has drifted from swamp's in about 25 places, so the registry and the CLI disagree until the registry switches.

What to do

  1. Import jsr:@swamp-club/extension-scoring@<first published version> (alongside @swamp/client; optionally move that import off the old @systeminit/swamp-lib scope while there).
  2. Replace with package imports, and delete:
    • lib/domain/scorecard/analysis-factors.ts, the factor composition in score.ts (keep ExtensionVersionMeta, ExtensionSettings and the badge; map them onto RubricInputs with repository: { kind: "confirmed", … } and scoredWith: versionMeta?.rubricVersion ?? 2), secret-docs.ts, secret-names.ts, doc-failure.ts, redact-paths.ts, lib/domain/spdx/, dependency-trust.ts, deno-lockfile.ts, dependency-audit-record.ts;
    • the pure parts of lib/infrastructure/http-dependency-trust-client.ts (packument, OSV and JSR parsing, severity, CVSS) and of lib/infrastructure/extension-tarball-analyzer.ts (README/LICENSE names, stripped config, controlled deno.json, entrypoint fields, manifest-root search, deno doc arguments). The client keeps fetching; it builds NpmLookup/JsrLookup/OSV Lookups, calls planNpmPackage → OSV → evaluateNpmPackage / evaluateJsrPackage, groupForAudit for the 50-dependency cap, and stores scorecardResult(...).
  3. Feed the analyzer's archive listing to the shared selectors: secretDocsTargets + markdownImports (every non-test code file and every Markdown file that is not a binary, include file or imported data; files over 1 MiB, or past 10,000, are unread and fail the factor) and isAuditedCodeFile + writtenSpecifiers for the dependency audit (every non-test code file, tests now excluded; commented-out imports and specifier-shaped strings no longer audited).
  4. Store the extensions content-metadata kind (method argument names and their sensitive flag): argumentSchemas counts them, and KNOWN_CONTENT_METADATA_KEYS currently drops it on purpose. Like the webhooks kind in swamp-club#3592, minus any listing display.
  5. Run the package's vectors (vectors/*.json, imported from the pinned version) in a registry test, through the registry's own client and analyzer wrappers; drop tests/fixtures/dependency_audit_vectors.json.
  6. Bump to RUBRIC_VERSION 5 (the package's) and rescore stored versions.

Behavior changes on the registry (accepted on swamp-club#3781)

  • Rubric: a description of only whitespace no longer earns description; a version with no README shows rich-readme missing, not pending.
  • Archive: README/LICENSE names match exactly from a listing (unchanged on Linux); manifest-root search order is sorted.
  • Secret-safe-docs: include files and code outside the kind directories are scanned; binaries: Markdown is skipped; Markdown any archived module imports is data; extensions: method arguments count as declared/sensitive.
  • Dependency trust: test files are not audited; commented-out imports and string literals are not dependencies; an OSV answer whose vulns is not a list leaves the dependency not audited; a version npm lists without a manifest is not audited; an OSV hit on an exact pin is reported even when npm is down; an invalid npm name is not fetched; a dependency license is checked as an SPDX expression (MIT OR Apache-2.0 passes) and no license warns; a missing maintainers list is no warning (unchanged); age uses days/30.44 months; an unknown OSV database_specific.severity falls back to CVSS, then UNKNOWN (unchanged).

Manual pages to update with it

  • reference/extensions/scorecard-rubric.md (rubric v5: factors, statuses incl. partial, /16 vs /14, what each factor reads)
  • how-to/extensions/improve-score.md
  • explanation/extension-scorecard.md, explanation/extension-trust.md
  • the extension quality reference: the new --isolated flag (runs deno doc with an empty cache and no inherited registry credentials, as the registry does); this one can land as soon as swamp releases.

Open question carried from swamp-club#3781

Push sends an extension-wide secret-unsafe-docs acceptance without a file, and the registry reads a fileless acceptance as covering only the description, while the CLI applies it to every finding. Decide the wire form (e.g. a scope: "extension" field) before or with this switch.

02Bog Flow
✓OPEN✓TRIAGED◉IN PROGRESS○SHIPPED+ 1 MOREASSIGNED+ 5 MOREREVIEW+ 1 MOREIMPLEMENTATION

In Progress

10/26/2026, 8:24:44 PM

Click a lifecycle step above to view its details.

03Sludge Pulse
skunk-ape assigned skunk-ape10/26/2026, 7:59:59 PM
Editable. Press Enter to edit.

system commented 10/26/2026, 7:32:04 PM

Classified automatically when this issue was filed.

  • Source: Swamp Club

If you feel this classification is incorrect, add a ripple to tell us so.

skunk-ape commented 10/26/2026, 8:15:17 PM

Open question on the fileless secret-unsafe-docs acceptance: resolved, no wire change needed. On swamp main the rule is site-scoped and quality.yaml refuses a fileless entry for it. The only fileless acceptance push sends is for a finding on the manifest description line, because manifest.yaml has no archive path. Both the CLI and the package's secretDocsAcceptanceMatcher read that entry as covering the description findings, and the package vector 'a fileless acceptance covers every description finding' pins it. No scope field, no swamp change.

The rescore will cover only the current scorecard version of each extension (resolveScorecardVersion). The scoring queue's claim race that a bulk enqueue would expose is filed as swamp-club#3979.

skunk-ape commented 10/26/2026, 8:17:20 PM

Scope change: the RUBRIC_VERSION 5 rescore of stored versions is split out to swamp-club#3980, which depends on swamp-club#3979. This issue still bumps RUBRIC_VERSION to 5, so newly scored versions are v5; versions scored earlier keep their v4 rows until #3980 sweeps them or an owner presses Rescore.

Sign in to post a ripple.