Relationships
#1636 extension quality awards repository-verified 2/2 for a nonexistent or private GitHub repo
Opened by aaronge · 8/13/2026· Shipped 8/13/2026
swamp extension quality checks that repository: points at an allowlisted host, but does not check that the repository actually exists or is publicly reachable. Any well-formed https://github.com/<owner>/<name> URL earns the full 2 points, so the local score can exceed what the server will award at publish.
Reproduce
Point repository: at a GitHub URL that cannot exist and score the manifest:
repository: https://github.com/aaron-dernley/this-repo-does-not-exist-9f3a2c
$ swamp extension quality manifest.yaml --json
total: 14/14 = 100%
repository-verified: 2/2 [earned] — Verified public repository (server confirms on publish)Observed matrix
Same manifest, only repository: varied:
| repository value | repository-verified | total |
|---|---|---|
https://github.com/<owner>/this-repo-does-not-exist-9f3a2c |
2/2 | 14/14 |
https://example.com/not-a-repo |
0/2 | 12/14 |
https://not-an-allowlisted-host.invalid/x |
0/2 | 12/14 |
ftp://nonsense |
0/2 | 12/14 |
So host allowlisting is enforced; existence and visibility are not.
How it was hit
Real sequence, not synthetic. The extension was prepared with the repo still
private. extension quality reported 14/14 = 100%, which reads as "ready to
publish at the third-party ceiling". The rubric states the factor is "confirmed
via that host's public API", and the score would in fact have been 12/14 at
publish because GitHub's public API cannot resolve a private repo. The gap was
only caught by having read the rubric — the command itself gives no signal.
Note in mitigation
The factor label does say "(server confirms on publish)", so the behaviour is
disclosed. The problem is that the disclosure sits inside one factor object while
the headline earnedPoints / percentage — the numbers people actually read, and
the ones a pre-publish check is consulted for — silently include the unverified
points.
Suggested fix
Any one of:
- Do the reachability check locally (an unauthenticated
GETof the API URL is enough to distinguish missing/private from public). - Exclude server-confirmed factors from the local total and report it as e.g.
12/14 (+2 pending server verification). - Mark the factor
status: "provisional"rather than"earned"so a caller can tell the difference programmatically.
Environment
swamp 20260809.004828.0-sha.b61c9de2, macOS (darwin arm64), rubric version 3
Shipped
Click a lifecycle step above to view its details.