Relationships
#508 Extension quality/adversarial-review: add a 'published-surface hygiene' check for real infra identifiers
Opened by stateless · 6/1/2026· Shipped 6/4/2026
Problem
The extension quality gates catch secrets (adversarial-review.md → "Credentials & Secrets": vault + .meta({sensitive:true})) and the safety analyzer catches file types/sizes/eval, but nothing audits the content of shipped non-code files for real infrastructure identifiers.
An extension can reach Grade A while its README.md examples (and additionalFiles, and repo-visible *_test.ts fixtures) embed real production IPs, hostnames, jump/bastion addresses, and internal topology. Authors only avoid this by personal diligence — exactly what a rubric should catch.
Concrete repro
Building @stateless/proxmox (a Proxmox guest-lifecycle model):
swamp extension quality manifest.yaml --json→earnedPoints: 14 / 14,percentage: 100,status: passed.swamp extension push --dry-run→ passed.
…while README.md config examples contained a real Tailscale jump IP, the node's real internal IP, and real internal subnet addressing (copied verbatim from a working globalArguments block). No gate flagged it. It was caught only by a human reviewer.
Proposed solution
Add a universal adversarial-review dimension — "Published-surface hygiene" — asserting that README, examples, additionalFiles, and *_test.ts fixtures use placeholder values (RFC 5737 192.0.2.0/24 / 198.51.100.0/24 / 203.0.113.0/24, RFC 1918 framed as examples, example.com) rather than real infra identifiers or internal topology.
Optionally enforce it as a quality-rubric factor: a heuristic scan at push/quality time over the shipped doc/example/fixture files that warns on IP-literal and hostname patterns, so it is enforced rather than advisory.
Alternatives considered
- Rely on author diligence (status quo) — demonstrably fails.
- Document the guidance in the swamp-extension skill without a check — weaker; easy to miss.
- A push-time lint flagging IP/hostname literals in
additionalFiles+README(warn, not hard-fail, to avoid false positives on intentional public examples).
Shipped
Click a lifecycle step above to view its details.
stack72 commented 6/4/2026, 5:02:22 PM
Thanks @stateless for reporting this! The fix has been merged and a release is on its way. We appreciate your contribution to swamp.
stateless commented 6/28/2026, 12:50:27 AM
Closing the loop: the deferred enforcement this issue proposed — "enforce it as a quality-rubric factor: a heuristic scan at push/quality time… enforced rather than advisory" — is now implemented as @stateless/redaction, a deterministic published-surface-hygiene gate (a classifier, not an LLM eval).
scanwalks files/dirs + inline text against: generic recognizers (private-range/CGNAT IPv4 full-quad only,*.ts.net) plus a fleet-aware denylist (owned hostnames/FQDNs/people, matched case-sensitively on word boundaries) and per-instance custom recognizers. It hard-halts (throws) on a hit by default; advisory mode optional.redactremaps detections to the placeholders the §8 dimension already mandates (RFC 5737 IPs, RFC 2606 example.com).- The denylist is derived via a CEL view over an inventory model, so it grows with the fleet and isn't hand-maintained; a community checkout with no denylist still runs the generic tier.
Intended call sites: a publish-workflow step, a git pre-push hook, a Claude Code PreToolUse hook, and CI (generic tier).
Verified before posting (and this comment was itself scanned clean by the gate): a known-good published artifact passes clean (parity with the existing check); an artifact carrying real infra identifiers is caught; 11 unit+integration tests green; swamp extension quality 14/14 — which includes the §8 hygiene dimension, so the gate passes its own check.
Sign in to post a ripple.