Relationships
#526 swamp-extension adversarial review skill needs mandatory mechanical verification checklist
Opened by webframp · 6/1/2026· Shipped 6/3/2026
Problem
The adversarial review reference in the swamp-extension skill provides excellent dimensional guidance (credentials, error handling, idempotency, etc.) but lacks a mandatory mechanical verification step that catches the most common class of bugs AI agents produce.
The Recurring Failure Pattern
When an AI agent writes extension code and then self-reviews using the adversarial review dimensions, it consistently misses:
- Schema/write mismatches: schema declares a field but the write stores incompatible data (e.g., schema says username but API returns assignee.login)
- Missing truncation fields: method paginates with a cap but output schema has no truncated field
- Placeholder values left in place: schema declares byUser with full structure but implementation writes byUser: [] and never implements it
- Wrong resource spec: method writes data to a spec whose schema doesn't match
These are NOT subtle logic errors. They are mechanical mismatches between declared schemas and actual writes.
Proposed Enhancement
Add a Mandatory Mechanical Checks section to the adversarial review reference that agents must execute BEFORE presenting findings:
- Schema conformance: For every writeResource call, enumerate every field in the spec schema. Verify each field appears in the data object with a non-hardcoded value.
- Truncation honesty: For every pagination loop, verify the output schema has truncated: boolean AND the write sets it.
- Instance name uniqueness: Verify no two methods write incompatible data to the same spec + instance.
- Schema field coverage: Read the schema. Read the write. Do they match 1:1?
Why This Matters
The CI adversarial review catches these every time because it reads every line equally. The agent self-review has confirmation bias — it reviews interesting code deeply and skims obvious code. The mechanical checklist forces equal attention on every method.
Evidence
This pattern occurred across multiple extensions: aws/inventory (instance name collision), aws/adopt (incomplete setupCommand), github-copilot-governance (SeatsSchema mismatch, byUser always empty, wrong spec, missing truncated). Each time the local adversarial review passed CLEAR and CI caught it.
Shipped
Click a lifecycle step above to view its details.