Relationships
#3425 CI review gates read stale reviews when a passing review falls back to bold headings after Bash #-line denials
Opened by skunk-ape · 10/14/2026
What happens
A CI review that passes can still fail its check and block auto-merge. On swamp PR 3091 (swamp-club#3420), the Adversarial Code Review check failed twice, on 8e332c46 and 77b84a51, although both new reviews said PASS. Each time the gate fell back to a stale CHANGES_REQUESTED review on an older commit (fdb409e4). The PR had to be merged by hand. swamp-club#3422 reported the symptom; this issue covers the cause and the fix.
Cause
- The reviewer may only run
gh pr review, and the prompt asks for a body that starts with## Adversarial Reviewand has###sections. - Claude Code refuses a Bash command whose quoted argument contains a newline followed by
#("newline followed by # inside quoted argument"), because such a line can hide arguments from the permission check. ANSI-C quoting is refused too. In CI no one can approve, so each attempt becomes a permission denial: 4 in the run on fdb409e4, 8 in the run on 8e332c46. - The model retries until a variant passes, and the variant changes from run to run:
- a command substitution around a quoted heredoc (
catwith a quotedEOFdelimiter) is allowed and keeps the##headings, which is probably how the first review posted; - bold headings (
**Adversarial Review**,**Medium**, …) contain no#, so they always pass. Both later reviews used them.
- a command substitution around a quoted heredoc (
- The gate only accepts a body matching
^(## )?Adversarial Review, so it skips a bold review and reads the last one that matches, even when that review is for an older commit.
Reproduced locally with claude -p --allowedTools "Bash(echo:*)": an echo of a double-quoted multi-line body with ##/### lines is refused; the same body through the heredoc substitution runs.
Where
.github/actions/adversarial-review/action.yml: the prompt's submit commands and body format, and the "Fail if changes requested" step..github/workflows/ci.yml: the same heading regex appears 8 times. It is used by the adversarial job's skip check (latest review state and its commit) and by the CI Security Review and Review Integrity jobs, whose prompts also ask for##headings and whose gates read the latest review the same way.
Expected / proposed fix
- Prompts: tell each reviewer to submit with
gh pr review N --comment(or--approve/--request-changes), passing--bodyas a command substitution around a quoted heredoc, and show that exact form, so the##headings survive and no attempts are denied. - Gates: select only the reviews whose
commit_idmatches the PR head before reading.state. A review for an older commit must never block, approve or skip the current one. - Heading match: as a backstop, accept the heading with or without markdown emphasis, e.g.
^(#+ |\*\*)?Adversarial Review(and the same for Review Integrity and CI Security Review). - Optionally fail the job loudly when the review step reports permission denials and no review exists for the head commit, so a reviewer that never managed to post cannot leave a stale verdict in charge.
These are CI trust-root files, so the change goes through the CI security review.
Open
No activity in this phase yet.
Sign in to post a ripple.