Skip to main content
← Back to list
01Issue
BugOpenSwamp CLIPublic
Assigneesstack72

Relationships

#3603 Multi-model skill eval: gpt-5.4 and gemini jobs fail preflight every run (OpenAI credits exhausted, Google API key invalid)

Opened by skunk-ape · 10/19/2026

The scheduled Multi-Model Skill Trigger Eval workflow (.github/workflows/multi-model-eval.yml) has concluded failure on every run since at least 2026-09-26. In the 2026-10-17 scheduled run (GitHub Actions run 39016870581) and a 2026-10-19 dispatch (run 39276795383), the jobs fail in the preflight API check in scripts/eval_skill_triggers_promptfoo.ts, before any eval runs:

  • gpt-5.4: HTTP 429 insufficient_quota, code credit_balance_exhausted (You have no credits remaining). The OpenAI organization behind the OPENAI_API_KEY repo secret needs credits.
  • gemini-2.5-pro and gemini-3.1-pro: HTTP 400 API_KEY_INVALID (API key not valid). The GOOGLE_API_KEY repo secret needs replacing.

The Anthropic jobs (sonnet, opus, fable) pass. Until the two secrets are fixed, the cross-model analysis only covers Anthropic models and the workflow shows red every Saturday.

Found while verifying swamp-club#3516.

02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

10/19/2026, 3:20:54 PM

No activity in this phase yet.

03Sludge Pulse
stack72 assigned stack7210/19/2026, 5:30:06 PM

Sign in to post a ripple.