Skip to main content
← Back to list
01Issue
BugClosedSwamp CLIPublic
Assigneesstack72

Relationships

#1938 serve still panics on successful OAuth device-flow completion after #1899 fix (200 branch only)

Opened by sntxrr · 9/1/2026

Summary

swamp serve still panics the instant an OAuth device-flow login succeeds. This is the crash reported in #1899, re-verified on a build that contains #1899's shipped fix. The panic is unchanged: a Rust-level abort in Deno's HTTP layer, exit code 139, no JS stack, no graceful shutdown.

Filing separately rather than rippling on #1899 because the shipped mitigation addresses a different failure class than the one occurring, and one of the two hypotheses offered in #1899 is now disconfirmed. The information below supersedes the "Notes toward a cause" section of that report.

Environment

  • swamp 20260901.180419.0-sha.eb8f3c77 (contains the #1899 and #1900 fixes — verified in fetched source: src/serve/unhandled_rejection_guard.ts present and installed at src/cli/commands/serve.ts:1397; Login.tsx has the AbortController + inFlight guard)
  • swamp serve --auth-mode oauth --oauth-provider https://<INTERNAL_HOST> --dashboard --host 0.0.0.0
  • Login initiated from the --dashboard web UI, approved at the upstream IdP
  • Self-hosted authorization server (an OIDC shim in front of Authentik)
  • Linux/amd64, Docker, restart: unless-stopped

The panic

system │ Minted OAuth server token "oauth-<REDACTED>" for "user:<USERNAME>"
system │ Stored OAuth access token for "oauth-<REDACTED>"
system │ OAuth device flow completed for "user:<USERNAME>"

thread 'main' (1) panicked at ext/http/service.rs:794:5:
HTTP state error: Expected to be last strong reference
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Container exits 139 (SIGSEGV) and is restarted by the restart policy.

Timestamps put the abort 17 microseconds after the final success log line:

2026-09-01T18:54:32.817831285Z  Stored OAuth access token
2026-09-01T18:54:32.817849150Z  OAuth device flow completed
2026-09-01T18:54:32.817866013Z  HTTP state error: Expected to be last strong reference

The token is minted and persisted. The process dies before the 200 reaches the client, so the login can never complete.

Reproduction

Deterministic — 3 of 3 attempts, across two builds (20260828.202201.0-sha.2a5bcb89 and 20260901.180419.0-sha.eb8f3c77), days apart, on two different container instances.

  1. swamp serve --auth-mode oauth --oauth-provider <non-swamp-club server> --dashboard
  2. Open /dashboard, click login
  3. Approve the device code at the IdP
  4. Serve aborts within microseconds of the successful token exchange

The differential: only the 200 branch aborts

This is the useful narrowing. POST /auth/device/token returns from a single handler through a single two-line helper (jsonResponse, src/serve/device_auth_handler.ts:101). Every status but 200 is fine.

Response Path Result
202 {"status":"pending"} ~10 polls over the 50 s the login was open no panic
400 {"error":"Missing or invalid deviceCode"} POST {} no panic (verified)
502 {"error":"Upstream provider returned an unexpected error"} POST a bogus deviceCode no panic (verified)
200 {token, principal} successful exchange panics, every time

The 400 and 502 rows were probed directly against the running instance after the crash; the server stayed up and RestartCount did not move. So this is not the route, not the handler, and not the Response construction helper — it is specific to the success branch.

What is unique to the success branch

src/serve/device_auth_handler.ts:227-255. Relative to the 202 path, the 200 path additionally performs:

  • deps.mintServerToken(...) — :228, writes token data via the datastore (DatastoreSyncService when configured)
  • deps.storeAccessToken(...) — :238, which calls VaultService.fromRepository(...) and vaultService.put(...) (:434-446). On this deployment the vault provider makes an outbound HTTPS call, so this is real async I/O inside the request handler.
  • returns a larger body containing principal.collectives / principal.groups

Any of those could be leaving a live reference on the request's HTTP record when the handler resolves. Expected to be last strong reference is Deno asserting Rc::strong_count == 1 while recycling the record, so the shape to look for is something retaining the request/response past the handler's return.

Why the shipped #1899 mitigations do not apply

Both are worth keeping, but neither can affect this crash:

  1. installUnhandledRejectionGuard() cannot catch it. It registers unhandledrejection and error listeners on globalThis. This is a Rust panic! that aborts the process — it never becomes a JS event, so no listener runs and preventDefault() is never reached. Good hardening for extension code; structurally unable to catch a runtime abort.

  2. The client-side poll guard did not prevent it. Login.tsx now has both the AbortController and the inFlight skip, and the crash reproduced anyway. That disconfirms the overlapping-request hypothesis I offered in #1899 — please disregard it. Concurrent polls are not the trigger.

The embedded Deno version is not pinned

Relevant because the panic is in Deno, not swamp. scripts/download_deno.ts:136 derives the embedded runtime version from whatever deno is on the build runner, and the workflows request a floating deno-version: v2.8.x (.github/workflows/publish-testing.yml:35, publish-client.yml:35). So the runtime patch level in any given release is not recorded in the repo and not reproducible from it. Pinning it exactly would make this class of bug bisectable, and would tell you whether the abort tracks a specific Deno patch.

I could not extract the version from the shipped binary to report it here.

Suggested reproduction harness (no IdP required)

handleDeviceAuth takes all of its I/O through the injectable DeviceAuthDeps interface (src/serve/device_auth_handler.ts:56-98), and src/serve/device_auth_handler_test.ts already stubs it. A real Deno.serve listener wrapping handleDeviceAuth with stubbed pollForToken / getUserInfo / mintServerToken / storeAccessToken, hit by a single POST, should reproduce without any OAuth provider. Suggested variables to bisect, in order:

  • stub storeAccessToken as a no-op vs. one performing an outbound fetch
  • stub mintServerToken with and without a datastore write
  • return the full {token, principal} body vs. a bare {token}

If a no-op-deps 200 does not panic, the trigger is in the async work, not the response — which would point at something in the vault or datastore path retaining the request.

Impact

  • OAuth dashboard login cannot be completed at all. Succeeding is what kills it, so there is no way through by retrying.
  • Process-level abort takes down everything the instance owns. On this deployment that includes the workflow scheduler: 8 cron triggers are interrupted and re-registered on restart, roughly 40 s of downtime per attempt.
  • A token is minted and written to the vault on every failed attempt, so each one leaves an orphaned credential behind.

Note

#1900's fix is working and made this diagnosable — the dashboard now reports "Unexpected response from server" (the JSON-parse branch, hit because the proxy returns an HTML error page once serve is dead) instead of spinning forever. Thank you for that one.

02Bog Flow
✓OPEN✓TRIAGED○IN PROGRESS◉CLOSED+ 1 MOREASSIGNED+ 4 MOREREVIEW

Closed

9/2/2026, 6:04:58 PM

No activity in this phase yet.

03Sludge Pulse
stack72 assigned stack729/1/2026, 11:16:07 PM
Editable. Press Enter to edit.

stack72 commented 9/2/2026, 6:04:56 PM

We can't reproduce this on 20260902.172610.0-sha.f7b0a954 which is a new deno / rust version

Sign in to post a ripple.