Relationships
#3520 Windows: non-ASCII workflow inputs reach behavior methods double-encoded (UTF-8 read as cp1252)
Opened by rogueco · 10/15/2026
Summary
On Windows, a string workflow input that contains non-ASCII characters reaches the behavior method garbled. The method sees the UTF-8 bytes decoded as Windows-1252, so an en dash (U+2013) arrives as U+00E2 U+20AC U+201C (the UTF-8 bytes of U+2013 read as cp1252) (U+00E2 U+20AC U+201C). The text is wrong in the method's arguments, and anything the method writes or sends (resource data, API calls) carries the garbled text.
We found this when a PR body passed through a workflow input reached GitHub as 6U+00E2 U+20AC U+201C (the UTF-8 bytes of U+2013 read as cp1252)12 instead of 6U+2013 (en dash)12.
Observed behaviour
- A workflow with a string input (
body) passes it to a step:inputs: { body: ${{ inputs.body }} }. - Run it with
swamp workflow run <wf> --stdin --json < inputs.json. The JSON file is ASCII-only, with the dash written as the escapeU+2013 (en dash), so the input file itself is not the problem. - The step's method stores
args.bodyin a resource.
The stored value contains 6U+00E2 U+20AC U+201C (the UTF-8 bytes of U+2013 read as cp1252)12. Reading the workflow's resolved input before the step shows the correct 6U+2013 (en dash)12, so the change happens somewhere between input resolution and the method receiving args.
A related symptom may share the cause: data written by @webframp/github get_pull_request_context (which reads gh output) contained emoji as U+00F0 U+0178 U+0178 U+00A1 (U+1F7E1 read as cp1252) (U+1F7E1) and U+2019 as U+00E2 U+20AC U+2122 (U+2019 read as cp1252).
Expected
Method arguments carry the same Unicode string that the workflow input resolved to.
Suggested repro
Use a minimal local extension method that writes args.text to a resource. Call it from a workflow with --input text=6U+2013 (en dash)12 and with --stdin {"text":"6U+2013 (en dash)12"}, then compare the stored resource with "6U+2013 (en dash)12".
Workaround
We now reject non-ASCII text before it goes into workflow inputs (titles and bodies are kept ASCII).
Environment
- Windows 11 Pro 10.0.26300
- swamp 20261014.204224.0-sha.8d6dfb6c
- Shell: Git Bash; PowerShell 7 is also installed. System locale is en-GB (default ANSI code page 1252).
Closed
No activity in this phase yet.
stack72 commented 10/16/2026, 1:51:55 AM
@rogueco thanks for the detailed report. We tried to reproduce this and couldn't:
- We built a minimal TypeScript extension method that writes
args.text, and the code point of each character, to a resource. We called it from a workflow step withtext: ${{ inputs.text }}. - We ran it with
--stdinand an ASCII-only JSON file (the dash as the escape–, plus🟡and’), and again with--input text=6–12 🟡 ’. - In both runs the stored resource held
6–12 🟡 ’with the right code points (8211, 128993, 8217), and the file on disk was valid UTF-8. - We also traced the code path from the resolved workflow input to the method's arguments. For a local run the method is called in the same process, and every decode on the way is UTF-8.
We ran that on macOS, so it doesn't rule out something Windows-specific. Could you share:
- The workflow step that garbles the text. Is it a TypeScript extension method, or a
command/shellstep (for example, one that runsgh)? - How you read the stored value:
swamp data get, opening the file, or something else (and if a shell, which one)? - Whether the minimal repro from your report shows the problem when you run it as written.
Our current suspect is Windows PowerShell 5.1, which runs command/shell steps on Windows. There, Get-Content and Set-Content treat UTF-8 files without a BOM as Windows-1252. That would turn an en dash into exactly the – you saw.
stack72 commented 10/16/2026, 1:53:25 AM
Correction to the second bullet above: the JSON file used backslash-u escapes (backslash, u, 2013 for the en dash, and so on), so it was pure ASCII, the same as yours. They were turned into the characters when this ripple was posted.
rogueco commented 10/16/2026, 12:47:46 PM
Really sorry, you were right, this isn't a swamp bug. I checked the raw bytes on the Windows machine where I saw it, and the garbling happened in my own script before anything reached swamp. It read swamp's output as Windows' cp1252 instead of UTF-8, then fed the garbled text back in. Swamp passed everything through correctly.
Sorry for wasting your time and effort. I'll close this now.
Sign in to post a ripple.