GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — test_subprocess: test_getoutput fails on non-English Windows, as text mode decodes cmd.exe's OEM output as UTF-8 · Issue #160652 · python/cpython

test_subprocess: test_getoutput fails on non-English Windows, as text mode decodes cmd.exe's OEM output as UTF-8 #160652

New issue
Copy link
New issue
Copy link
Open
#160664
Open
test_subprocess: test_getoutput fails on non-English Windows, as text mode decodes cmd.exe's OEM output as UTF-8#160652
#160664
Copy link
Labels
OS-windowstestsTests in the Lib/test dirTests in the Lib/test dirtopic-subprocessSubprocess issues.Subprocess issues.

Description

@summeriver13
summeriver13
opened on Oct 25, 2026
Issue body actions

Bug report

test_subprocess.test_getoutput fails on any Windows whose console code page is not UTF-8, because the child's output is decoded as UTF-8. The dependency on the system language can be removed — a child that simply writes non-UTF-8 bytes shows the same thing:

import subprocess, sys
subprocess.run([sys.executable, "-c",
                "import sys; sys.stdout.buffer.write('版本'.encode('cp936'))"],
               stdout=subprocess.PIPE, text=True).stdout
interpreter sys.flags.utf8_mode result
3.14.7 0 '\u7248\u672c'
3.16.0a1 (main) 1 UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 0
3.12.10 with -X utf8=1 1 same UnicodeDecodeError

Lib/subprocess.py:1063 resolves text mode through _text_encoding(), which returns utf-8 whenever UTF-8 mode is on — the default since 3.15 (PEP 686). The child is not under Python's control: cmd.exe built-ins still write the console's OEM code page (936 here; GetACP() = GetOEMCP() = GetConsoleOutputCP() = 936, while locale.getencoding() stays cp936).

That is what breaks the test: it runs getstatusoutput("type <missing file>") and only asserts a non-zero status, but dies while decoding the localized error message:

File "Lib/subprocess.py", line 1382, in communicate
    stdout = self.stdout.read()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 4: invalid continuation byte

On the same machine, chcp 437 turns the message into ASCII (Access is denied.) and the test passes, and -X utf8=0 makes it pass too — the failure tracks the child's code page, not the build. CPython's Windows CI runs English images, so it stays green.

Suggested fix (test-side): stop depending on the child's localized text, e.g. subprocess.run([sys.executable, "-c", "sys.exit(1)"], stdout=subprocess.PIPE, text=True) and assert the status only.

Two neighbours, both distinct: gh-158006 (with two pipes the decode error stays in the reader thread and stdout becomes None), gh-158633 (the docs don't say the default encoding is a guess about the child).

environment and full traceback
$ ./PCbuild/amd64/python_d.exe -c "import ctypes,locale,sys;k=ctypes.windll.kernel32;print(k.GetACP(),k.GetOEMCP(),k.GetConsoleOutputCP(),locale.getencoding(),locale.getpreferredencoding(False),sys.flags.utf8_mode)"
936 936 936 cp936 utf-8 1

$ ./PCbuild/amd64/python_d.exe -m test -v test_subprocess -m test_getoutput
ERROR: test_getoutput (test.test_subprocess.MiscTests.test_getoutput)
  File "Lib/test/test_subprocess.py", line 4135, in test_getoutput
    status, output = subprocess.getstatusoutput(
  File "Lib/subprocess.py", line 792, in getstatusoutput
    data = check_output(cmd, shell=True, text=True, stderr=STDOUT,
  File "Lib/subprocess.py", line 1382, in communicate
    stdout = self.stdout.read()
  File "Lib/codecs.py", line 327, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 4: invalid continuation byte

Machine: Windows 11 (10.0.26100), zh-CN, CPython built from heads/main:11598362a3e with MSVC 14.44, Debug x64.

CPython versions tested on:

3.12, 3.14, 3.16, main

Operating systems tested on:

Windows


AI assistance: drafted with deepseek-v4.1-flash in the pi coding agent; every command above was run locally on the machine described, and the text was reviewed by the author before posting.

Linked PRs

  • gh-160652: Make test_getoutput independent of the console code page #160664

Activity

  1. aisk
    added
    testsTests in the Lib/test dirTests in the Lib/test dir
    OS-windows
    topic-subprocessSubprocess issues.Subprocess issues.
    on Oct 25, 2026
  2. aisk commented on Oct 25, 2026

    @aisk
    aisk
    on Oct 25, 2026
    Member
    More actions

    I can verify this on a Chinese locale Windows machine (code page 936) with current main.

  3. aisk commented on Oct 25, 2026

    @aisk
    aisk
    on Oct 25, 2026
    Member
    More actions

    Using sys.exit(1) from the suggested fix would skip getstatusoutput(), but that is what this test checks. I suggest passing errors='replace' instead, so the test still uses getstatusoutput().

  4. summeriver13 commented on Oct 25, 2026

    @summeriver13
    summeriver13
    on Oct 25, 2026
    Author
    More actions

    Agreed — a sys.exit(1) child would stop exercising getstatusoutput(), which is the point of the test. Opened #160664 with errors="replace": on CP936 the same call now returns (1, <mojibake>) instead of raising, and -m test test_subprocess passes (run=371 skipped=216).

    It needs the skip news label — test-only change, no blurb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    OS-windowstestsTests in the Lib/test dirTests in the Lib/test dirtopic-subprocessSubprocess issues.Subprocess issues.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      You are visitor 01402503 — GeezerVison does not store the pages you look up.
      Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.