Bug report

Summary

When an encoder returns the input str itself, TextIOWrapper.write treats it as ASCII and writes the str's internal storage to the file. For non-ASCII text the file is silently corrupted.

Expected: TypeError: encoder should return a bytes object, not 'str'.

Reproducer

We use a 44-character string that ends in an emoji, so CPython internally stores it as four bytes per character.

import codecs

class Enc(codecs.IncrementalEncoder):
    def encode(self, text, final=False):
        return text

info = codecs.CodecInfo(None, None, incrementalencoder=Enc,
                        incrementaldecoder=codecs.getincrementaldecoder("utf-8"),
                        name="passthrough")
codecs.register(lambda n: info if n == "passthrough" else None)

with open("out.txt", "w", encoding="passthrough") as f:
    f.write("The quick brown fox jumps over the lazy dog🐍")

print(open("out.txt").read())

The quick b

Only the first 44 bytes (out of 44*4 = 176) were written to the file.

Root cause

Modules/_io/textio.c, in _io_TextIOWrapper_write_impl:

The exemption for b == text is meant for the ASCII fast path, which sets b = text only for ASCII text.

A Python encoder that returns its input also satisfies it, with any text.

Suggested fix

Tie the exemption to what the fast path relies on, that the text is ASCII:

if (!PyBytes_Check(b) && !(b == text && PyUnicode_IS_ASCII(text))) {

With this the reproducer raises the TypeError above.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-161518: Reject a non-bytes encoder result unless it is the ASCII fast path #161531