Description

StanFromIreland

opened on Feb 25, 2026

Issue body actions

Bug report

Bug description:

Found by OSS-Fuzz.

The testcase is:

#coding=U7+AA0''

resulting in:

$ ./python -c "import py_compile; py_compile.compile('testcase')"
Sorry: SystemError: Parser/string_parser.c:286: bad argument to internal function

I gather this is because +AA0 decodes to \r in UTF-7, however, _PyTokenizer_translate_newlines is called before the codec decoding so it slips by and blows up (at least it apologizes😆). I think it could be fixed by calling _PyTokenizer_translate_newlines again after decoding if a \r is introduced, WDYT?

cc @lysnikolaou @pablogsal

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

• gh-145234: Fix SystemError in parser when \r is introduced after code… #145276
• gh-145234: Normalize decoded CR in string tokenizer #145281
• [3.14] gh-145234: Normalize decoded CR in string tokenizer (GH-145281) #145310
• [3.13] gh-145234: Normalize decoded CR in string tokenizer (GH-145281) #145312

Activity

1. StanFromIreland

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   topic-parser

   on Feb 25, 2026
2. gourijain029-del

   added a commit that references this issue on Feb 26, 2026

   pythongh-145234: Fix SystemError in parser when \r is introduced afte…

   ccf6915
3. pablogsal

   added 4 commits that reference this issue on Feb 26, 2026

   pythongh-145234: Normalize decoded CR in string tokenizer

   d701ca0

   pythongh-145234: Add patch attribution in NEWS entry

   6545f1f

   pythongh-145234: keep tokenizer source pointer after UTF-8 translation

   cb60a16

   gh-145234: Normalize decoded CR in string tokenizer (#145281)

   98b1e51
4. miss-islington

   added a commit that references this issue on Feb 27, 2026

   pythongh-145234: Normalize decoded CR in string tokenizer (pythonGH-1…

   0142d1c
5. pablogsal

   added 3 commits that reference this issue on Feb 27, 2026

   [3.14] gh-145234: Normalize decoded CR in string tokenizer (GH-145281) (

   f04aefa

   [3.13] pythongh-145234: Normalize decoded CR in string tokenizer (pyt…

   cf5c995

   [3.13] gh-145234: Normalize decoded CR in string tokenizer (GH-145281) (

   2daece9
6. pablogsal

   closed this as completedon Feb 28, 2026
7. pablogsal commented on Feb 28, 2026

   

   pablogsal

   on Feb 28, 2026

   Member

   More actions

   Thanks for the report @StanFromIreland !

   Reacted by Stan Ulbrych
8. brijkapadia

   added a commit that references this issue on Feb 28, 2026

   pythongh-145234: Normalize decoded CR in string tokenizer (python#145281

   aca1c73
9. ljfp

   added a commit that references this issue on Apr 25, 2026

   pythongh-145234: Normalize decoded CR in string tokenizer (python#145281

   90949fa

Issue actions