Description

KotlinIsland

opened on Feb 28, 2023

Issue body actions

b"Ā"

👉 python test.py
  File "/test.py", line 1
    b"Ā"
         ^
SyntaxError: bytes can only contain ASCII literal characters

Here the caret is pointing to blackspace after the code

So each non-ASCII character is adding more incorrect offset:

👉 $c:temp = "b'ĀĀĀĀĀĀĀĀ'"
👉 py temp
  File "C:\temp", line 1
    b'ĀĀĀĀĀĀĀĀ'
                       ^

Expected

👉 python test.py
  File "/test.py", line 1
    b"Ā"
      ^
SyntaxError: bytes can only contain ASCII literal characters

Linked PRs

• gh-102310: Change error range for invalid bytes literals #103663
• [3.11] gh-102310: Change error range for invalid bytes literals (GH-103663) #103703

Reacted by DetachHead and Oleg Iarygin

Activity

1. KotlinIsland

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Feb 28, 2023
2. AlexWaygood

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   on Feb 28, 2023
3. arhadthedev commented on Feb 28, 2023

   

   arhadthedev

   on Feb 28, 2023

   Member

   More actions

   Can confirm on Python 3.12.0a5+ (heads/main:bcadcde712, Feb 26 2023, 12:54:46) [MSC v.1929 64 bit (AMD64)] on win32.

   Note: Ā here is UTF8-encoded as C4 80 (a single U+0100 LATIN CAPITAL LETTER A WITH MACRON). So incorrect row advancing on combined code points is totally ruled out until text normalization happens inside the parser.
4. KotlinIsland

   changed the title [-](🐞) Column marker in error message off for bytes with non ASCII character[/-] [+](🐞) Column marker in error message off when line contains non ASCII character[/+] on Feb 28, 2023
5. arhadthedev commented on Feb 28, 2023

   

   arhadthedev

   on Feb 28, 2023

   Member

   More actions

   @benjaminp, @pablogsal (as parser experts)

   update: @gvanrossum, @lysnikolaou (as new parser experts)
6. terryjreedy commented on Feb 28, 2023

   

   terryjreedy

   on Feb 28, 2023

   Member

   More actions

   In IDLE, Python 3.12.0a5+ (heads/main:0f89acf6cc, Feb 27 2023, 21:33:28) says "incomplete input" for this and "\N{a}", reported in #102312, and marks the closing quote, but this is effect of how codeop compiles.
   In REPL, I see the above and the #102312 message. I closed #102312 as duplicate.
7. terryjreedy

   added

   3.11only security fixesonly security fixes

   3.10 (EOL)end of lifeend of life

   3.12only security fixesonly security fixes

   on Feb 28, 2023
8. terryjreedy commented on Feb 28, 2023

   

   terryjreedy

   on Feb 28, 2023

   Member

   More actions

   Verified bug in installed 3.10.10, 3.11.2, and 3.12.a5.
9. sobolevn commented on Feb 28, 2023

   

   sobolevn

   on Feb 28, 2023

   Member

   More actions

   What do you think about this?

   » ./python.exe ex.py
     File "/Users/sobolev/Desktop/cpython/ex.py", line 1
       x = b"Ā"
           ^^^^^
   SyntaxError: bytes can only contain ASCII literal characters
   

   This can be achieved with:
   

   Or do we need to highlight the symbol exactly?
   If my solution is fine, I can add a test case and send a PR.

   Reacted by Oleg Iarygin
10. arhadthedev commented on Apr 21, 2023

    

    arhadthedev

    on Apr 21, 2023

    Member

    More actions

    If my solution is fine, I can add a test case and send a PR.

    @sobolevn In absence of other feedback, your solution looks better than nothing. So it's worth to create a PR and get it merged before 3.12b1.
11. sobolevn commented on Apr 21, 2023

    

    sobolevn

    on Apr 21, 2023

    Member

    More actions

    Will do :)

    Reacted by Oleg Iarygin
12. sobolevn

    added a commit that references this issue on Apr 21, 2023

    pythongh-102310: Change error range for invalid bytes literals

    9307160
13. pablogsal

    added a commit that references this issue on Apr 23, 2023

    gh-102310: Change error range for invalid bytes literals (#103663)

    0fd3891
14. miss-islington

    added a commit that references this issue on Apr 23, 2023

    pythongh-102310: Change error range for invalid bytes literals (pytho…

    b7224b9
15. pablogsal

    added a commit that references this issue on Apr 23, 2023

    [3.11] gh-102310: Change error range for invalid bytes literals (GH-1…

    7b2ac6c
16. lysnikolaou commented on Apr 25, 2023

    

    lysnikolaou

    on Apr 25, 2023

    Member

    More actions

    Can this be closed?

    Edit: Actually, I'm closing this. Re-open in case there's more to it.
17. lysnikolaou

    closed this as completedon Apr 25, 2023

Issue actions