GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-96670: Raise SyntaxError when parsing NULL bytes by pablogsal · Pull Request #97594 · python/cpython

Conversation

@pablogsal

@pablogsal pablogsal commented Sep 27, 2022 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member
  • Issue: mishandling of c-strings in parser #96670

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
asottile reacted with hooray emoji
All reactions
  • 🎉 1 reaction
@pablogsal
pablogsal requested a review from lysnikolaou as a code owner September 27, 2022 11:00
@bedevere-bot bedevere-bot added the awaiting core review label Sep 27, 2022
@pablogsal
pablogsal requested a review from gvanrossum September 27, 2022 11:00
@pablogsal
pablogsal force-pushed the gh-96670 branch 5 times, most recently from 97a8f83 to cb89392 Compare September 27, 2022 12:25
gvanrossum
gvanrossum reviewed Sep 27, 2022
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's up with python3.12.abi.new having 22,644 additions?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
gvanrossum
gvanrossum reviewed Sep 27, 2022
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG except for one word in an error message, I think.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Parser/tokenizer.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@pablogsal

pablogsal commented Sep 27, 2022

Copy link
Copy Markdown
Member Author

What's up with python3.12.abi.new having 22,644 additions?

Wow, that's some error. Let me clean that up
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@pablogsal
pablogsal force-pushed the gh-96670 branch from cb89392 to 490f5bd Compare September 27, 2022 18:15
@pablogsal
pablogsal requested a review from gvanrossum September 27, 2022 18:15
lysnikolaou
lysnikolaou reviewed Sep 27, 2022
View reviewed changes
Comment thread Objects/fileobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@pablogsal
pablogsal force-pushed the gh-96670 branch from 490f5bd to 4b1105c Compare September 27, 2022 18:20
@pablogsal
pythongh-96670: Raise SyntaxError when parsing NULL bytes
e10909d 
Signed-off-by: Pablo Galindo <[email protected]>
@pablogsal
pablogsal force-pushed the gh-96670 branch from 4b1105c to e10909d Compare September 27, 2022 18:38
@gvanrossum

gvanrossum commented Sep 27, 2022

Copy link
Copy Markdown
Member

Ugh, force-push. :-(
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
gvanrossum
gvanrossum reviewed Sep 27, 2022
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see nothing else wrong, let's wait for Lysandros' LGTM.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
lysnikolaou
lysnikolaou approved these changes Sep 27, 2022
View reviewed changes

@lysnikolaou lysnikolaou left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as well!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Sep 27, 2022
@mdboom

mdboom commented Sep 27, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

I was going to ask "doesn't this break things for encodings where NULLs are normal, like UTF-16", but then realized we never supported that anyway:

PEP 253: "It does not include encodings which use two or more bytes for all characters like e.g. UTF-16. The reason for this is to keep the encoding detection algorithm in the tokenizer simple."

In that sense, we get a better error message for UTF-16 files with this PR than before. Prior, it would silently fail (behave as if an empty file), and with this PR:

  File "x.py", line 1
    #
SyntaxError: source code cannot contain null bytes
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@pablogsal
pablogsal merged commit aab01e3 into python:main Sep 27, 2022
@pablogsal
pablogsal deleted the gh-96670 branch September 27, 2022 22:23
@bedevere-bot bedevere-bot removed the awaiting merge label Sep 27, 2022
@pablogsal

pablogsal commented Sep 27, 2022

Copy link
Copy Markdown
Member Author

Landing, thank you everyone for your great review and comments! And also thank @mdboom for checking the UTF-16, that was a good consideration :)

P.S. Sorry for the force push :S
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner vstinner mentioned this pull request Sep 28, 2022
mishandling of c-strings in parser #96670
Closed
@bskinn bskinn mentioned this pull request Apr 24, 2023
Null characters in strings cause a C SystemError #97556
Closed
lysnikolaou added a commit to lysnikolaou/cpython that referenced this pull request May 5, 2023
@lysnikolaou @pablogsal
[3.11] pythongh-96670: Raise SyntaxError when parsing NULL bytes (pyt…
e6d28ba 
…honGH-97594).

(cherry picked from commit aab01e3)

Co-authored-by: Pablo Galindo Salgado <[email protected]>
@bedevere-bot

bedevere-bot commented May 5, 2023

Copy link
Copy Markdown

GH-104195 is a backport of this pull request to the 3.11 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
pablogsal pushed a commit that referenced this pull request May 7, 2023
@lysnikolaou
[3.11] gh-96670: Raise SyntaxError when parsing NULL bytes (GH-97594) (…
a09d390 
…#104195)
@mkniewallner mkniewallner mentioned this pull request Jun 18, 2023
fix(imports): handle SyntaxError osprey-oss/deptry#426
Merged
1 of 4 tasks
@bdraco bdraco mentioned this pull request May 12, 2024
Crash 2024.5.2 - docker home-assistant/core#117160
Closed
This was referenced Jul 30, 2024
Incorrect information about exception raised if source contains null bytes for ast.parse() and compile() #122461
Open
gh-122461: Document that compile() and ast.parse() raise SyntaxError for null bytes #122462
Open
@ashm-dev ashm-dev mentioned this pull request Nov 1, 2026
gh-97556: Fix crash on null characters in string literal parser #97577
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gvanrossum gvanrossum gvanrossum left review comments
@lysnikolaou lysnikolaou lysnikolaou approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@pablogsal @gvanrossum @mdboom @bedevere-bot @lysnikolaou
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.