GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-103578: Fix pdb reading code with non-utf8 encoding by gaogaotiantian · Pull Request #103581 · python/cpython

Conversation

@gaogaotiantian

@gaogaotiantian gaogaotiantian commented Apr 16, 2023 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

pdb should use io.open_code to open code to avoid encoding issue.

  • Issue: python -m pdb <utf-8 encoding file.py> return UnicodeDecodeError on Windows 8.1/10 (code page 936) #103578

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jingxu94, junbinhuang, infstellar, w1770946466, and ProgramRipper reacted with thumbs up emoji
All reactions
  • 👍 5 reactions
@gaogaotiantian
Fix pdb reading code with non-utf8 encoding issue
abbc7d5
@bedevere-bot bedevere-bot mentioned this pull request Apr 16, 2023
python -m pdb <utf-8 encoding file.py> return UnicodeDecodeError on Windows 8.1/10 (code page 936) #103578
Closed
@bedevere-bot bedevere-bot added the awaiting review label Apr 16, 2023
blurb-it Bot and others added 2 commits April 16, 2023 18:29
@blurb-it
📜🤖 Added by blurb_it.
cad5b05
@gaogaotiantian
Add python ext check
aba11f9
hauntsaninja
hauntsaninja reviewed Apr 24, 2023
View reviewed changes

@hauntsaninja hauntsaninja left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR!

We should probably mention in the news entry that because we use io.open_code, we'll now trigger auditing events hooks

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@hauntsaninja hauntsaninja added the needs backport to 3.11 only security fixes label Apr 24, 2023
@gaogaotiantian

gaogaotiantian commented Apr 24, 2023

Copy link
Copy Markdown
Member Author

Thanks for this PR!

We should probably mention in the news entry that because we use io.open_code, we'll now trigger auditing events

io.open() also raise auditing events. According to the docs,

However, assuming that path is a str and an absolute path, open_code(path) should always behave the same as open(path, 'rb')

io.open_code() should behave the same as io.open(path, 'rb'), so is there an auditing event difference?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hauntsaninja

hauntsaninja commented Apr 24, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Ah, I meant the PyFile_SetOpenCodeHook() hook from PEP 578. I'm not sure what the best phrase is, but it was described as auditing event in #17127. Maybe "auditing hook" as per the PEP?
(that PR got regressed in #26992, this PR fixes :-) )

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gaogaotiantian

gaogaotiantian commented Apr 24, 2023

Copy link
Copy Markdown
Member Author

So it seems like changing it to io.open_code() makes it possible to call into the hook set by PyFile_SetOpenCodeHook()? Is that something expected in the news entry?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hauntsaninja

hauntsaninja commented Apr 24, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Yup! I think can't hurt to mention it in the news entry, since it is a (good) change in behaviour and one that seems potentially noteworthy.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gaogaotiantian
Update 2023-04-16-18-29-04.gh-issue-103578.fly1wc.rst
1d8fc05
@gaogaotiantian

gaogaotiantian commented Apr 24, 2023

Copy link
Copy Markdown
Member Author

Yup! I think can't hurt to mention it in the news entry, since it is a (good) change in behaviour and one that seems potentially noteworthy.

I updated the news entry. Let me know if you think that requires extra polish!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hauntsaninja
hauntsaninja approved these changes Apr 24, 2023
View reviewed changes

@hauntsaninja hauntsaninja left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting review labels Apr 24, 2023
@hauntsaninja
Merge branch 'main' into pdb-encoding-issue
f2a96fc
@hauntsaninja
hauntsaninja merged commit 31acfd7 into python:main Apr 26, 2023
@miss-islington

miss-islington commented Apr 26, 2023

Copy link
Copy Markdown
Contributor

Thanks @gaogaotiantian for the PR, and @hauntsaninja for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting merge label Apr 26, 2023
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Apr 26, 2023
@gaogaotiantian @miss-islington
pythongh-103578: Fix pdb reading code with non-utf8 encoding (pythonG…
044efd0
…H-103581)

`pdb` should use `io.open_code` to open code to avoid encoding issue.
(cherry picked from commit 31acfd7)

Co-authored-by: Tian Gao <[email protected]>
@bedevere-bot

bedevere-bot commented Apr 26, 2023

Copy link
Copy Markdown

GH-103867 is a backport of this pull request to the 3.11 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Apr 26, 2023
@gaogaotiantian
gaogaotiantian deleted the pdb-encoding-issue branch April 26, 2023 05:13
hauntsaninja pushed a commit that referenced this pull request Apr 26, 2023
@miss-islington @gaogaotiantian
[3.11] gh-103578: Fix pdb reading code with non-utf8 encoding (GH-103581
f11ba1c
) (#103867)

`pdb` should use `io.open_code` to open code to avoid encoding issue.
(cherry picked from commit 31acfd7)

Co-authored-by: Tian Gao <[email protected]>
itamaro pushed a commit to itamaro/cpython that referenced this pull request Apr 26, 2023
@gaogaotiantian @itamaro
pythongh-103578: Fix pdb reading code with non-utf8 encoding (python#…
8c02f95
…103581)

`pdb` should use `io.open_code` to open code to avoid encoding issue.
@gaogaotiantian gaogaotiantian mentioned this pull request Apr 27, 2023
trace.__main__ does not use io.open_code #103935
Closed
@zooba zooba added needs backport to 3.8 labels Apr 27, 2023
@miss-islington

miss-islington commented Apr 27, 2023

Copy link
Copy Markdown
Contributor

Thanks @gaogaotiantian for the PR, and @hauntsaninja for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington

miss-islington commented Apr 27, 2023

Copy link
Copy Markdown
Contributor

Thanks @gaogaotiantian for the PR, and @hauntsaninja for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington

miss-islington commented Apr 27, 2023

Copy link
Copy Markdown
Contributor

Thanks @gaogaotiantian for the PR, and @hauntsaninja for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington

miss-islington commented Apr 27, 2023

Copy link
Copy Markdown
Contributor

Sorry @gaogaotiantian and @hauntsaninja, I had trouble checking out the 3.8 backport branch.
Please retry by removing and re-adding the "needs backport to 3.8" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker 31acfd78a0810f84898d36a8289e407d3754b823 3.8

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington miss-islington assigned hauntsaninja Apr 27, 2023
@miss-islington

miss-islington commented Apr 27, 2023

Copy link
Copy Markdown
Contributor

Sorry, @gaogaotiantian and @hauntsaninja, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 31acfd78a0810f84898d36a8289e407d3754b823 3.9

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington

miss-islington commented Apr 27, 2023

Copy link
Copy Markdown
Contributor

Sorry @gaogaotiantian and @hauntsaninja, I had trouble checking out the 3.10 backport branch.
Please retry by removing and re-adding the "needs backport to 3.10" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker 31acfd78a0810f84898d36a8289e407d3754b823 3.10

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@zooba zooba added needs backport to 3.10 and removed needs backport to 3.10 labels Apr 27, 2023
@miss-islington

miss-islington commented Apr 27, 2023

Copy link
Copy Markdown
Contributor

Thanks @gaogaotiantian for the PR, and @hauntsaninja for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington

miss-islington commented Apr 27, 2023

Copy link
Copy Markdown
Contributor

Sorry, @gaogaotiantian and @hauntsaninja, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 31acfd78a0810f84898d36a8289e407d3754b823 3.10

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@zooba zooba removed needs backport to 3.8 labels Apr 27, 2023
@zooba

zooba commented Apr 27, 2023

Copy link
Copy Markdown
Member

No backport needed - this was fine in 3.8-3.10 and was regressed in 3.11.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hauntsaninja hauntsaninja hauntsaninja approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@hauntsaninja hauntsaninja

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@gaogaotiantian @hauntsaninja @miss-islington @bedevere-bot @zooba
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.