GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters by vstinner · Pull Request #24843 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Mar 13, 2021 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Python no longer fails at startup with a fatal error if a command
line argument contains an invalid Unicode character.

The Py_DecodeLocale() function now escapes byte sequences which would
be decoded as Unicode characters outside the [U+0000; U+10ffff]
range.

Use MAX_UNICODE constant in unicodeobject.c.

https://bugs.python.org/issue35883

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner
bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters
3566743
Python no longer fails at startup with a fatal error if a command
line argument contains an invalid Unicode character.

The Py_DecodeLocale() function now escapes byte sequences which would
be decoded as Unicode characters outside the [U+0000; U+10ffff]
range.

Use MAX_UNICODE constant in unicodeobject.c.
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Mar 13, 2021
@bedevere-bot bedevere-bot added the awaiting core review label Mar 13, 2021
@vstinner vstinner added needs backport to 3.8 labels Mar 13, 2021
@vstinner
vstinner merged commit 9976834 into python:master Mar 17, 2021
@miss-islington

miss-islington commented Mar 17, 2021

Copy link
Copy Markdown
Contributor

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8, 3.9.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting core review label Mar 17, 2021
@vstinner
vstinner deleted the decode_locale branch March 17, 2021 20:46
@bedevere-bot

bedevere-bot commented Mar 17, 2021

Copy link
Copy Markdown

GH-24905 is a backport of this pull request to the 3.9 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.9 label Mar 17, 2021
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Mar 17, 2021
@vstinner @miss-islington
bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters (pyth…
5bf6a64
…onGH-24843)

Python no longer fails at startup with a fatal error if a command
line argument contains an invalid Unicode character.

The Py_DecodeLocale() function now escapes byte sequences which would
be decoded as Unicode characters outside the [U+0000; U+10ffff]
range.

Use MAX_UNICODE constant in unicodeobject.c.
(cherry picked from commit 9976834)

Co-authored-by: Victor Stinner <[email protected]>
@bedevere-bot

bedevere-bot commented Mar 17, 2021

Copy link
Copy Markdown

GH-24906 is a backport of this pull request to the 3.8 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.8 label Mar 17, 2021
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Mar 17, 2021
@vstinner @miss-islington
bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters (pyth…
614e836
…onGH-24843)

Python no longer fails at startup with a fatal error if a command
line argument contains an invalid Unicode character.

The Py_DecodeLocale() function now escapes byte sequences which would
be decoded as Unicode characters outside the [U+0000; U+10ffff]
range.

Use MAX_UNICODE constant in unicodeobject.c.
(cherry picked from commit 9976834)

Co-authored-by: Victor Stinner <[email protected]>
miss-islington added a commit that referenced this pull request Mar 17, 2021
@miss-islington @vstinner
bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters (GH-2…
aa967ec
…4843)

Python no longer fails at startup with a fatal error if a command
line argument contains an invalid Unicode character.

The Py_DecodeLocale() function now escapes byte sequences which would
be decoded as Unicode characters outside the [U+0000; U+10ffff]
range.

Use MAX_UNICODE constant in unicodeobject.c.
(cherry picked from commit 9976834)

Co-authored-by: Victor Stinner <[email protected]>
@bedevere-bot

bedevere-bot commented Mar 17, 2021

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 RHEL8 3.9 has failed when building commit aa967ec.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/41/builds/279) and take a look at the build logs.
  4. Check if the failure is related to this commit (aa967ec) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/41/builds/279

Summary of the results of the build (if available):

== Tests result: ENV CHANGED ==

411 tests OK.

10 slowest tests:

  • test_concurrent_futures: 3 min 56 sec
  • test_capi: 3 min 13 sec
  • test_peg_generator: 3 min 4 sec
  • test_unparse: 2 min 59 sec
  • test_tokenize: 2 min 46 sec
  • test_multiprocessing_spawn: 2 min 23 sec
  • test_asyncio: 2 min 14 sec
  • test_gdb: 2 min 10 sec
  • test_lib2to3: 2 min 2 sec
  • test_multiprocessing_forkserver: 1 min 27 sec

1 test altered the execution environment:
test_asyncio

13 tests skipped:
test_devpoll test_ioctl test_kqueue test_msilib test_ossaudiodev
test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
test_winreg test_winsound test_zipfile64

Total duration: 6 min 5 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.9.cstratak-RHEL8-aarch64/build/Lib/asyncio/sslproto.py", line 321, in __del__
    self.close()
  File "/home/buildbot/buildarea/3.9.cstratak-RHEL8-aarch64/build/Lib/asyncio/sslproto.py", line 316, in close
    self._ssl_protocol._start_shutdown()
  File "/home/buildbot/buildarea/3.9.cstratak-RHEL8-aarch64/build/Lib/asyncio/sslproto.py", line 590, in _start_shutdown
    self._abort()
  File "/home/buildbot/buildarea/3.9.cstratak-RHEL8-aarch64/build/Lib/asyncio/sslproto.py", line 731, in _abort
    self._transport.abort()
  File "/home/buildbot/buildarea/3.9.cstratak-RHEL8-aarch64/build/Lib/asyncio/selector_events.py", line 680, in abort
    self._force_close(None)
  File "/home/buildbot/buildarea/3.9.cstratak-RHEL8-aarch64/build/Lib/asyncio/selector_events.py", line 731, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/home/buildbot/buildarea/3.9.cstratak-RHEL8-aarch64/build/Lib/asyncio/base_events.py", line 746, in call_soon
    self._check_closed()
  File "/home/buildbot/buildarea/3.9.cstratak-RHEL8-aarch64/build/Lib/asyncio/base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jab added a commit to jab/cpython that referenced this pull request Mar 20, 2021
@jab
Merge branch 'master' into justin39/aiter-c
97d06e5
* master: (129 commits)
  bpo-43452: Micro-optimizations to PyType_Lookup (pythonGH-24804)
  bpo-43517: Fix false positive in detection of circular imports (python#24895)
  bpo-43494: Make some minor changes to lnotab notes (pythonGH-24861)
  Mention that code.co_lnotab is deprecated in what's new for 3.10. (python#24902)
  bpo-43244: Remove symtable.h header file (pythonGH-24910)
  bpo-43466: Add --with-openssl-rpath configure option (pythonGH-24820)
  Fix a typo in c-analyzer (pythonGH-24468)
  bpo-41561: Add workaround for Ubuntu's custom security level (pythonGH-24915)
  bpo-43521: Allow ast.unparse with empty sets and NaN (pythonGH-24897)
  bpo-43244: Remove the PyAST_Validate() function (pythonGH-24911)
  bpo-43541: Fix PyEval_EvalCodeEx() regression (pythonGH-24918)
  bpo-43244: Fix test_peg_generators on Windows (pythonGH-24913)
  bpo-39342: Expose X509_V_FLAG_ALLOW_PROXY_CERTS in ssl module (pythonGH-18011)
  bpo-43244: Fix test_peg_generator for PyAST_Validate() (pythonGH-24912)
  bpo-42128: Add 'missing :' syntax error message to match statements (pythonGH-24733)
  bpo-43244: Add pycore_ast.h header file (pythonGH-24908)
  bpo-43244: Rename pycore_ast.h to pycore_ast_state.h (pythonGH-24907)
  Remove unnecessary imports in the grammar parser (pythonGH-24904)
  bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters (pythonGH-24843)
  Add PEP 626 to what's new in 3.10. (python#24892)
  ...
vstinner added a commit that referenced this pull request Mar 29, 2021
@miss-islington @vstinner
bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters (GH-2…
3b6e61e
…4843) (GH-24906)

Python no longer fails at startup with a fatal error if a command
line argument contains an invalid Unicode character.

The Py_DecodeLocale() function now escapes byte sequences which would
be decoded as Unicode characters outside the [U+0000; U+10ffff]
range.

Use MAX_UNICODE constant in unicodeobject.c.
(cherry picked from commit 9976834)

Co-authored-by: Victor Stinner <[email protected]>

Co-authored-by: Victor Stinner <[email protected]>
@Neui Neui mannequin mentioned this pull request Apr 10, 2022
Python startup fails with a fatal error if a command line argument contains an invalid Unicode character #80064
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@vstinner @miss-islington @bedevere-bot @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.