GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-130804: Fix support of typing unicode chars in pyrepl by sergey-miryanov · Pull Request #130805 · python/cpython

Conversation

@sergey-miryanov

@sergey-miryanov sergey-miryanov commented Mar 3, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Fixed typing of unicode chars (I tested on Cyrillic chars on Windows)

  • Issue: [Windows] New REPL doesn't allow to input non-ASCII Unicode characters #130804

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@sergey-miryanov
Fix support of typing unicode chars in pyrepl
3d674cf
@sergey-miryanov
sergey-miryanov requested review from ambv, lysnikolaou and pablogsal as code owners March 3, 2025 18:57
@bedevere-app bedevere-app Bot added the awaiting review label Mar 3, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Mar 3, 2025
[Windows] New REPL doesn't allow to input non-ASCII Unicode characters #130804
Closed
sergey-miryanov added 3 commits March 4, 2025 00:45
@sergey-miryanov
Make mypy happy
c1f3aa8
@sergey-miryanov
Add assert to check if passed char is str
ef0d0ee
@sergey-miryanov
Merge branch 'main' into pythongh-130804-pyrepl-unicode-chars
497a0bb
encukou
encukou reviewed Mar 10, 2025
View reviewed changes

@encukou encukou left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does fix the issue on Windows. I've also tested on Linux and saw no breakage.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/_pyrepl/base_eventqueue.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

sergey-miryanov and others added 2 commits March 11, 2025 01:06
@sergey-miryanov @encukou
Update Lib/_pyrepl/base_eventqueue.py
bfa2e4e
Co-authored-by: Petr Viktorin <[email protected]>
@sergey-miryanov
Merge branch 'main' into pythongh-130804-pyrepl-unicode-chars
c667049
pablogsal
pablogsal approved these changes Mar 10, 2025
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Mar 10, 2025
@pablogsal pablogsal added needs backport to 3.13 only security fixes and removed awaiting merge labels Mar 10, 2025
@pablogsal

pablogsal commented Mar 10, 2025

Copy link
Copy Markdown
Member

LGTM

Thanks a lot for the fix @sergey-miryanov

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal
Add NEWS entry
7cc16ea
@pablogsal
pablogsal enabled auto-merge (squash) March 10, 2025 21:48
@pablogsal
pablogsal disabled auto-merge March 10, 2025 21:48
@pablogsal
pablogsal enabled auto-merge (squash) March 10, 2025 21:48
@pablogsal
pablogsal merged commit 7c98b06 into python:main Mar 10, 2025
@miss-islington-app

miss-islington-app Bot commented Mar 10, 2025

Copy link
Copy Markdown

Thanks @sergey-miryanov for the PR, and @pablogsal for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington-app

miss-islington-app Bot commented Mar 10, 2025

Copy link
Copy Markdown

Sorry, @sergey-miryanov and @pablogsal, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 7c98b0674daa3e4eb3e8f35afb61a0dba61d1780 3.13
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington-app miss-islington-app Bot assigned pablogsal Mar 10, 2025
@StanFromIreland

StanFromIreland commented Mar 10, 2025

Copy link
Copy Markdown
Member

Tests have failed.
workflow

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented Mar 10, 2025

Copy link
Copy Markdown
Member

How was the PR landed then?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented Mar 10, 2025

Copy link
Copy Markdown
Member

Test were ok in https://github.com/python/cpython/actions/runs/13775445246/job/38523588646

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@StanFromIreland

StanFromIreland commented Mar 10, 2025

Copy link
Copy Markdown
Member

It's odd, it's the same set up yet somehow they are all failing now?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented Mar 10, 2025

Copy link
Copy Markdown
Member

This PR also only modifies test_multiple_statements_fail_early so I don't see how is this related

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented Mar 10, 2025

Copy link
Copy Markdown
Member

Reverting doesn't fix anything:

~/github/python/main main ⇡
❯ git revert 7c98b0674daa3e4eb3e8f35afb61a0dba61d1780
[main 46a7449b8e9] Revert "gh-130804: Fix support of typing unicode chars in pyrepl (#130805)"
 3 files changed, 3 insertions(+), 17 deletions(-)
 delete mode 100644 Misc/NEWS.d/next/Core_and_Builtins/2025-03-10-21-46-37.gh-issue-130804.0PpcTx.rst

~/github/python/main main ⇡
❯ make -j -s
Checked 112 modules (34 built-in, 77 shared, 1 n/a on linux-x86_64, 0 disabled, 0 missing, 0 failed on import)

~/github/python/main main ⇡
❯ ./python -m test test_repl -vvv
== CPython 3.14.0a5+ (heads/main:7c98b0674da, Mar 10 2025, 22:34:19) [GCC 14.2.1 20250207]
== Linux-6.13.5-arch1-1-x86_64-with-glibc2.41 little-endian
== Python build: debug
== cwd: /home/pablogsal/github/python/main/build/test_python_worker_1544359æ
== CPU count: 36
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 1616529274
0:00:00 load avg: 9.94 Run 1 test sequentially in a single process
0:00:00 load avg: 9.94 [1/1] test_repl
test_multiple_statements_fail_early (test.test_repl.TestAsyncioREPL.test_multiple_statements_fail_early) ... FAIL
test_toplevel_contextvars_async (test.test_repl.TestAsyncioREPL.test_toplevel_contextvars_async) ... ok
test_toplevel_contextvars_sync (test.test_repl.TestAsyncioREPL.test_toplevel_contextvars_sync) ... ok
test_asyncio_repl_is_ok (test.test_repl.TestInteractiveInterpreter.test_asyncio_repl_is_ok) ... ok
test_asyncio_repl_reaches_python_startup_script (test.test_repl.TestInteractiveInterpreter.test_asyncio_repl_reaches_python_startup_script) ... ok
test_close_stdin (test.test_repl.TestInteractiveInterpreter.test_close_stdin) ... ok
test_interactive_source_is_in_linecache (test.test_repl.TestInteractiveInterpreter.test_interactive_source_is_in_linecache) ... ok
test_interactive_traceback_reporting (test.test_repl.TestInteractiveInterpreter.test_interactive_traceback_reporting) ... ok
test_interactive_traceback_reporting_multiple_input (test.test_repl.TestInteractiveInterpreter.test_interactive_traceback_reporting_multiple_input) ... ok
test_multiline_string_parsing (test.test_repl.TestInteractiveInterpreter.test_multiline_string_parsing) ... ok
test_no_memory (test.test_repl.TestInteractiveInterpreter.test_no_memory) ... ok
test_runsource_show_syntax_error_location (test.test_repl.TestInteractiveInterpreter.test_runsource_show_syntax_error_location) ... ok
test_interactive_syntax_error_correct_line (test.test_repl.TestInteractiveModeSyntaxErrors.test_interactive_syntax_error_correct_line) ... ok

======================================================================
FAIL: test_multiple_statements_fail_early (test.test_repl.TestAsyncioREPL.test_multiple_statements_fail_early)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pablogsal/github/python/main/Lib/test/test_repl.py", line 304, in test_multiple_statements_fail_early
    self.assertNotIn("afterwards", output)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'afterwards' unexpectedly found in 'asyncio REPL 3.14.0a5+ (heads/main:7c98b0674da, Mar 10 2025, 22:34:19) [GCC 14.2.1 20250207] on linux\nUse "await" directly instead of "asyncio.run()".\nType "help", "copyright", "credits" or "license" for more information.\n>>> import asyncio\n>>> Traceback (most recent call last):\n  File "/home/pablogsal/github/python/main/Lib/concurrent/futures/_base.py", line 448, in result\n    return self.__get_result()\n           ~~~~~~~~~~~~~~~~~^^\n  File "/home/pablogsal/github/python/main/Lib/concurrent/futures/_base.py", line 393, in __get_result\n    raise self._exception\n  File "/home/pablogsal/github/python/main/Lib/asyncio/__main__.py", line 42, in callback\n    coro = func()\n  File "<stdin>", line 1, in <module>\n    1 / 0; print(\'afterwards\')\n    ~~^~~\nZeroDivisionError: division by zero\n>>> \nexiting asyncio REPL...\nTraceback (most recent call last):\n  File "/home/pablogsal/github/python/main/Lib/runpy.py", line 198, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n                     "__main__", mod_spec)\n  File "/home/pablogsal/github/python/main/Lib/runpy.py", line 88, in _run_code\n    exec(code, run_globals)\n    ~~~~^^^^^^^^^^^^^^^^^^^\n  File "/home/pablogsal/github/python/main/Lib/asyncio/__main__.py", line 204, in <module>\n    sys.exit(return_code)\n    ~~~~~~~~^^^^^^^^^^^^^\nSystemExit: 0\n>>> \n'

----------------------------------------------------------------------
Ran 13 tests in 1.223s

FAILED (failures=1)
test test_repl failed
0:00:01 load avg: 9.94 [1/1/1] test_repl failed (1 failure)

== Tests result: FAILURE ==

1 test failed:
    test_repl

Total duration: 1.3 sec
Total tests: run=13 failures=1
Total test files: run=1/1 failed=1
Result: FAILURE
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@StanFromIreland

StanFromIreland commented Mar 10, 2025

Copy link
Copy Markdown
Member

Its this PR: 6ab5c4a

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented Mar 10, 2025

Copy link
Copy Markdown
Member

Doesn't seem like it:

~/github/python/main main ⇡
❯ git revert 6ab5c4aa05bf35832a3ccd1e71b28b8475fa30f4
[main 1428e541980] Revert "gh-124927: Fix conversion issue between coordinates and position in REPL (#125001)"
 4 files changed, 1 insertion(+), 12 deletions(-)
 delete mode 100644 Misc/NEWS.d/next/Library/2024-10-05-13-25-07.gh-issue-124927.uzNA32.rst

~/github/python/main main ⇡
❯ make
Checked 112 modules (34 built-in, 77 shared, 1 n/a on linux-x86_64, 0 disabled, 0 missing, 0 failed on import)
./python -E ./Tools/build/generate-build-details.py `cat pybuilddir.txt`/build-details.json

~/github/python/main main ⇡
❯ ./python -m test test_repl -vvv
== CPython 3.14.0a5+ (bisect/good-ecdf6b15b0c0570c3c3302ab95bdbfd3007ea941-2-g6ab5c4aa05b:6ab5c4aa05b,) [GCC 14.2.1 20250207]
== Linux-6.13.5-arch1-1-x86_64-with-glibc2.41 little-endian
== Python build: debug
== cwd: /home/pablogsal/github/python/main/build/test_python_worker_1588881æ
== CPU count: 36
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 4038269957
0:00:00 load avg: 3.85 Run 1 test sequentially in a single process
0:00:00 load avg: 3.85 [1/1] test_repl
test_multiple_statements_fail_early (test.test_repl.TestAsyncioREPL.test_multiple_statements_fail_early) ... FAIL
test_toplevel_contextvars_async (test.test_repl.TestAsyncioREPL.test_toplevel_contextvars_async) ... ok
test_toplevel_contextvars_sync (test.test_repl.TestAsyncioREPL.test_toplevel_contextvars_sync) ... ok
test_asyncio_repl_is_ok (test.test_repl.TestInteractiveInterpreter.test_asyncio_repl_is_ok) ... ok
test_asyncio_repl_reaches_python_startup_script (test.test_repl.TestInteractiveInterpreter.test_asyncio_repl_reaches_python_startup_script) ... ok
test_close_stdin (test.test_repl.TestInteractiveInterpreter.test_close_stdin) ... ok
test_interactive_source_is_in_linecache (test.test_repl.TestInteractiveInterpreter.test_interactive_source_is_in_linecache) ... ok
test_interactive_traceback_reporting (test.test_repl.TestInteractiveInterpreter.test_interactive_traceback_reporting) ... ok
test_interactive_traceback_reporting_multiple_input (test.test_repl.TestInteractiveInterpreter.test_interactive_traceback_reporting_multiple_input) ... ok
test_multiline_string_parsing (test.test_repl.TestInteractiveInterpreter.test_multiline_string_parsing) ... ok
test_no_memory (test.test_repl.TestInteractiveInterpreter.test_no_memory) ... ok
test_runsource_show_syntax_error_location (test.test_repl.TestInteractiveInterpreter.test_runsource_show_syntax_error_location) ... ok
test_interactive_syntax_error_correct_line (test.test_repl.TestInteractiveModeSyntaxErrors.test_interactive_syntax_error_correct_line) ... ok

======================================================================
FAIL: test_multiple_statements_fail_early (test.test_repl.TestAsyncioREPL.test_multiple_statements_fail_early)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pablogsal/github/python/main/Lib/test/test_repl.py", line 304, in test_multiple_statements_fail_early
    self.assertNotIn("afterwards", output)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'afterwards' unexpectedly found in 'asyncio REPL 3.14.0a5+ (bisect/good-ecdf6b15b0c0570c3c3302ab95bdbfd3007ea941-2-g6ab5c4aa05b:6ab5c4aa05b,) [GCC 14.2.1 20250207] on linux\nUse "await" directly instead of "asyncio.run()".\nType "help", "copyright", "credits" or "license" for more information.\n>>> import asyncio\n>>> Traceback (most recent call last):\n  File "/home/pablogsal/github/python/main/Lib/concurrent/futures/_base.py", line 448, in result\n    return self.__get_result()\n           ~~~~~~~~~~~~~~~~~^^\n  File "/home/pablogsal/github/python/main/Lib/concurrent/futures/_base.py", line 393, in __get_result\n    raise self._exception\n  File "/home/pablogsal/github/python/main/Lib/asyncio/__main__.py", line 42, in callback\n    coro = func()\n  File "<stdin>", line 1, in <module>\n    1 / 0; print(\'afterwards\')\n    ~~^~~\nZeroDivisionError: division by zero\n>>> \nexiting asyncio REPL...\nTraceback (most recent call last):\n  File "/home/pablogsal/github/python/main/Lib/runpy.py", line 198, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n                     "__main__", mod_spec)\n  File "/home/pablogsal/github/python/main/Lib/runpy.py", line 88, in _run_code\n    exec(code, run_globals)\n    ~~~~^^^^^^^^^^^^^^^^^^^\n  File "/home/pablogsal/github/python/main/Lib/asyncio/__main__.py", line 204, in <module>\n    sys.exit(return_code)\n    ~~~~~~~~^^^^^^^^^^^^^\nSystemExit: 0\n>>> \n'

----------------------------------------------------------------------
Ran 13 tests in 1.400s

FAILED (failures=1)
test test_repl failed
0:00:01 load avg: 3.85 [1/1/1] test_repl failed (1 failure)

== Tests result: FAILURE ==

1 test failed:
    test_repl

Total duration: 1.5 sec
Total tests: run=13 failures=1
Total test files: run=1/1 failed=1
Result: FAILURE
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@StanFromIreland

StanFromIreland commented Mar 10, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Weird? Only one left then would be yours a931a8b maybe it has something to do with the traceback modifications?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented Mar 10, 2025

Copy link
Copy Markdown
Member

I'm pretty sure it's a931a8b

StanFromIreland reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented Mar 10, 2025

Copy link
Copy Markdown
Member

#131065

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented Mar 10, 2025

Copy link
Copy Markdown
Member

@sergey-miryanov Can you check #130805 (comment) ?

Seems the backport needs some adaptations to 3.13. Could you take a look?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@sergey-miryanov

sergey-miryanov commented Mar 11, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Seems the backport needs some adaptations to 3.13. Could you take a look?

@pablogsal
There is a conflict because a65366e doesn't backported to 3.13 and this PR fixes regression from this commit. Should we backport a65366e first?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal

pablogsal commented Mar 11, 2025

Copy link
Copy Markdown
Member

That's tricky. That may be considered a feature so I am not sure we can. @ambv what do you think?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@sergey-miryanov sergey-miryanov mentioned this pull request Mar 29, 2025
New REPL on Windows exits when accented character is pasted/typed #131878
Closed
chris-eibl added a commit to chris-eibl/cpython that referenced this pull request Apr 26, 2025
@chris-eibl
Revert "pythongh-130804: Fix support of typing unicode chars in pyrepl (
a06c42a
python#130805)"

This reverts commit 7c98b06.
@chris-eibl chris-eibl mentioned this pull request Apr 26, 2025
gh-131878: Fix input of unicode characters with two or more code points in the REPL on Windows in vt mode #133030
Closed
This was referenced May 5, 2025
gh-131878: Fix input of unicode characters with two or more code points in new pyrepl on Windows #131901
Merged
GH-132439: Fix REPL swallowing characters entered with AltGr on cmd.exe #132440
Merged
ambv pushed a commit to ambv/cpython that referenced this pull request May 5, 2025
@sergey-miryanov @ambv
[3.13] pythongh-130804: Fix support of typing unicode chars in pyrepl (…
e980f8b
…pythonGH-130805)

(cherry picked from commit 7c98b06)

Co-authored-by: Sergey Miryanov <[email protected]>
@bedevere-app

bedevere-app Bot commented May 5, 2025

Copy link
Copy Markdown

GH-133462 is a backport of this pull request to the 3.13 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 only security fixes label May 5, 2025
ambv added a commit that referenced this pull request May 5, 2025
@ambv @sergey-miryanov
[3.13] gh-130804: Fix support of typing unicode chars in pyrepl (GH-1…
76f52c4
…30805) (GH-133462)

(cherry picked from commit 7c98b06)

Co-authored-by: Sergey Miryanov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@encukou encukou encukou left review comments
@pablogsal pablogsal pablogsal approved these changes
@lysnikolaou lysnikolaou Awaiting requested review from lysnikolaou lysnikolaou is a code owner
@ambv ambv Awaiting requested review from ambv ambv is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@pablogsal pablogsal

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@sergey-miryanov @pablogsal @StanFromIreland @encukou
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.