GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-111201: Support pyrepl on Windows by DinoV · Pull Request #119559 · python/cpython

Conversation

@DinoV

@DinoV DinoV commented May 25, 2024 •
edited by github-actions Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

This adds Windows support for the new pyrepl.

Support is a mix of vt100 codes, enabling the vt100 codes on the Windows console, and Windows console APIs.

Minimum version required in Windows 10.

  • Issue: A new Python REPL #111201

📚 Documentation preview 📚: https://cpython-previews--119559.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

stonebig and ambv reacted with hooray emoji AlexWaygood, itamaro, marlenezw, and ambv reacted with heart emoji
All reactions
  • 🎉 2 reactions
  • ❤️ 4 reactions
DinoV and others added 30 commits May 16, 2024 18:58
@DinoV
Windows repl support
1061d22
@DinoV
Arrow key support
fa0f538
@DinoV
Make backspace clear char
6f35100
@DinoV
Fix missing newline after input
7772787
@DinoV
Make insert work
aced5ae
@DinoV
Fix delete in middle
fa3815f
@DinoV
Fix crash on invalid command key
b30b105
@DinoV
More fixes
df262c4
@DinoV
Colorize
6e57316
@DinoV
Use constants
18ecc2e
@DinoV
Use UnicodeChar
60525eb
@DinoV
Simplify
af79a17
@DinoV
Fix scrolling on input which is longer than screen/scrollback
5f99256
@tonybaloney @DinoV
fix pager typo and refactor some unused branches (python#41)
bdff535
@tonybaloney @DinoV
Implement screen clear for Windows (python#42)
88d64f5
@DinoV
Fix word wrap not being enabled in Windows Terminal
8a74306
Fix scrolling into input that has scrolled out of the buffer
@DinoV
Fix culmitive errors in wrapping as lines proceed
38e9c58
@DinoV
Fix issues with inputs longer than a single line
9cddace
@DinoV
Resize WIP
fc4efee
@tonybaloney
set compat based on Windows build version
564e6e1
@tonybaloney
use escape sequence for clearing screen and setting cursor position
3ae4316
@tonybaloney
Remove unused imports
5bb02a2
@DinoV
Windows repl support
500761e
@DinoV
Arrow key support
98f16b9
@DinoV
Make backspace clear char
911a76a
@DinoV
Fix missing newline after input
77f1c42
@DinoV
Make insert work
243817d
@DinoV
Fix delete in middle
25f51b4
@DinoV
Fix crash on invalid command key
5242239
@DinoV
More fixes
febe424
@ambv @tonybaloney
Update Lib/test/test_pyrepl/test_windows_console.py
db191a2
Co-authored-by: Anthony Shaw <[email protected]>
@lysnikolaou lysnikolaou added OS-windows topic-repl Related to the interactive shell labels May 30, 2024
zadjii-msft
zadjii-msft approved these changes May 30, 2024
View reviewed changes

@zadjii-msft zadjii-msft left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey so sorry for coming out of nowhere to leave notes on this. I'm super excited to see this land! I took a spin through this PR just to see if I could find anything egregious in here from a Windows Console standpoint, and nothing really popped out.

I obviously don't really know the pyrepl codebase closely, so my review probably isn't the most valuable. I left my own notes-to-self with a "📝:", mostly just for things I found confusing. Definitely feel free to ignore those.

overall I'd say: I'm super glad y'all aren't going to be using COOKED_READ in the console anymore 🎉

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/_pyrepl/windows_console.py
info = CONSOLE_SCREEN_BUFFER_INFO()
if not GetConsoleScreenBufferInfo(OutHandle, info):
raise WinError(GetLastError())
return info.dwCursorPosition.X, info.dwCursorPosition.Y

@zadjii-msft zadjii-msft May 30, 2024

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝: This is relative to the origin of the buffer, not of the viewport. is this supposed to be viewport relative in the unix version?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/_pyrepl/windows_console.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/_pyrepl/windows_console.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/_pyrepl/windows_console.py

return Event(evt="key", data=key, raw=rec.Event.KeyEvent.uChar.UnicodeChar)

def push_char(self, char: int | bytes) -> None:

@zadjii-msft zadjii-msft May 30, 2024

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose I don't totally understand how this is supposed to be used, but there is WriteConsoleInput which sounds similar

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@DinoV DinoV May 31, 2024

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is specifically about push_char? I don't think it's actually used but it does seem to technically be public API surface. The unix version calls this on inputted chars.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/_pyrepl/windows_console.py

self.__posxy = 0, 0
self.__gone_tall = 0
self.__offset = 0

@zadjii-msft zadjii-msft May 30, 2024

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝: the unix version also:

  • enables bracketed paste
  • does a smkx (\x1b[?1h)
  • gets the original size (to restore on exit?)

I'm pretty sure the console supported ?1h back to RS1, and bracketed paste as of microsoft/terminal#15155 (not sure exactly where in Windows 11 that gets enabled)

You're probably fine continuing to not do them here, but they could maybe get added later.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@tonybaloney tonybaloney May 30, 2024

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fwiw, this REPL is only activated on builds >= 10586
https://github.com/python/cpython/pull/119559/files#diff-45ae56fc192319dbdbe6aca3aa4c7abd31a4a5a38b775805249ba765c9476976R8

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@pablogsal

pablogsal commented May 30, 2024

Copy link
Copy Markdown
Member

Thanks for your notes @zadjii-msft!

One problem we are still having I think is that resizing the terminal when there are wide Unicode characters such as emojis in many lines doesn't work (the characters don't get resposition after a resize makes the lines warp around and back to normal). This looks like it's a windows only problem because the unix terminal handles that correctly.

I think @ambv or @DinoV may be able to provide more information.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

DinoV added 4 commits May 30, 2024 23:10
@DinoV
Use _WindowsConsoleIO for output
fba84aa
@DinoV
Merge branch 'winrepl' of github.com:DinoV/cpython into winrepl
42e3e8e
@DinoV
Remove get_abs_positon
63d854e
@DinoV
Fix tests on windows
513a148
@ambv

ambv commented May 31, 2024

Copy link
Copy Markdown
Contributor

Unicode input and emojis work now. Amazing progress! I'm landing this, we can add further improvements like bracketed paste in future updates. Thanks, @DinoV! ✨ 🍰 ✨

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ambv
ambv merged commit 0d07182 into python:main May 31, 2024
@bedevere-app bedevere-app Bot removed the awaiting core review label May 31, 2024
@bedevere-bot

bedevere-bot commented May 31, 2024

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 FreeBSD14 3.x has failed when building commit 0d07182.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1232/builds/2462) and take a look at the build logs.
  4. Check if the failure is related to this commit (0d07182) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1232/builds/2462

Failed tests:

  • test_pyrepl

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.opsec-fbsd14/build/Lib/test/libregrtest/single.py", line 181, in _runtest_env_changed_exc
    _load_run_test(result, runtests)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.opsec-fbsd14/build/Lib/test/libregrtest/single.py", line 138, in _load_run_test
    regrtest_runner(result, test_func, runtests)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.opsec-fbsd14/build/Lib/test/libregrtest/single.py", line 91, in regrtest_runner
    test_result = test_func()
  File "/home/buildbot/buildarea/3.x.opsec-fbsd14/build/Lib/test/libregrtest/single.py", line 135, in test_func
    return run_unittest(test_mod)
  File "/home/buildbot/buildarea/3.x.opsec-fbsd14/build/Lib/test/libregrtest/single.py", line 35, in run_unittest
    raise Exception("errors while loading tests")
Exception: errors while loading tests


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.opsec-fbsd14/build/Lib/unittest/loader.py", line 396, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/buildbot/buildarea/3.x.opsec-fbsd14/build/Lib/unittest/loader.py", line 339, in _get_module_from_name
    __import__(name)
    ~~~~~~~~~~^^^^^^
  File "/home/buildbot/buildarea/3.x.opsec-fbsd14/build/Lib/test/test_pyrepl/test_windows_console.py", line 5, in <module>
    from _pyrepl.windows_console import (
    ...<5 lines>...
    )
  File "/home/buildbot/buildarea/3.x.opsec-fbsd14/build/Lib/_pyrepl/windows_console.py", line 30, in <module>
    import ctypes
  File "/home/buildbot/buildarea/3.x.opsec-fbsd14/build/Lib/ctypes/__init__.py", line 8, in <module>
    from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ambv

ambv commented May 31, 2024

Copy link
Copy Markdown
Contributor

Buildbot failure related. Fixing now.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ambv ambv mentioned this pull request May 31, 2024
gh-111201: [pyrepl] Ensure optional platform-specific imports are optional #119834
Merged
@ambv ambv added needs backport to 3.13 only security fixes labels May 31, 2024
@miss-islington-app

miss-islington-app Bot commented May 31, 2024

Copy link
Copy Markdown

Thanks @DinoV for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington-app

miss-islington-app Bot commented May 31, 2024

Copy link
Copy Markdown

Thanks @DinoV for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 31, 2024
@DinoV @tonybaloney
@ambv @miss-islington
pythongh-111201: Support pyrepl on Windows (pythonGH-119559)
60332af
(cherry picked from commit 0d07182)

Co-authored-by: Dino Viehland <[email protected]>
Co-authored-by: Anthony Shaw <[email protected]>
Co-authored-by: Łukasz Langa <[email protected]>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 31, 2024
@DinoV @tonybaloney
@ambv @miss-islington
pythongh-111201: Support pyrepl on Windows (pythonGH-119559)
d03284b
(cherry picked from commit 0d07182)

Co-authored-by: Dino Viehland <[email protected]>
Co-authored-by: Anthony Shaw <[email protected]>
Co-authored-by: Łukasz Langa <[email protected]>
@bedevere-app

bedevere-app Bot commented May 31, 2024

Copy link
Copy Markdown

GH-119850 is a backport of this pull request to the 3.13 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 only security fixes label May 31, 2024
@zadjii-msft

zadjii-msft commented May 31, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

One problem we are still having I think is that resizing the terminal when there are wide Unicode characters such as emojis in many lines doesn't work (the characters don't get resposition after a resize makes the lines warp around and back to normal). This looks like it's a windows only problem because the unix terminal handles that correctly

Oh that for sure sounds like a us bug. We've been making pretty dramatic improvements to the unicode / emoji support over the last couple years, so I'm guessing you're seeing different behavior even depending on what version of Windows & the Terminal you're running.

If you've got specific bugs on your tracker for those, point me at 'em, and I can get the team to double check them ☺️


EDIT: derp I only read one mail, then replied to it before I saw #119559 (comment).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

DHowett
DHowett reviewed May 31, 2024
View reviewed changes
Comment thread Lib/_pyrepl/__main__.py
if sys.platform != "win32":
CAN_USE_PYREPL = True
else:
CAN_USE_PYREPL = sys.getwindowsversion().build >= 10586 # Windows 10 TH2

@DHowett DHowett May 31, 2024

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This merged as-is which is OK, but I wanted to chip in...

There is a minimal risk of cutting out terminals that would support pyrepl on earlier versions of Windows. The "most correct" (also most soapboxy, sorry!) way to determine support for this is to just try to set ENABLE_VIRTUAL_TERMINAL_PROCESSING and only fall back to basic console APIs if it fails. :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@ambv ambv May 31, 2024

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DHowett sounds like an improvement we can still make! Care to PR this?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@DHowett DHowett May 31, 2024

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try - at the very least, it would be a great opportunity for me to learn how to build and deploy Python 🙂

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@ambv ambv May 31, 2024

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DHowett see here and let me know if anything's missing:
https://devguide.python.org/getting-started/setup-building/#windows

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
ambv added a commit that referenced this pull request May 31, 2024
@miss-islington @DinoV
@tonybaloney @ambv
[3.13] gh-111201: Support pyrepl on Windows (GH-119559) (GH-119850)
d6faac6
(cherry picked from commit 0d07182)

Co-authored-by: Dino Viehland <[email protected]>
Co-authored-by: Anthony Shaw <[email protected]>
Co-authored-by: Łukasz Langa <[email protected]>
estyxx pushed a commit to estyxx/cpython that referenced this pull request Jul 17, 2024
@DinoV @tonybaloney
@ambv @estyxx
pythongh-111201: Support pyrepl on Windows (python#119559)
e637683
Co-authored-by: Anthony Shaw <[email protected]>
Co-authored-by: Łukasz Langa <[email protected]>
Jose1654
Jose1654 approved these changes May 3, 2025
View reviewed changes
@bedevere-app bedevere-app Bot added the awaiting core review label May 3, 2025
Jose1654
Jose1654 approved these changes May 3, 2025
View reviewed changes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AlexWaygood AlexWaygood AlexWaygood left review comments
@pablogsal pablogsal Awaiting requested review from pablogsal pablogsal is a code owner
@lysnikolaou lysnikolaou Awaiting requested review from lysnikolaou lysnikolaou is a code owner
@ambv ambv Awaiting requested review from ambv ambv is a code owner
+4 more reviewers
@DHowett DHowett DHowett left review comments
@tonybaloney tonybaloney tonybaloney left review comments
@zadjii-msft zadjii-msft zadjii-msft approved these changes
@Jose1654 Jose1654 Jose1654 approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting core review OS-windows topic-repl Related to the interactive shell

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

10 participants

@DinoV @AlexWaygood @pablogsal @ambv @bedevere-bot @zadjii-msft @DHowett @tonybaloney @Jose1654 @lysnikolaou
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.