GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — GH-130328: further speedup of pasting in new REPL on Windows by reading in chunks by chris-eibl · Pull Request #132889 · python/cpython

GH-130328: further speedup of pasting in new REPL on Windows by reading in chunks #132889

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

chris-eibl wants to merge 14 commits into python:main from chris-eibl:read_chunked

Uh oh!

There was an error while loading. Please reload this page.

Closed

GH-130328: further speedup of pasting in new REPL on Windows by reading in chunks #132889

Changes from all commits

Commits

Show all changes

14 commits

Select commit Hold shift + click to select a range

0a38c2e

fix WindowsConsole.wait()

chris-eibl Apr 24, 2025

05d26ff

blurb it

chris-eibl Apr 24, 2025

cad6bfb

fix timeout=None

chris-eibl Apr 24, 2025

3d1da66

fix mypy for ctypes.get_last_error

chris-eibl Apr 24, 2025

4098e3d

fix return value of wait() and use it in _read_input()

chris-eibl Apr 24, 2025

a64f4c4

read in chunks

chris-eibl Apr 24, 2025

937b6bd

fix missing return in getpending

chris-eibl Apr 24, 2025

bba26c9

use self.paste_mode instead of self.in_bracketed_paste

chris-eibl Apr 28, 2025

9794664

Merge branch 'fix_wait' into read_chunked

chris-eibl Apr 28, 2025

3c0cae9

Revert "use self.paste_mode instead of self.in_bracketed_paste"

chris-eibl Apr 28, 2025

b6ea252

Merge branch 'fix_wait' into read_chunked

chris-eibl Apr 28, 2025

8fa84d1

Merge branch 'main' into read_chunked

chris-eibl May 2, 2025

5b055f1

remove copying of the raw member of Event, because

chris-eibl May 2, 2025

a7ebeed

Merge branch 'read_chunked' of https://github.com/chris-eibl/cpython …

chris-eibl May 2, 2025

File filter

Filter by extension .py (1) All 1 file type selected

Viewed files

Clear filters

Conversations

Failed to load comments.

Uh oh!

There was an error while loading. Please reload this page.

Jump to

Jump to file

Failed to load files.

Uh oh!

There was an error while loading. Please reload this page.

Diff view

Unified

Split

Hide whitespace

Diff view

Unified

Split

Hide whitespace

There are no files selected for viewing

43 changes: 38 additions & 5 deletions 43 Lib/_pyrepl/windows_console.py

Show comments

View file

Open in desktop

Original file line number Diff line number Diff line change

Expand Up @@ -64,6 +64,8 @@ def __init__(self, err: int | None, descr: str | None = None) -> None:

if TYPE_CHECKING:

from typing import IO

INPUT_BUFFER_LEN = 10 * 1024

# Virtual-Key Codes: https://learn.microsoft.com/en-us/windows/win32/inputdev/virtual-key-codes

VK_MAP: dict[int, str] = {

0x23: "end", # VK_END

Expand Down Expand Up @@ -165,6 +167,10 @@ def __init__(

# Console I/O is redirected, fallback...

self.out = None

self.input_buffer = (INPUT_RECORD * INPUT_BUFFER_LEN)()

self.input_buffer_pos = 0

self.input_buffer_count = 0

def refresh(self, screen: list[str], c_xy: tuple[int, int]) -> None:

"""

Refresh the console screen.

Expand Down Expand Up @@ -415,16 +421,27 @@ def _getscrollbacksize(self) -> int:

return info.srWindow.Bottom # type: ignore[no-any-return]

def more_in_buffer(self) -> bool:

return self.input_buffer_pos < self.input_buffer_count - 1

def _read_input(self, block: bool = True) -> INPUT_RECORD | None:

if not block and not self.wait(timeout=0):

return None

rec = INPUT_RECORD()

if self.more_in_buffer():

self.input_buffer_pos += 1

return self.input_buffer[self.input_buffer_pos]

# read next chunk

self.input_buffer_pos = 0

self.input_buffer_count = 0

read = DWORD()

if not ReadConsoleInput(InHandle, rec, 1, read):

if not ReadConsoleInput(InHandle, self.input_buffer, INPUT_BUFFER_LEN, read):

raise WinError(GetLastError())

return rec

self.input_buffer_count = read.value

return self.input_buffer[0]

def get_event(self, block: bool = True) -> Event | None:

"""Return an Event instance. Returns None if |block| is false

Expand Down Expand Up @@ -521,9 +538,15 @@ def forgetinput(self) -> None:

def getpending(self) -> Event:

"""Return the characters that have been typed but not yet

processed."""

return Event("key", "", b"")

e = Event("key", "", b"")

Comment thread

chris-eibl Apr 24, 2025

Copy link

Copy Markdown

Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just copy-pasted this from

cpython/Lib/_pyrepl/unix_console.py

Lines 536 to 555 in e1c09ff

	def getpending(self):
	"""
	Get pending events from the console event queue.

	Returns:
	- Event: Pending event from the event queue.
	"""
	e = Event("key", "", b"")

	while not self.event_queue.empty():
	e2 = self.event_queue.get()
	e.data += e2.data
	e.raw += e.raw

	amount = 10000
	raw = self.__read(amount)
	data = str(raw, self.encoding, "replace")
	e.data += data
	e.raw += raw
	return e

and dropped the reading of the next chunk, since IMHO this does not fit the doc string.

mypy fires here

Lib/_pyrepl/windows_console.py:545: error: Item "None" of "Event | None" has no
attribute "data"  [union-attr]
                e.data += e2.data
                          ^~~~~~~
Found 1 error in 1 file (checked 25 source files)

and is indeed correct. Have to dig in why we do not see this in case of unix_console.py ...

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

while not self.event_queue.empty():

e2 = self.event_queue.get()

e.data += e2.data

def wait(self, timeout: float | None) -> bool:

return e

def wait_for_event(self, timeout: float | None) -> bool:

"""Wait for an event."""

if timeout is None:

timeout = INFINITE

Expand All @@ -536,6 +559,16 @@ def wait(self, timeout: float | None) -> bool:

return False

return True

def wait(self, timeout: float | None = None) -> bool:

Comment thread

chris-eibl Apr 24, 2025

Copy link

Copy Markdown

Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Full in sync with unix_console.py now :)

cpython/Lib/_pyrepl/unix_console.py

Lines 419 to 427 in e1c09ff

	def wait(self, timeout: float \| None = None) -> bool:
	"""
	Wait for events on the console.
	"""
	return (
	not self.event_queue.empty()
	or self.more_in_buffer()
	or bool(self.pollob.poll(timeout))
	)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

"""

Wait for events on the console.

"""

return (

not self.event_queue.empty()

or self.more_in_buffer()

or self.wait_for_event(timeout)

)

def repaint(self) -> None:

raise NotImplementedError("No repaint support")

Expand Down

Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.