GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-60107: Remove a copy from RawIOBase.read by cmaloney · Pull Request #141532 · python/cpython

Conversation

@cmaloney

@cmaloney cmaloney commented Nov 13, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

If the underlying I/O class keeps a reference to the memory raise BufferError.

Uses gh-139871 to implement.

--
If want to maintain closer compatibility when a BufferError occurs can fall back to a copy (copying is as safe as the original code). I have a slight preference to erroring as I think keeping a reference to the memory is uncommon (and probably unintended).

  • Issue: Remove copy from bytearray to bytes in rawiobase_read() implementation #60107

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@cmaloney
pythongh-60107: Remove a copy from RawIOBase.read
f904fce 
If the underlying I/O class keeps a reference to the memory raise BufferError.
@bedevere-app bedevere-app Bot added the awaiting review label Nov 13, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Nov 13, 2025
Remove copy from bytearray to bytes in rawiobase_read() implementation #60107
Closed
@cmaloney
Add missing res = NULL
6c9d14e
cmaloney
cmaloney commented Nov 13, 2025
View reviewed changes
Comment thread Lib/test/test_io/test_general.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
cmaloney
cmaloney commented Nov 13, 2025
View reviewed changes
Comment thread Modules/_io/iobase.c Outdated
}
return res;
}
res = PyObject_CallMethod(b, "take_bytes", NULL);

@cmaloney cmaloney Nov 13, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vstinner : Not sure how common this "resize/discard then take_bytes" is going to be; might make sense to change to take_bytes(n=None, /, *, discard=False)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@cmaloney cmaloney Nov 13, 2025

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for now planning to keep that in back pocket until need many ways (ba.resize(n) or del ba[:n] gives the same capability)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
cmaloney and others added 7 commits November 13, 2025 14:30
@cmaloney
Better place for test. Remove excess newlines. Remove old TODO
847c7ed
@cmaloney
Fix indentation
bfb0c71
@cmaloney
Simplify with Py_CLEAR
3709cf9
@cmaloney
Simplify with gotos
63ceed7
@cmaloney
Fix ubsan
a1daaae
@cmaloney
Just use the C API... Sorry for the onoise...
13abb41
@cmaloney
Merge branch 'main' into gh60107_take_bytes
7dc7e92
vstinner
vstinner reviewed Nov 15, 2025
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also update RawIOBase.read() in the _pyio module?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@cmaloney

cmaloney commented Nov 15, 2025

Copy link
Copy Markdown
Contributor Author

Have that in #141539 which does all the cases .take_bytes can be used in _pyio :).
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
vstinner
vstinner reviewed Nov 18, 2025
View reviewed changes
Comment thread Modules/_io/iobase.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
cmaloney and others added 2 commits November 18, 2025 14:35
@cmaloney @vstinner
Update Modules/_io/iobase.c
82643f8 
Co-authored-by: Victor Stinner <[email protected]>
@cmaloney
Regen globals and simplify more
a9a1f14
vstinner
vstinner approved these changes Nov 20, 2025
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
cmaloney reacted with hooray emoji
All reactions
  • 🎉 1 reaction
Comment thread Misc/NEWS.d/next/Library/2025-11-13-13-11-02.gh-issue-60107.LZq3QF.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Nov 20, 2025
@vstinner
Update Misc/NEWS.d/next/Library/2025-11-13-13-11-02.gh-issue-60107.LZ…
8ca1cef 
…q3QF.rst
@vstinner
vstinner enabled auto-merge (squash) November 20, 2025 17:06
@vstinner
vstinner merged commit e90061f into python:main Nov 20, 2025
46 checks passed

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the awaiting merge label Nov 20, 2025
@cmaloney
cmaloney deleted the gh60107_take_bytes branch November 20, 2025 17:37
@cmaloney cmaloney mentioned this pull request Nov 20, 2025
gh-60107: avoid io.RawIOBase.read from reading into a temporary bytearray #138616
Closed
StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
@cmaloney @vstinner @StanFromIreland
pythongh-60107: Remove a copy from RawIOBase.read (python#141532)
3a9ae83 
If the underlying I/O class keeps a reference to the memory, raise BufferError.

Co-authored-by: Victor Stinner <[email protected]>
ashm-dev pushed a commit to ashm-dev/cpython that referenced this pull request Dec 8, 2025
@cmaloney @vstinner @ashm-dev
pythongh-60107: Remove a copy from RawIOBase.read (python#141532)
c37a25e 
If the underlying I/O class keeps a reference to the memory, raise BufferError.

Co-authored-by: Victor Stinner <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@cmaloney @vstinner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.