GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Fix Stream.readuntil with non-bytes buffer objects by bmerry · Pull Request #117653 · python/cpython

Conversation

@bmerry

@bmerry bmerry commented Apr 8, 2024

Copy link
Copy Markdown
Contributor

PR #16429 introduced support for an iterable of separators in Stream.readuntil. Since bytes-like types are themselves iterable, this can introduce ambiguities in deciding whether the argument is an iterator of separators or a singleton separator. In #16429, only 'bytes' was considered a singleton, but this will break code that passes other buffer object types.

The Python library docs don't indicate what separator types were permitted in Python <=3.12, but comments in typeshed indicate that it would work with types that implement the buffer protocol and provide a len(). To keep those cases working the way they did before, I've changed the detection logic to consider any instance of collections.abc.Buffer as a singleton separator.

There may still be corner cases where this doesn't do what the user wants e.g. a numpy array of byte strings will implement the buffer protocol and hence be treated as a singleton; but at least those corner cases should behave the same in 3.13 as they did in 3.12.

Relates to #81322.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bmerry
Fix Stream.readuntil with non-bytes buffer objects
c364ee8
PR python#16429 introduced support for an iterable of separators in
Stream.readuntil. Since bytes-like types are themselves iterable, this
can introduce ambiguities in deciding whether the argument is an
iterator of separators or a singleton separator. In python#16429, only 'bytes'
was considered a singleton, but this will break code that passes other
buffer object types.

The Python library docs don't indicate what separator types were
permitted in Python <=3.12, but comments in typeshed indicate that it
would work with types that implement the buffer protocol and provide a
len(). To keep those cases working the way they did before, I've changed
the detection logic to consider any instance of collections.abc.Buffer
as a singleton separator.

There may still be corner cases where this doesn't do what the user
wants e.g. a numpy array of byte strings will implement the buffer
protocol and hence be treated as a singleton; but at least those corner
cases should behave the same in 3.13 as they did in 3.12.

Relates to python#81322.
@bmerry
bmerry requested review from 1st1, asvetlov, gvanrossum, kumaraditya303 and willingc as code owners April 8, 2024 19:08
@bedevere-app bedevere-app Bot added the awaiting review label Apr 8, 2024
@bmerry bmerry mentioned this pull request Apr 8, 2024
Allow multiple separators in Stream.readuntil #81322
Closed
@gvanrossum

gvanrossum commented Apr 8, 2024

Copy link
Copy Markdown
Member

Let's hold off until we've decided that we really need a general iterable, rather than just a tuple (see my comment on the issue).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bmerry

bmerry commented Apr 8, 2024

Copy link
Copy Markdown
Contributor Author

Let's hold off until we've decided that we really need a general iterable, rather than just a tuple (see my comment on the issue).

Agreed. I'll close this and raise a new PR to only support tuples.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bmerry bmerry closed this Apr 8, 2024
@gvanrossum

gvanrossum commented Apr 8, 2024

Copy link
Copy Markdown
Member

Also, I'm sorry -- when I first saw the PR I immediately had the thought "why allow arbitrary iterators" but suppressed it because there was no reason in the code itself to disallow it.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@1st1 1st1 Awaiting requested review from 1st1 1st1 is a code owner
@asvetlov asvetlov Awaiting requested review from asvetlov asvetlov is a code owner
@gvanrossum gvanrossum Awaiting requested review from gvanrossum
@kumaraditya303 kumaraditya303 Awaiting requested review from kumaraditya303 kumaraditya303 is a code owner
@willingc willingc Awaiting requested review from willingc willingc is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@bmerry @gvanrossum
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.