GeezerVison

CI all green! :)

Hum, I see multiple discussions about BytesIO.peek(0) limit. Should it return the whole buffer? Or limit to io.DEFAULT_BUFFER_SIZE (128 kiB currently)?

io.BufferedReader.peek(0) is limited to io.BufferedReader.buffer_size which is io.DEFAULT_BUFFER_SIZE by default.

If the BytesIO size if 1 GiB, returning 1 GiB in peek(0) may not be the expected behavior. Calling peek(0) after read(1) has to copy 1 GiB (minus 1 byte) which can be slow.

IMO limiting BytesIO.peek(0) to io.DEFAULT_BUFFER_SIZE would avoid bad surprises. If the caller wants more, they can pass a size of peek(size).

I have updated the behavior such that at most DEFAULT_BUFFER_SIZE bytes are returned if size is less than one or omitted.

Maybe for size=-1, the behavior could be that the full remainder of the buffer is returned, but I don’t mind either way.

Maybe for size=-1, the behavior could be that the full remainder of the buffer is returned, but I don’t mind either way.

Ideally this would use PEP-661 so we could distinguish the cases "not provided" and "all" easily but the .peek() API predates that. I do think it may make sense to modify behavior across all them with new sentinels but that raises some breaking change possibility. I really want a reliable .peek(10) returning 10 bytes on regular blocking files... would make a lot of code simpler.

@marcelm: Can you please update your PR and solve the merge conflict on Doc/whatsnew/3.16.rst?

Updated (The lzma whatsnew entry moved to be alphabetical / after logging)

Congrats @marcelm! Finally, I merged your PR! Sorry for all the back and forth, but it was really hard to decide on the exact API, especially for peek() and peek(0). I like the final API :-)

Oh, nice, happy to see this go in just before going on vacation! 😄

I think what was a bit difficult was that it was hard to tell who gets to decide, that is, whose requests I should implement if there was a disagreement (between core devs). This was also the reason for the long delay between Nov. 2023 and then April this year.

I’m glad it worked out now, even if it took ~180 comments and 54 commits for a ~200 line change. Thanks everyone for your help, I learned a lot along the way!

~180 comments and 54 commits for a ~200 line change

That's a very long history. Thanks for your tenacity :-D

Thanks for all the work @marcelm!

Noted on the decision and debate parts. Really appreciate working through it and think we ended up with a really nice new feature :)