GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-159999: Fix needs_input for bz2 and zlib decompressors by iamsaugatpandey · Pull Request #160696 · python/cpython

Conversation

@iamsaugatpandey

@iamsaugatpandey iamsaugatpandey commented Oct 26, 2026 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

BZ2Decompressor and zlib._ZlibDecompressor had a bug where, if the input ran out right as the output buffer filled up, needs_input was set to true even though there was still decompressed data waiting. Callers would go looking for more input when they should have been draining the buffer.

While in there, I found a related problem in bz2’s decompress_buf(). It checked for exhausted input before checking for a full output buffer, so even with max_length=-1 it could hand back just the first 32 KiB of a block’s output. I reordered the checks.

Both fixes follow what was done for lzma in bpo-21872 (GH-14048).

One knock-on effect: gzip._GzipReader was relying on the old behavior. It raised EOFError whenever decompress(b"") returned nothing, but that can now legitimately happen on a valid stream. It now only raises when reading the file gives back no data, which is what DecompressReader already does.

Compatibility note: if anyone copied gzip’s old loop for use with BZ2Decompressor, they could see the same spurious EOFError. lzma has worked this way since 3.9, so this just makes bz2 and zlib consistent with it.

  • Issue: BZ2Decompressor and zlib._ZlibDecompressor set needs_input=True while output is still buffered #159999

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@iamsaugatpandey
pythongh-159999: Fix needs_input for bz2 and zlib decompressors
9a6c9d1
BZ2Decompressor and zlib._ZlibDecompressor had a bug where, if the input
ran out right as the output buffer filled up, needs_input was set to
true even though there was still decompressed data waiting. Callers
would go looking for more input when they should have been draining the
buffer.

While in there, I found a related problem in bz2’s decompress_buf(). It
checked for exhausted input before checking for a full output buffer,
so even with max_length=-1 it could hand back just the first 32 KiB of
a block’s output. I reordered the checks.

Both fixes follow what was done for lzma in bpo-21872 (pythonGH-14048).

One knock-on effect: gzip._GzipReader was relying on the old behavior.
It raised EOFError whenever decompress(b"") returned nothing, but that
can now legitimately happen on a valid stream. It now only raises when
reading the file gives back no data, which is what DecompressReader
already does.

Compatibility note: if anyone copied gzip’s old loop for use with
BZ2Decompressor, they could see the same spurious EOFError. lzma has
worked this way since 3.9, so this just makes bz2 and zlib consistent
with it.
@iamsaugatpandey
iamsaugatpandey requested a review from StanFromIreland as a code owner October 26, 2026 07:14
Copilot AI balanced review requested due to automatic review settings October 26, 2026 07:14
@python-cla-bot

python-cla-bot Bot commented Oct 26, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot added the awaiting review label Oct 26, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Oct 26, 2026
BZ2Decompressor and zlib._ZlibDecompressor set needs_input=True while output is still buffered #159999
Open
Copilot AI reviewed Oct 26, 2026

Copilot AI left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@iamsaugatpandey

iamsaugatpandey commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

Gentle ping: this is ready for review whenever someone has time. All checks pass and the CLA is signed.

A quick summary for reviewers:

  • bz2.BZ2Decompressor and zlib._ZlibDecompressor set needs_input=True while decompressed output was still buffered (the case in BZ2Decompressor and zlib._ZlibDecompressor set needs_input=True while output is still buffered #159999).
  • bz2 also stopped early with no max_length, returning only the first 32 KiB of a block's output when the input ran out at that point.
  • Both fixes follow the lzma fix in bpo-21872 (bpo-21872: fix lzma library decompresses data incompletely #14048). gzip._GzipReader now raises EOFError only when reading the file returns no data, as DecompressReader already does. Without that, the zlib fix would cause spurious EOFErrors on valid data.

Minimal reproducer (prints True before the fix, False after):

import bz2
d = bz2.BZ2Decompressor()
d.decompress(bz2.compress(b"\x00\x10 0@" * 2)[:40], max_length=0)
print(d.needs_input)   # but d.decompress(b"") still returns data

I'm happy to make any changes.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@StanFromIreland StanFromIreland Awaiting requested review from StanFromIreland StanFromIreland is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@iamsaugatpandey
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.