GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — bpo-44439: _ZipWriteFile.write() handle buffer protocol correctly · Pull Request #29468 · python/cpython Conversation
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| if isinstance(data, (bytes, bytearray)): | ||
| nbytes = len(data) | ||
| else: | ||
| data = memoryview(data) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I think it's recommended to manually release a memoryview. Implementations other than CPython may use a garbage collector that doesn't immediately finalize unreferenced objects. Maybe it's simpler to always use a memoryview:
# Accept any data that supports the buffer protocol with memoryview(data) as data: nbytes = data.nbytes self._file_size += nbytes self._crc = crc32(data, self._crc) if self._compressor: data = self._compressor.compress(data) self._compress_size += len(data) self._fileobj.write(data) return nbytes
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Good point.
There are some sites in stdlib have this problem, maybe they can be solved in another issue together.
In addition, I suspect if other Python implementations don't release the underlying buffer of memoryview, the program will run abnormally.
Let @serhiy-storchaka decide.
update: I ran a benchmark, the performances are no significant different.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
@eryksun is right. But it may be simpler to merge the current solution and fix issues with releasing memoryviews in all places uniformly later.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Fix this in another issue.
|
This PR is stale because it has been open for 30 days with no activity. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Ran 258 tests in 27.542s
OK (skipped=1)
== Tests result: SUCCESS ==
Look ok to me.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| with zipfile.ZipFile(io.BytesIO(), 'w') as zip: | ||
| with zip.open('data', 'w') as data: | ||
| self.assertEqual(data.write(q), LENGTH) | ||
| self.assertEqual(data._file_size, LENGTH) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
_file_size is a private attribute. It is better to use a public API in tests. For example read the content of the file.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| # quickly. | ||
| self.assertRaises(OSError, zipfile.ZipFile, TESTFN) | ||
|
|
||
| def test_issue44439(self): |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
It would be better to move this test to AbstractWriterTests and test with different compressions.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| if isinstance(data, (bytes, bytearray)): | ||
| nbytes = len(data) | ||
| else: | ||
| data = memoryview(data) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
@eryksun is right. But it may be simpler to merge the current solution and fix issues with releasing memoryviews in all places uniformly later.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| @@ -0,0 +1,2 @@ | |||
| Fix in `_ZipWriteFile.write()` method, when the input data is an object that | |||
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
_ZipWriteFile is a private class. It would be better to reword the NEWS entry in terms of the public API.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Thanks. LGTM.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Thanks for your review. Close and reopen to trigger tests. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@serhiy-storchaka |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Thanks @animalize for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Thanks @animalize for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
GH-31755 is a backport of this pull request to the 3.9 branch. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…thonGH-29468) Co-authored-by: Marco Ribeiro <[email protected]> (cherry picked from commit 36dd739) Co-authored-by: Ma Lin <[email protected]>
|
GH-31756 is a backport of this pull request to the 3.10 branch. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…thonGH-29468) Co-authored-by: Marco Ribeiro <[email protected]> (cherry picked from commit 36dd739) Co-authored-by: Ma Lin <[email protected]>
…-29468) Co-authored-by: Marco Ribeiro <[email protected]> (cherry picked from commit 36dd739) Co-authored-by: Ma Lin <[email protected]>
…-29468) Co-authored-by: Marco Ribeiro <[email protected]> (cherry picked from commit 36dd739) Co-authored-by: Ma Lin <[email protected]>
…thonGH-29468) Co-authored-by: Marco Ribeiro <[email protected]> (cherry picked from commit 36dd739) Co-authored-by: Ma Lin <[email protected]>
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.