GeezerVison is showing github.com — 0.6.3: testing stalls at test_pack (gitdb.test.test_pack.TestPack) ... · Issue #19 · gitpython-developers/gitdb
This repository was archived by the owner on Jul 26, 2026. It is now read-only.
This repository was archived by the owner on Jul 26, 2026. It is now read-only.
0.6.3: testing stalls at test_pack (gitdb.test.test_pack.TestPack) ... #19DescriptionJust mentioned that a sweep of builds I ran a few days back got stalled while building for Debian wheezy at
test_pack (gitdb.test.test_pack.TestPack) ...
yet not sure if wheezy specific or sporadic or what... but just wanted to report if may be there are immediate ideas Activityyarikoptic commented on Jan 11, 2015 ContributorAuthor More actions Ctrl-C'ing of the next build (so reproducible) lead to following backtrace if it makes any sense
...
File "/tmp/buildd/python-gitdb-0.6.3/gitdb/test/test_pack.py", line 141, in test_pack
self._assert_pack_file(pack, version, size)
File "/tmp/buildd/python-gitdb-0.6.3/gitdb/test/test_pack.py", line 90, in _assert_pack_file
for obj in pack.stream_iter():
File "/tmp/buildd/python-gitdb-0.6.3/gitdb/pack.py", line 555, in _iter_objects
cur_offset += (data_offset - ostream.pack_offset) + ostream.stream.compressed_bytes_read()
File "/tmp/buildd/python-gitdb-0.6.3/gitdb/stream.py", line 179, in compressed_bytes_read
self.read(mmap.PAGESIZE)
File "/tmp/buildd/python-gitdb-0.6.3/gitdb/stream.py", line 211, in read
size = min(size, self._s - self._br)
KeyboardInterrupt
Byron commented on Jan 11, 2015 Thank you.
I just ran the tests on the latest version and they ran through in 25s. However, before the next release I shall run the tests on the source distribution, maybe something is different there.
Just for completeness, it would be interesting to hear which python version you have been using. kevin-brown commented on Jan 11, 2015
Ctrl-C'ing of the next build (so reproducible) lead to following backtrace if it makes any sense
A KeyboardInterrupt is raised whenever you hit CTRL + C when running a Python program. I'm not sure if that is actually related to your original issue though. yarikoptic commented on Jan 11, 2015 ContributorAuthor More actions yarikoptic commented on Jan 11, 2015 ContributorAuthor More actions keyboard interrupt track should be close to the place where it was stuck when I pressed it thus I hoped providing information on where that test was stuck yarikoptic commented on Jan 11, 2015 ContributorAuthor More actions FWIW -- none of the compatible ubuntus was affected, so solely a wheezy issue. I will try to get a better clue yarikoptic commented on Jan 11, 2015 ContributorAuthor More actions FWIW here is some information from arrogant me since I am not sure about details but it never exits that loop on wheezy:
root@head2:/home/neurodebian/deb/builds/python-gitdb/0.6.3-1/python-gitdb-0.6.3# nosetests -s -v --pdb --pdb-failures /home/neurodebian/deb/builds/python-gitdb/0.6.3-1/python-gitdb-0.6.3/gitdb/test/test_pack.py
test_pack (gitdb.test.test_pack.TestPack) ... ENTERING LOOP
> /home/neurodebian/deb/builds/python-gitdb/0.6.3-1/python-gitdb-0.6.3/gitdb/stream.py(180)compressed_bytes_read()
-> while not self._zip.unused_data and self._cbr != len(self._m):
(Pdb) print self._zip
<zlib.Decompress object at 0x1129d40>
(Pdb) print self._cbr
0
(Pdb) print len(self._m)
3646
(Pdb) n
> /home/neurodebian/deb/builds/python-gitdb/0.6.3-1/python-gitdb-0.6.3/gitdb/stream.py(181)compressed_bytes_read()
-> self.read(mmap.PAGESIZE)
(Pdb) n
> /home/neurodebian/deb/builds/python-gitdb/0.6.3-1/python-gitdb-0.6.3/gitdb/stream.py(180)compressed_bytes_read()
-> while not self._zip.unused_data and self._cbr != len(self._m):
(Pdb) print len(self._m)
3646
(Pdb) print self._cbr
0
(Pdb) n
> /home/neurodebian/deb/builds/python-gitdb/0.6.3-1/python-gitdb-0.6.3/gitdb/stream.py(181)compressed_bytes_read()
-> self.read(mmap.PAGESIZE)
(Pdb) print mmap.PAGESIZE
4096
(Pdb) l
176 else:
177 # pass in additional pages, until we have unused data
178 print "ENTERING LOOP"
179 import pdb; pdb.set_trace()
180 while not self._zip.unused_data and self._cbr != len(self._m):
181 -> self.read(mmap.PAGESIZE)
182 print "LEAVING LOOP"
183 # END scrub-loop default zlib
184 # END handle stream scrubbing
185
186 # reset bytes read, just to be sure
(print statements were added by me) yarikoptic commented on Jan 11, 2015 ContributorAuthor More actions ok -- added a print statement
print "unused %s cbr %s len %s" %(self._zip.unused_data, self._cbr, len(self._m))
at the end of the loop
and here is how it looks on my laptop
$> nosetests -s -v gitdb/test/test_pack.py
test_pack (gitdb.test.test_pack.TestPack) ... unused ½x%ÇÁ 0À�ªØ
lC�ÚBԃ, cbr 43 len 1633
Ø¢x340031 cbr 43 len 1503
unused ¢x cbr 12 len 1457
unused d>xuà´|¢x340031 cbr 43 len 1443
unused ¢x cbr 12 len 1397
unused ½IxuSÉnÛ0=_1·.¨©EÂ@ cbr 43 len 1383
unused fx;϶ cbr 21 len 720
...
and on wheezy:
test_pack (gitdb.test.test_pack.TestPack) ... ENTERING LOOP
unused cbr 0 len 3646
unused cbr 0 len 3646
unused cbr 0 len 3646
unused cbr 0 len 3646
unused cbr 0 len 3646
unused cbr 0 len 3646
unused cbr 0 len 3646
...
if that is of help -- dig into zip and self._cbr ;) Byron commented on Jan 12, 2015 Thank you, I believe you pinpointed the spot where it hangs quite precisely !
Do you have any pointers on where I can most easily get a (small) image of wheezy ? I will setup a new VM just to reproduce (and hopefully fix) the issue. Byron commented on Jan 12, 2015 I have made some intense cross-testing just to figure out what works and what doesn't work. With a changed logic, I was at least able to get it to work with most configuration (basically I just checked for zlib version 1.2.7 on wheezy, and made a special case for osx and zlib version 1.2.5).
| OS |
zlib version |
Python Version |
Works with Old Logic |
Works with Changed Logic |
| Wheezy |
1.2.7 |
2.6,2.7,3.2 |
2.6 only |
✅ |
| OSX 10.10 |
1.2.5 |
2.6,2.7.3.4 |
✅ |
✅ |
| centos 6.5 |
1.2.3 |
2.6 |
✅ |
✅ |
| xubuntu 14.04.1 LTS |
1.2.8 |
2.7,3.4 |
✅ |
✅ |
| fedora 17 |
1.2.5 |
2.7,3.2 |
❌ |
✅ |
| windows 7 |
1.2.3 |
2.7,3.4 |
✅ |
✅ |
In other words, with the changed logic, it works on all tested platforms, but I expect this to fail in the future, as the actual problem is not solved. Thus I could not tell why the changed logic at this spot is even necessary. Byron commented on Jan 12, 2015 Development videos can be found here:
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|