GeezerVison is showing github.com — crash appending list and namedtuple · Issue #53093 · python/cpython
crash appending list and namedtuple #53093Description
| BPO |
8847 |
| Nosy |
@loewis, @birkenfeld, @rhettinger, @terryjreedy, @jcea, @atsuoishimoto, @amauryfa, @ncoghlan, @pitrou, @vstinner, @jackdied, @tjguk, @merwok, @alex, @bitdancer, @briancurtin, @skrah, @meadori |
| Files |
issue8847.diffissue8847-3.2.diffissue8847-3.3.diff |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2012-08-01.09:13:02.819>
created_at = <Date 2010-05-28.22:53:45.970>
labels = ['interpreter-core', 'deferred-blocker', 'library', 'OS-windows', 'type-crash']
title = 'crash appending list and namedtuple'
updated_at = <Date 2012-08-01.13:32:32.136>
user = 'https://bugs.python.org/benrg' bugs.python.org fields:
activity = <Date 2012-08-01.13:32:32.136>
actor = 'loewis'
assignee = 'none'
closed = True
closed_date = <Date 2012-08-01.09:13:02.819>
closer = 'loewis'
components = ['Interpreter Core', 'Library (Lib)', 'Windows']
creation = <Date 2010-05-28.22:53:45.970>
creator = 'benrg'
dependencies = []
files = ['26623', '26633', '26634']
hgrepos = []
issue_num = 8847
keywords = ['patch']
message_count = 42.0
messages = ['106695', '106720', '107103', '107108', '107111', '107112', '107115', '107116', '107117', '107119', '107120', '107124', '107147', '130233', '166907', '166908', '166909', '166929', '166935', '166936', '166942', '166943', '166946', '166954', '166964', '166969', '166971', '167009', '167026', '167033', '167036', '167037', '167057', '167062', '167063', '167098', '167100', '167101', '167106', '167111', '167120', '167127']
nosy_count = 21.0
nosy_names = ['loewis', 'georg.brandl', 'rhettinger', 'terry.reedy', 'jcea', 'ishimoto', 'amaury.forgeotdarc', 'ncoghlan', 'pitrou', 'vstinner', 'jackdied', 'tim.golden', 'eric.araujo', 'mrabarnett', 'alex', 'r.david.murray', 'brian.curtin', 'skrah', 'meador.inge', 'benrg', 'python-dev']
pr_nums = []
priority = 'deferred blocker'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue8847'
versions = ['Python 3.2', 'Python 3.3'] Activitybenrg commented on May 28, 2010 MannequinAuthor More actions c:\>python
Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from collections import namedtuple
>>> foo = namedtuple('foo', '')
>>> [1] + foo()At this point the interpreter crashes. Also happens when foo has named arguments, and in batch scripts. foo() + [1] throws a TypeError as expected. [] + foo() returns (). The immediate cause of the crash is the CALL instruction at 1E031D5A in python31.dll jumping into uninitialized memory. bitdancer commented on May 29, 2010 I can't reproduce this on either 3.1.2 or py3k trunk. terryjreedy commented on Jun 4, 2010 Running the exact same binary on winxp with an amd athlon processor,
I *did* get a crash after about 5 seconds. "python.exe has encountered a problem and needs to close. We are sorry for the inconvenience."
Trying again with IDLE instead of the command window, I get the same message (for pythonw instead of python) and the shell restarts when I close the message.
Even though foo()+[1] correctly raises a TypeError, the reverse [1] + foo() very bizarrely produces a length 1 tuple whose bizarre member is supposedly an instance of a empty list [] with length 1
from collections import namedtuple
foo = namedtuple('foo', '')
a = [1] + foo()
b=a[0]
print (type(a), len(a), type(b), len(type(b)), type(type(b)))# <class 'tuple'> 1 [] 1 <class 'list'>
([2]+foo() produces same output)
Other than the above, any attempt to do anything with b or type(b) that I tried crashes. I presume that this is due to attribute lookups on the invalid type(b) (or something like that), which type(b) must bypass. In particular, the OP's crash is due to trying to print the tuple which tries to print its member which looks for type(b).__str__ which crashes.
To anyone: investigating crashers like this is a lot easier with IDLE and an edit window than with the interactive command window. terryjreedy commented on Jun 4, 2010 More experiments
from collections import namedtuple
foo = namedtuple('foo', '')
a = [] + foo()
print (a, type(a), len(a))
# () <class 'tuple'> 0ie, a standard empty tuple, whereas
a = [1,1] + foo()
crashes immediately. So the behavior of list()+namedtuple depends on the length of the list.
There are also some funny interactions. Adding
try:
a = foo()+[]
except TypeError:
print("correct TypeError")
after the 'foo = ' line in my original 5 line example causes the final print to crash, whereas adding the same 4 lines to the 4 line example at the beginning of this message does not obviously change anything.
David, since you omitted all details, I wonder if you tested in batch mode, as I did, and hence never tried to print the malformed object, or used different OS or hardware. jackdied commented on Jun 4, 2010 I can't reproduce on 3k trunk with Ubuntu 10.04, gcc 4.4.3
namedtuples are just a subclass of tuple with only two dunder methods defined (a plain __new__ with empty __slots__). Can you provoke the same behavior with plain tuples, or a subclass of tuple that looks like one of these?
class Crasher(tuple): pass
class Crasher(tuple):
__slots__ = ()
class Crasher(tuple):
def __new__(cls,): return tuple.__new__(cls,)
__slots__ = () terryjreedy commented on Jun 4, 2010 Substituting
foo = tuple()
TypeError: can only concatenate list (not "tuple") to list
class Crasher(tuple): pass
foo = Crasher()
a = [1] + foo
b=a[0]
print (type(a), len(a), type(b), len(type(b)), type(type(b))) <class 'tuple'> 1 [] 1 <class 'list'>
as before, so namedtuple is not, in particular, the culprit.
Other two Crasher versions do the same. I also get a delayed pythonw error message after the print that does not cause the shell to restart. This may partly be an IDLE artifact. jackdied commented on Jun 5, 2010 Two more probes:
-
does it also have the same strange/crashy behavior when you subclass list and concat that to a tuple?
-
does dropping the optimization level down to -O help? This has "compiler quirk" written all over it. The C-code for list and tuple concat are almost identical, and both start with a type check.
terryjreedy commented on Jun 5, 2010 "can't reproduce" does not inform as to what *did* happen with which code.
More experiments:
foo = str()
TypeError: can only concatenate list (not "str") to list
class s(str): pass
foo = s()
TypeError: Can't convert 'list' object to str implicitly Why is it trying to do that? Of course, the interpreter can (implicitly) convert list to tuple, which must be what happens in OP example.
The subclasses of tuple and str do not gain an __radd__ method. If we add one
class s(str):
def __radd__(s,o): print('in radd, type(o)')
foo = s()
a = [1] + foo
# prints "in radd <class 'list'>"no implicit conversion is done.
Reversing tuple and list
class Crasher(list): pass
a = () + Crasher() # or Crasher([1])
print(a, type(a), len(a))
#[] <class 'list'> 0 # or [1] <class 'list'> 1 whereas
a = (1,) + Crasher()
crashes, so not completely symmetrical terryjreedy commented on Jun 5, 2010
- answered before you asked (yes, similar)
- (same thought) I am using PSF windows installer, however that was prepared. Martin?
jackdied commented on Jun 5, 2010 if the id() of the left operand is identical to the id() of the first element in the result it would strongly support compiler skulldugerry.
class Crasher(tuple): pass
foo = Crasher()
x = [1]
a = x + foo
b=a[0]
if id(b) == id(x):
raise Exception("It's the C compiler what did it!")The only way I can think of this coming about is the right_op getting new'd and then .extend'ing(left_op). That extend() must be going batsh*t and inserting the left_op instead of it's contained items. The C-code for extend is more fiddly than the code for concatenation so there is more room for the compiler to generate bad code. terryjreedy commented on Jun 5, 2010 Good try, but for one run, the ids of foo, x, a, and b are
>>>
15719440 15717880 15273104 12266976 pitrou commented on Jul 31, 2012
Antoine Pitrou <[email protected]> wrote:
> I think we want to add those tests to the test suite as well.
What's a good place? Shall we just add one of the tests to test_tuple?
Sounds good. And another of them to test_list perhaps as well :)
Also, the only person to run the tests with the PGO build will probably
be Martin just before the releases. :)
True, but other people may still run them *after* the release. skrah commented on Jul 31, 2012 New patches with tests for 3.2 and 3.3. For 3.2 I determined empirically
that EnableCOMDATFolding="1" (and not "0") turns on NOICF.
If anyone can confirm that this is the case or has a pointer to
the relevant vcproj docs, I'd be thrilled. loewis commented on Jul 31, 2012
Also, the only person to run the tests with the PGO build will probably
be Martin just before the releases. :)
We could set up a buildbot slave which does PGO builds, provided somebody
volunteered an installation (including VS Pro), and somebody contributed
a build script that deviates from the regular build (perhaps including some
training on the PGI). This is a separate issue, of course. loewis commented on Jul 31, 2012 python-dev commented on Aug 1, 2012 python-dev commented on Aug 1, 2012 loewis commented on Aug 1, 2012 Thanks for the research and the fix! vstinner commented on Aug 1, 2012 You didn't add any test for non regression?? loewis commented on Aug 1, 2012
You didn't add any test for non regression??
Please rephrase your question: what tests did I not add?
I did add the tests that Stefan proposed. vstinner commented on Aug 1, 2012
Please rephrase your question: what tests did I not add?
I did add the tests that Stefan proposed.
Ah yes, you added new tests to Python 3.2 and 3.3, but no to Python
2.7. Why not adding these new tests to Python 2.7? loewis commented on Aug 1, 2012
Ah yes, you added new tests to Python 3.2 and 3.3, but no to Python
2.7. Why not adding these new tests to Python 2.7?
The tests don't crash Python 2.7. So they are not useful as a test whether
the bug has been worked-around. I actually don't know how to test this
compiler
bug in Python 2.7 (except for writing specific C code that tries to trigger
the bug).
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|