GeezerVison is showing github.com — IDLE hangs while printing instance of Unicode subclass · Issue #63680 · python/cpython
IDLE hangs while printing instance of Unicode subclass #63680Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2013-12-10.08:34:56.373>
created_at = <Date 2013-11-03.04:29:15.991>
labels = ['expert-IDLE', 'type-bug']
title = 'IDLE hangs while printing instance of Unicode subclass'
updated_at = <Date 2015-03-04.15:00:12.514>
user = 'https://github.com/tim-one' bugs.python.org fields:
activity = <Date 2015-03-04.15:00:12.514>
actor = 'mjpieters'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2013-12-10.08:34:56.373>
closer = 'serhiy.storchaka'
components = ['IDLE']
creation = <Date 2013-11-03.04:29:15.991>
creator = 'tim.peters'
dependencies = []
files = ['32472', '32490', '32491']
hgrepos = []
issue_num = 19481
keywords = ['patch']
message_count = 17.0
messages = ['201991', '202003', '202004', '202005', '202070', '202072', '202093', '202096', '202098', '202102', '205732', '205740', '205741', '205775', '205776', '237180', '237182']
nosy_count = 9.0
nosy_names = ['tim.peters', 'terry.reedy', 'kbk', 'mjpieters', 'ned.deily', 'ezio.melotti', 'roger.serwy', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue19481'
versions = ['Python 2.7', 'Python 3.3', 'Python 3.4'] Activitytim-one commented on Nov 3, 2013 terryjreedy commented on Nov 3, 2013 Win 7, console 2.7.5+, 32 bit, compiled Aug 24, does not have the problem. Idle started with 'import idlelib.idle' does, but only for 'print foo', as Tim reported. When I close the hung process with [X], there is no error message in the console. Installed 64bit 2.7.5 fails with 'print foo' also. I actually used F and f instead of Foo and foo, so it is not name specific. A subclass of str works fine.
Current 3.4a4 Idle works fine. The SO OP also reported that there is no problem is the class is imported from another file.
We need a test on something other than Windows, preferably both mac and linux. ned-deily commented on Nov 3, 2013 It's reproducible on OS X as well with a 32-bit Python 2.7.5 and a 64-bit Python 2.7.6rc1. However, the example works OK if I start IDLE with no subprocess (-n). serhiy-storchaka commented on Nov 3, 2013 This patch fixes symptoms. ned-deily commented on Nov 4, 2013 tim-one commented on Nov 4, 2013 Do we have a theory for _why_ IDLE goes nuts? I'd like to know whether the patch is fixing the real problem, or just happens to work in this particular test case ;-) terryjreedy commented on Nov 4, 2013 I am curious too, so I traced through the call chain.
In PyShell.py
1343: PseudoOutputFile.write(s) calls: self.shell.write(s, self.tags)
914: shell is an instance of PyShell and self.tags is 'stdout', 'stderr', or 'console'.
1291: PyShell.write(s,tags) calls:
OutputWindow.write(self, s, tags, "iomark")
(where 'iomark' must have been defined elsewhere, and the 'gravity' calls should not matter)
In OutputWindow.py
46: OutputWindow(EditorWindow).write(s,tags,mark='insert') calls: self.text.insert(mark, s, tags)
after trying to encode s if isinstance(s, str). It follows with:
self.text.see(mark)
self.text.update()
but if the insert succeeds, these should not care about the source of the inserted chars.
In EditorWindow.py
187: self.text = MultiCallCreator(Text)(text_frame, **text_options)
In MultiCall.py,
304: MultiCallCreator wraps a tk widget in a MultiCall instance that adds event methods but otherwise passes calls to the tk widget.
So PseudoOutputFile(s) becomes tk.Text().insert('iomark', s, 'stdout').
which becomes (lib-tk/tkinter.py, 3050)
self.tk.call((self._w, 'insert', 'iomark', s) + args)
Tk handles either Latin-1 bytes or BMP unicode. It seems fine with a unicode subclass:
>>> import Tkinter as tk
>>> t = tk.Text()
>>> class F(unicode): pass
>>> f = F('foo')
>>> t.insert('1.0', u'abc', 'stdout') # 'iomark' is not defined
>>> t.insert('1.0', f, 'stdout')
>>> t.get('1.0', 'end')
u'abcfoo\n'I remain puzzled. serhiy-storchaka commented on Nov 4, 2013 I suppose this is related to pickling.
I were puzzled why it works with bytearray subclasses. But now I investigated that print() implicitly converts str and bytearray subclasses to str and left unicode subclasses as is. You can reproduce this bug for str and bytearray subclasses if use sys.stdout.write() instead of print().
Here is a patch for 2.7 which fixes the issue for str and bytearray subclasses too. 3.x needs patch too.
>> class U(unicode): pass
>> class S(str): pass
>> class BA(bytearray): pass
>>> import sys
>>> sys.stdout.write(u'\u20ac')
€
>>> sys.stdout.write('\xe2\x82\xac')
€
>>> sys.stdout.write(bytearray('\xe2\x82\xac'))
€
>>> sys.stdout.write(U(u'\u20ac'))
€
>>> sys.stdout.write(S('\xe2\x82\xac'))
€
>>> sys.stdout.write(BA('\xe2\x82\xac'))
€serhiy-storchaka commented on Nov 4, 2013 And here is a patch for 3.x. Without it following code hangs.
>> class S(str): pass
>>> import sys
>>> sys.stdout.write('\u20ac')
€1
>>> sys.stdout.write(S('\u20ac'))
€1ned-deily commented on Nov 4, 2013 Pickling for the RPC protocol between the GUI process and the interpreter subprocess, which would explain why there is no problem when running idle -n (no subproces)? serhiy-storchaka commented on Dec 9, 2013
Pickling for the RPC protocol between the GUI process and the interpreter subprocess, which would explain why there is no problem when running idle -n (no subproces)?
Yes, it is.
If there are no objections I'll commit these patches. terryjreedy commented on Dec 9, 2013
[2.7] print() implicitly converts str and bytearray subclasses to str and left unicode subclasses as is.
This strikes me as possibly a bug in print, but even if that were changed, there is still the issue of sys.stdout.write and pickle. While the patch is a great improvement, it changes the behavior of sys.stdout.write(s), which acts like it calls str.__str__(s) rather than str(s) == s.__str__
---
class S(str):
def __str__(self):
return 'S: ' + str.__str__(self)
s = S('foo')
print(s, str(s), str.__str__(s))
import sys
sys.stdout.write(s)
S: foo S: foo foo
foo
on the console (hang after first line on Idle)
I am testing the patch with str(s) changed to str.__str__(s). terryjreedy commented on Dec 9, 2013 Confirmed that the revised patch for 3.3 fixes the hang and matches the console interpreter output. serhiy-storchaka commented on Dec 10, 2013 Good suggestion Terry. And for unicode in 2.7 we can use unicode.__getslice__(s, None, None) (because there is no unicode.__unicode__). python-dev commented on Dec 10, 2013 mjpieters commented on Mar 4, 2015 This changes causes printing BeautifulSoup NavigableString objects to fail; the code actually could never work as unicode.__getslice__ insists on getting passed in integers, not None.
To reproduce, create a new file in IDLE and paste in:
from bs4 import BeautifulSoup
html_doc = """<title>The Dormouse's story</title>"""
soup = BeautifulSoup(html_doc)
print soup.title.string Then pick *Run Module* to see:
Traceback (most recent call last):
File "/private/tmp/test.py", line 4, in <module>
print soup.title.string
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/idlelib/PyShell.py", line 1353, in write
s = unicode.__getslice__(s, None, None)
TypeError: an integer is requiredThe same error can be induced with:
unicode.__getslice__(u'', None, None) while specifying a start and end index (0 and len(s)) should fix this. mjpieters commented on Mar 4, 2015
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|