GeezerVison is showing github.com — Include column offsets for bytecode instructions · Issue #88116 · python/cpython
Include column offsets for bytecode instructions #88116Description
| BPO |
43950 |
| Nosy |
@terryjreedy, @nedbat, @aroberge, @markshannon, @serhiy-storchaka, @ammaraskar, @pablogsal, @miss-islington, @brandtbucher, @isidentical, @sweeneyde, @sobolevn |
| PRs |
bpo-43950: Initial base implementation for PEP 657 #26955bpo-43950: Print columns in tracebacks (PEP 657) #26958bpo-43950: optimize column table assembling with pre-sizing the column table #26997bpo-43950: use 0-indexed column offsets for bytecode positions #27011bpo-43950: include position in dis.Instruction #27015bpo-43950: Add option to opt-out of PEP-657 #27023bpo-43950: Add option to opt-out of PEP-657 #27023bpo-43950: Add option to opt-out of PEP-657 #27023bpo-43950: Specialize tracebacks for subscripts/binary ops #27037bpo-43950: Add documentation for PEP-657 #27047bpo-43950: implement on-the-fly source tracking for interactive mode #27117bpo-43950: make BinOp specializations more reliable #27126bpo-43950: distinguish errors happening on character offset decoding #27217bpo-43950: ensure source_line is present when specializing the traceback #27313bpo-43950: support long lines in traceback.py #27336bpo-43950: check against the raw string, not the pyobject #27337bpo-43950: support some multi-line expressions for PEP 657 #27339bpo-43950: support positions for dis.Instructions created through dis.Bytecode #28142bpo-43950: handle wide unicode characters in tracebacks #28150 |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2021-04-27.10:58:38.225>
labels = []
title = 'Include column offsets for bytecode instructions'
updated_at = <Date 2022-01-18.12:31:06.245>
user = 'https://github.com/pablogsal' bugs.python.org fields:
activity = <Date 2022-01-18.12:31:06.245>
actor = 'sobolevn'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2021-04-27.10:58:38.225>
creator = 'pablogsal'
dependencies = []
files = []
hgrepos = []
issue_num = 43950
keywords = ['patch']
message_count = 39.0
messages = ['392054', '392062', '392081', '392082', '392520', '392540', '392541', '392555', '392557', '396866', '396874', '396950', '396957', '396962', '397109', '397352', '397368', '397589', '397666', '397667', '397669', '397673', '397829', '397837', '397844', '397882', '397909', '397910', '397911', '397914', '397915', '398089', '398144', '398169', '398170', '398172', '398197', '400998', '410856']
nosy_count = 12.0
nosy_names = ['terry.reedy', 'nedbat', 'aroberge', 'Mark.Shannon', 'serhiy.storchaka', 'ammar2', 'pablogsal', 'miss-islington', 'brandtbucher', 'BTaskaya', 'Dennis Sweeney', 'sobolevn']
pr_nums = ['26955', '26958', '26997', '27011', '27015', '27023', '27023', '27023', '27037', '27047', '27117', '27126', '27217', '27313', '27336', '27337', '27339', '28142', '28150']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue43950'
versions = [] Activitypablogsal commented on Apr 27, 2021 If we could include column offsets from the AST nodes for every bytecode instructions we could leverage these to offer much better tracebacks and a lot more information to debuggers and similar tools. For instance, in an expression such as:
z['aaaaaa']['bbbbbb']['cccccc']['dddddd].sddfsdf.sdfsdf
where one of these elements is None, we could tell exactly what of these pieces is the actual None and we could do some cool highlighting on tracebacks.
Similarly, coverage tools and debuggers could also make this distinction and therefore could offer reports with more granularity.
The cost is not 0: it would be two integers per bytecode instruction, but I think it may be worth the effort. pablogsal commented on Apr 27, 2021 I'm going to prepare a PEP since the discussion regarding if the two integers per bytecode are worth enough is going to be eternal. markshannon commented on Apr 27, 2021 The additional cost will not only be the line number table, but we need to store the line for exceptions that are reraised after cleanup.
Adding a column will mean more stack consumption. pablogsal commented on Apr 27, 2021 Yup, but I still think is worth the cost, giving that debugging improvements are usually extremely popular among users. terryjreedy commented on Apr 30, 2021 Specific examples of current messages and proposed improvements would help focus discussion.
If you are willing to only handle code lines up to 256 chars, only 2 bytes should be needed. (0,0) or (255,255) could mean 'somewhere beyond the 256th char'. pablogsal commented on Apr 30, 2021
Specific examples of current messages and proposed improvements would help focus discussion.
Yeah, I am proposing going from:
>>> x['aaaaaa']['bbbbbb']['cccccc']['dddddd'].sddfsdf.sdfsdf
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not subscriptable to
>>> x['aaaaaa']['bbbbbb']['cccccc']['dddddd'].sddfsdf.sdfsdf
^^^^^^^^^^
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not subscriptablepablogsal commented on Apr 30, 2021 Basically, to highlight in all exceptions the range in the displayed line where the error ocurred. For instance:
>>> foo(a, b/z+2, c, 132432 /x, d /y)
^^^^^^^^^
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ZeroDivisionError: division by zeroterryjreedy commented on May 1, 2021 Marking what expression evaluated to None would be extremely helpful. So would marking the 0 denominator when there is more than one candidate: "e = a/b + c/d". It should be easy to revise IDLE Shell's print_exception to tag the span. In some cases, the code that goes from a traceback line to a line in a file and marks it could do the same.
What would you do when the expression is not the last line?
try:
x/y
...
except Exception as e:
...
raise e
The except and raise might even be in a separate module.
I look forward to the PEP and discussion. pablogsal commented on May 1, 2021
So would marking the 0 denominator when there is more than one candidate: "e = a/b + c/d".
No, it will mark the offset of the bytecode that was getting executed when the exception was raised. Is just a way to mark what is raising the particular exception
What would you do when the expression is not the last line?
There is some logic needed to re-raise exceptions. But basically it boils down to comparate the line numbers to decide if you want to propagate or not the offsets. pablogsal commented on Jul 2, 2021 miss-islington commented on Jul 2, 2021 miss-islington commented on Jul 4, 2021 miss-islington commented on Jul 4, 2021 pablogsal commented on Jul 4, 2021 pablogsal commented on Jul 7, 2021 gpshead commented on Apr 17, 2022 As awkward of a beast as a tuple subclass emulating the previous fixed size namedtuple while adding additional attributes seems, it is very practical.
Users don't care about the specifics. They just need existing code to keep working and will want access to new values when updating this code or writing new code. Which means it should still be a tuple subclass for typing reasons known as inspect.Traceback and inspect.FrameInfo, support the existing indices and unpacking semantics, and support named field access. That this would be adding additional fields that indexing does not provide is a mere curiosity that most users would not notice as modern code should be field based anyways.
Nobody should care if the implementation behind the returned instances is unusual. The behavior will be what they expect. It avoids the need to expand into multiple different return types via an arg flag or the creation of parallel APIs.
This is a good practical example of why we should not design APIs to return tuples if they could conceivably change in the future. And why people unpacking on API calls that return tuples might find it wise to always [:slice] the return value as a style idiom.
All this said... we could also just add another field to the namedtuple. Some code inspection among existing API users is required, but I expect most users either immediately unpack upon return value assignment or use field names. Mix and matching of indexing and named access on a single return value is probably rare. The easy modification to existing code using unpacking to deal with this kind of API change upon version upgrade is to append a [:5] or similar to the API call. We have had other tuple returning APIs add fields in the past where this was the workaround though I've forgotten what they were off the top of my head.
This would be the more disruptive option. It is nice to avoid this if possible as it delays people's ability to update to and even test their code on 3.11. So I still lean towards the hybrid "half namedtuple" class. Reacted by Jelle Zijlstra added 2 commits that reference this issue on Apr 21, 2022GH-88116: Use a compact format to represent end line and column offse… GH-88116: Document that PyCodeNew is dangerous, and make PyCode_NewEm… added a commit that references this issue on Apr 23, 2022gh-88116: Enhance the inspect frame APIs to use the extended position… added 2 commits that reference this issue on Jun 28, 2022pythongh-88116: Avoid undefined behavior when decoding varints in cod… gh-88116: Avoid undefined behavior when decoding varints in code obje… added 2 commits that reference this issue on Jun 28, 2022pythongh-88116: Avoid undefined behavior when decoding varints in cod… gh-88116: Avoid undefined behavior when decoding varints in code obje… pablogsal commented on Jun 28, 2022 @markshannon I'm noticing that the implementation for varints and signed varints that we use has two encodings for 0 (both uval==0 and uval==1 decode to 0), and no way to encode -2**31. Is this a problem? What prevents us to need to encode -2**31? pablogsal commented on Jun 28, 2022 Also, weirdly there are two ways to encode 0 and no ways to encode -INT_MIN. @markshannon is this expected?
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|