GeezerVison is showing github.com — Crash on AST with misordered linenos · Issue #92597 · python/cpython
Crash on AST with misordered linenos #92597Description
- Compile 3.11 branch
./python.exe -m ensurepip./python.exe -m pip install pytest-cov./python.exe -m pytest
Crashes with Assertion failed: (i->i_end_lineno >= i->i_lineno), function write_location_info_long_form, file compile.c, line 7532.
lldb stack trace:
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert
frame #0: 0x00007fff203cd92e libsystem_kernel.dylib`__pthread_kill + 10
frame #1: 0x00007fff203fc5bd libsystem_pthread.dylib`pthread_kill + 263
frame #2: 0x00007fff20351406 libsystem_c.dylib`abort + 125
frame #3: 0x00007fff203507d8 libsystem_c.dylib`__assert_rtn + 314
* frame #4: 0x000000010027c183 python.exe`write_location_info_long_form.cold.2 at compile.c:7532:5 [opt]
frame #5: 0x000000010018f469 python.exe`write_location_info_long_form(a=<unavailable>, i=<unavailable>, length=<unavailable>) at compile.c:7532:5 [opt]
frame #6: 0x000000010018f27d python.exe`write_location_info_entry(a=0x00007ffeefbfe5d0, i=0x000000010388ac98, isize=1) at compile.c:7590:5 [opt]
frame #7: 0x000000010018d2cd python.exe`assemble_emit_location(a=0x00007ffeefbfe5d0, i=0x000000010388ac98) at compile.c:7605:12 [opt]
frame #8: 0x000000010017e9fa python.exe`assemble(c=<unavailable>, addNone=<unavailable>) at compile.c:8411:18 [opt]
frame #9: 0x000000010017d4a3 python.exe`compiler_mod(c=0x00007ffeefbfe6b8, mod=<unavailable>) at compile.c:2203:10 [opt]
frame #10: 0x000000010017d2b3 python.exe`_PyAST_Compile(mod=0x00000001038852b0, filename=0x000000010654d630, flags=<unavailable>, optimize=<unavailable>, arena=0x00000001064b0100) at compile.c:581:10 [opt]
frame #11: 0x00000001001634e0 python.exe`builtin_compile_impl(module=<unavailable>, source=0x00000001065a3fc0, filename=0x000000010654d630, mode=<unavailable>, flags=<unavailable>, dont_inherit=1, optimize=-1, feature_version=-1) at bltinmodule.c:818:33 [opt]
frame #12: 0x0000000100161fb6 python.exe`builtin_compile(module=<unavailable>, args=0x00007ffeefbfe7f0, nargs=<unavailable>, kwnames=<unavailable>) at bltinmodule.c.h:328:20 [opt]
frame #13: 0x00000001000cbb08 python.exe`cfunction_vectorcall_FASTCALL_KEYWORDS(func=0x0000000100b9f4d0, args=0x0000000100c6fe00, nargsf=<unavailable>, kwnames=0x0000000102c72260) at methodobject.c:443:24 [opt]
frame #14: 0x0000000100087018 python.exe`_PyObject_VectorcallTstate(tstate=0x0000000100486930, callable=0x0000000100b9f4d0, args=0x0000000100c6fe00, nargsf=9223372036854775811, kwnames=0x0000000102c72260) at pycore_call.h:92:11 [opt]
frame #15: 0x0000000100086fa0 python.exe`PyObject_Vectorcall(callable=<unavailable>, args=<unavailable>, nargsf=<unavailable>, kwnames=<unavailable>) at call.c:299:12 [opt]
frame #16: 0x0000000100170d7c python.exe`_PyEval_EvalFrameDefault(tstate=0x0000000100486930, frame=0x0000000100c6fd70, throwflag=<unavailable>) at ceval.c:4826:23 [opt]
frame #17: 0x000000010016680d python.exe`_PyEval_EvalFrame(tstate=<unavailable>, frame=<unavailable>, throwflag=<unavailable>) at pycore_ceval.h:66:16 [opt]
frame #18: 0x0000000100166726 python.exe`_PyEval_Vector(tstate=0x0000000100486930, func=0x0000000104737d80, locals=0x0000000000000000, args=0x00007ffeefbfeb80, argcount=3, kwnames=<unavailable>) at ceval.c:6468:24 [opt]
frame #19: 0x0000000100087330 python.exe`_PyFunction_Vectorcall(func=0x0000000104737d80, stack=0x00007ffeefbfeb80, nargsf=<unavailable>, kwnames=0x0000000000000000) at call.c:0:45 [opt]
frame #20: 0x0000000100089fd8 python.exe`_PyObject_VectorcallTstate(tstate=0x0000000100486930, callable=0x0000000104737d80, args=0x00007ffeefbfeb80, nargsf=3, kwnames=0x0000000000000000) at pycore_call.h:92:11 [opt]
frame #21: 0x0000000100089644 python.exe`method_vectorcall(method=<unavailable>, args=0x00000001064c9788, nargsf=<unavailable>, kwnames=0x0000000000000000) at classobject.c:89:18 [opt]
frame #22: 0x0000000100086f3d python.exe`_PyVectorcall_Call(tstate=0x0000000100486930, func=(python.exe`method_vectorcall at classobject.c:44), callable=0x000000010640d310, tuple=0x00000001064c9770, kwargs=0x0000000000000000) at call.c:245:16 [opt]
frame #23: 0x0000000100087148 python.exe`_PyObject_Call(tstate=0x0000000100486930, callable=0x000000010640d310, args=0x00000001064c9770, kwargs=0x0000000000000000) at call.c:328:16 [opt]
frame #24: 0x0000000100087209 python.exe`PyObject_Call(callable=<unavailable>, args=<unavailable>, kwargs=<unavailable>) at call.c:355:12 [opt]
frame #25: 0x0000000100177419 python.exe`do_call_core(tstate=0x0000000100486930, func=0x000000010640d310, callargs=0x00000001064c9770, kwdict=0x0000000000000000, use_tracing=<unavailable>) at ceval.c:0:13 [opt]
frame #26: 0x00000001001733b0 python.exe`_PyEval_EvalFrameDefault(tstate=0x0000000100486930, frame=0x0000000100c6f580, throwflag=<unavailable>) at ceval.c:5431:22 [opt]
frame #27: 0x000000010016680d python.exe`_PyEval_EvalFrame(tstate=<unavailable>, frame=<unavailable>, throwflag=<unavailable>) at pycore_ceval.h:66:16 [opt]
frame #28: 0x0000000100166726 python.exe`_PyEval_Vector(tstate=0x0000000100486930, func=0x0000000104053490, locals=0x0000000000000000, args=0x00000001064c0a48, argcount=1, kwnames=<unavailable>) at ceval.c:6468:24 [opt]
frame #29: 0x0000000100087330 python.exe`_PyFunction_Vectorcall(func=0x0000000104053490, stack=0x00000001064c0a48, nargsf=<unavailable>, kwnames=0x00000001064c8f50) at call.c:0:45 [opt]
frame #30: 0x00000001000866b2 python.exe`_PyObject_FastCallDictTstate(tstate=0x0000000100486930, callable=0x0000000104053490, args=0x00007ffeefbfef70, nargsf=1, kwargs=0x00000001064c6d50) at call.c:152:15 [opt]
frame #31: 0x00000001000875e4 python.exe`_PyObject_Call_Prepend(tstate=<unavailable>, callable=<unavailable>, obj=0x000000010622f3a0, args=<unavailable>, kwargs=<unavailable>) at call.c:482:24 [opt]
frame #32: 0x00000001000ee1f9 python.exe`slot_tp_call(self=0x000000010622f3a0, args=0x000000010046c5a0, kwds=0x00000001064c6d50) at typeobject.c:7604:15 [opt]
frame #33: 0x000000010008689c python.exe`_PyObject_MakeTpCall(tstate=0x0000000100486930, callable=0x000000010622f3a0, args=0x0000000100c6f408, nargs=<unavailable>, keywords=0x0000000104694960) at call.c:214:18 [opt]
frame #34: 0x0000000100087049 python.exe`_PyObject_VectorcallTstate(tstate=0x0000000100486930, callable=0x000000010622f3a0, args=0x0000000100c6f408, nargsf=9223372036854775808, kwnames=0x0000000104694960) at pycore_call.h:90:16 [opt]
frame #35: 0x0000000100086fa0 python.exe`PyObject_Vectorcall(callable=<unavailable>, args=<unavailable>, nargsf=<unavailable>, kwnames=<unavailable>) at call.c:299:12 [opt]
frame #36: 0x0000000100170d7c python.exe`_PyEval_EvalFrameDefault(tstate=0x0000000100486930, frame=0x0000000100c6f380, throwflag=<unavailable>) at ceval.c:4826:23 [opt]
frame #37: 0x000000010016680d python.exe`_PyEval_EvalFrame(tstate=<unavailable>, frame=<unavailable>, throwflag=<unavailable>) at pycore_ceval.h:66:16 [opt]
frame #38: 0x0000000100166726 python.exe`_PyEval_Vector(tstate=0x0000000100486930, func=0x0000000102881b20, locals=0x0000000102830350, args=0x0000000000000000, argcount=0, kwnames=<unavailable>) at ceval.c:6468:24 [opt]
frame #39: 0x0000000100166615 python.exe`PyEval_EvalCode(co=<unavailable>, globals=0x0000000102830350, locals=0x0000000102830350) at ceval.c:1207:21 [opt]
frame #40: 0x0000000100163b09 python.exe`builtin_exec_impl(module=<unavailable>, source=0x000000010617f230, globals=0x0000000102830350, locals=0x0000000102830350, closure=<unavailable>) at bltinmodule.c:1080:17 [opt]
frame #41: 0x000000010016224d python.exe`builtin_exec(module=<unavailable>, args=0x0000000100c6f180, nargs=2, kwnames=0x0000000000000000) at bltinmodule.c.h:465:20 [opt]
frame #42: 0x00000001000cbb08 python.exe`cfunction_vectorcall_FASTCALL_KEYWORDS(func=0x0000000100b9f6b0, args=0x0000000100c6f180, nargsf=<unavailable>, kwnames=0x0000000000000000) at methodobject.c:443:24 [opt]
frame #43: 0x0000000100087018 python.exe`_PyObject_VectorcallTstate(tstate=0x0000000100486930, callable=0x0000000100b9f6b0, args=0x0000000100c6f180, nargsf=9223372036854775810, kwnames=0x0000000000000000) at pycore_call.h:92:11 [opt]
frame #44: 0x0000000100086fa0 python.exe`PyObject_Vectorcall(callable=<unavailable>, args=<unavailable>, nargsf=<unavailable>, kwnames=<unavailable>) at call.c:299:12 [opt]
frame #45: 0x0000000100170d7c python.exe`_PyEval_EvalFrameDefault(tstate=0x0000000100486930, frame=0x0000000100c6f0d8, throwflag=<unavailable>) at ceval.c:4826:23 [opt]
frame #46: 0x000000010016680d python.exe`_PyEval_EvalFrame(tstate=<unavailable>, frame=<unavailable>, throwflag=<unavailable>) at pycore_ceval.h:66:16 [opt]
frame #47: 0x0000000100166726 python.exe`_PyEval_Vector(tstate=0x0000000100486930, func=0x0000000102c5eaf0, locals=0x0000000000000000, args=0x0000000102894ba8, argcount=2, kwnames=<unavailable>) at ceval.c:6468:24 [opt]
frame #48: 0x0000000100087330 python.exe`_PyFunction_Vectorcall(func=0x0000000102c5eaf0, stack=0x0000000102894ba8, nargsf=<unavailable>, kwnames=0x0000000000000000) at call.c:0:45 [opt]
frame #49: 0x0000000100086f3d python.exe`_PyVectorcall_Call(tstate=0x0000000100486930, func=(python.exe`_PyFunction_Vectorcall at call.c:385), callable=0x0000000102c5eaf0, tuple=0x0000000102894b90, kwargs=0x0000000000000000) at call.c:245:16 [opt]
frame #50: 0x0000000100087148 python.exe`_PyObject_Call(tstate=0x0000000100486930, callable=0x0000000102c5eaf0, args=0x0000000102894b90, kwargs=0x0000000000000000) at call.c:328:16 [opt]
frame #51: 0x0000000100087209 python.exe`PyObject_Call(callable=<unavailable>, args=<unavailable>, kwargs=<unavailable>) at call.c:355:12 [opt]
frame #52: 0x00000001001d9f3d python.exe`pymain_run_module(modname=<unavailable>, set_argv0=<unavailable>) at main.c:300:14 [opt]
frame #53: 0x00000001001d982d python.exe`pymain_run_python(exitcode=0x00007ffeefbff7dc) at main.c:595:21 [opt]
frame #54: 0x00000001001d9718 python.exe`Py_RunMain at main.c:680:5 [opt]
frame #55: 0x00000001001d99af python.exe`pymain_main(args=<unavailable>) at main.c:710:12 [opt]
frame #56: 0x00000001001d99fb python.exe`Py_BytesMain(argc=<unavailable>, argv=<unavailable>) at main.c:734:12 [opt]
frame #57: 0x0000000100001999 python.exe`main(argc=<unavailable>, argv=<unavailable>) at python.c:15:12 [opt]
frame #58: 0x00007fff20417f3d libdyld.dylib`start + 1
Your environment
- CPython versions tested on: 3.11 current head
- Operating system and architecture: macOS
I put print statements in that show the file it crashes on is '/usr/local/lib/python3.11/site-packages/pytest_cov/plugin.py', but just running Python on that file doesn't reproduce the crash. I'll try to debug some more. ActivityJelleZijlstra commented on May 10, 2022 Simple repro:
from ast import *
tree = Module(body=[
Import(names=[alias(name='builtins', lineno=1, col_offset=0)], lineno=1, col_offset=0),
Import(names=[alias(name='traceback', lineno=0, col_offset=0)], lineno=0, col_offset=1)
], type_ignores=[])
compile(tree, "x", "exec", dont_inherit=True)% ./python.exe wronglinenos.py
Assertion failed: (i->i_end_lineno >= i->i_lineno), function write_location_info_long_form, file compile.c, line 7532.
zsh: abort ./python.exe wronglinenos.py
changed the title [-]Compiler crash on pytest-cov[/-] [+]Crash on AST with misordered linenos[/+] on May 10, 2022kumaraditya303 commented on May 10, 2022 944fffe is the first bad commit
commit 944fffee8916cb94321fa33cd3a43f4108717746
Author: Mark Shannon <[email protected]>
Date: Thu Apr 21 16:10:37 2022 +0100
GH-88116: Use a compact format to represent end line and column offsets. (GH-91666)
* Stores all location info in linetable to conform to PEP 626.
* Remove column table from code objects.
* Remove end-line table from code objects.
* Document new location table format
iritkatriel commented on May 10, 2022 @markshannon
Looks like the ast defaults the end_lineno to None, which then becomes 0 in the compiler. Maybe the ast should default it be equal to lineno? Or should None become -1 in c? markshannon commented on May 10, 2022 Both, perhaps.
If no end_lineno is specified, then it should probably default to lineno
If the end_lineno is None then we should convert that to -1, as we do for lineno. iritkatriel commented on May 11, 2022 pablogsal commented on May 11, 2022
Both, perhaps.
If no end_lineno is specified, then it should probably default to lineno
If the end_lineno is None then we should convert that to -1, as we do for lineno.
That's what we do in some parts that deal with these information. I can prepare a PR pablogsal commented on May 11, 2022 The question is if we should manage this in the AST creation routines or in the compiler. I assume that the first is more resilient but what do others think? The-Compiler commented on May 25, 2022 FWIW, this seems to affect pytest's assertion rewriting as well. As soon as I'm running pytest against e.g.:
with a Python 3.11.0b1 configured with --with-pydebug, I get:
collecting ... python: Python/compile.c:7531: write_location_info_long_form: Assertion `i->i_end_lineno >= i->i_lineno' failed.
Fatal Python error: Aborted
Current thread 0x00007f2555185740 (most recent call first):
File ".../site-packages/_pytest/assertion/rewrite.py", line 361 in _rewrite_test
File ".../site-packages/_pytest/assertion/rewrite.py", line 159 in exec_module
...
which is here:
https://github.com/pytest-dev/pytest/blob/c988e49af63a50f0704a46a93e73463666451af0/src/_pytest/assertion/rewrite.py#L361
and indeed i_end_lineno is 0:
(gdb) pp i
i =
autoderefcount="1",[
i_opcode = <int> = {"100"}
i_oparg = <int> = {"0"}
i_target = <struct basicblock_*> = {"0x0"}
i_except = <struct basicblock_*> = {"0x0"}
i_lineno = <int> = {"1"}
i_end_lineno = <int> = {"0"}
i_col_offset = <int> = {"0"}
i_end_col_offset = <int> = {"0"}
],<struct instr> = {"{...}"}
Turning pytest's assertion rewriting off with --assert=plain helps, and I can confirm it works fine before 944fffe.
Passing the end_lineno where we generate fake imports for the pytest assertion rewriting helpers seems to help:
diff --git i/src/_pytest/assertion/rewrite.py w/src/_pytest/assertion/rewrite.py
index 81096764e..5d7392dd4 100644
--- i/src/_pytest/assertion/rewrite.py
+++ w/src/_pytest/assertion/rewrite.py
@@ -727,7 +727,7 @@ def run(self, mod: ast.Module) -> None:
ast.alias("_pytest.assertion.rewrite", "@pytest_ar"),
]
imports = [
- ast.Import([alias], lineno=lineno, col_offset=0) for alias in aliases
+ ast.Import([alias], lineno=lineno, end_lineno=item.end_lineno, col_offset=0) for alias in aliases
]
mod.body[pos:pos] = imports
cc @bluetech - though I suppose we don't necessarily need to fix this in pytest, hence I didn' t open an issue over there. The-Compiler commented on May 25, 2022 Another project affected in the wild seems to be flask:
$ python3.11 -X faulthandler -c "import flask; flask.Flask('test')"
python: Python/compile.c:7534: write_location_info_long_form: Assertion `i->i_end_lineno >= i->i_lineno' failed.
Fatal Python error: Aborted
Current thread 0x00007fd44da08740 (most recent call first):
File ".../lib/python3.11/site-packages/werkzeug/routing.py", line 1073 in _compile_builder
File ".../lib/python3.11/site-packages/werkzeug/routing.py", line 885 in compile
File ".../lib/python3.11/site-packages/werkzeug/routing.py", line 805 in bind
File ".../lib/python3.11/site-packages/werkzeug/routing.py", line 1546 in add
File ".../lib/python3.11/site-packages/flask/app.py", line 1086 in add_url_rule
File ".../lib/python3.11/site-packages/flask/scaffold.py", line 56 in wrapper_func
File ".../lib/python3.11/site-packages/flask/app.py", line 511 in __init__
File "<string>", line 1 in <module>
pablogsal commented on May 25, 2022 I'm working on a patch, will have a draft PR ready soon markshannon commented on May 27, 2022 @pablogsal Do you have time to work on this, or should I fix it? pablogsal commented on May 30, 2022
@pablogsal Do you have time to work on this, or should I fix it?
I have opened a PR for this. added a commit that references this issue on May 31, 2022gh-92597: Ensure that AST nodes without explicit end positions can be… added 2 commits that reference this issue on May 31, 2022gh-92597: Ensure that AST nodes without explicit end positions can be… pythongh-92597: Improve error message for AST nodes with invalid rang… added 2 commits that reference this issue on Jun 1, 2022gh-92597: Improve error message for AST nodes with invalid ranges (GH… gh-92597: Improve error message for AST nodes with invalid ranges (GH…
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|