GeezerVison is showing github.com — Python/compile.c:7482: int compute_code_flags(struct compiler *): Assertion `IS_TOP_LEVEL_AWAIT(c) || _PyST_IsFunctionLike(ste)' failed · Issue #121637 · python/cpython
Python/compile.c:7482: int compute_code_flags(struct compiler *): Assertion `IS_TOP_LEVEL_AWAIT(c) || _PyST_IsFunctionLike(ste)' failed #121637DescriptionCrash report
What happened?
~/p/cpython ❯❯❯ ./python.exe -c 'compile("assert await u", "", "exec", optimize=2)'
Assertion failed: (IS_TOP_LEVEL_AWAIT(c) || _PyST_IsFunctionLike(ste)), function compute_code_flags, file compile.c, line 7482.
fish: Job 1, './python.exe -c 'compile("asser…' terminated by signal SIGABRT (Abort)
Full stacktrace:
fuzz_pycompile: Python/compile.c:7482: int compute_code_flags(struct compiler *): Assertion `IS_TOP_LEVEL_AWAIT(c) \|\| _PyST_IsFunctionLike(ste)' failed.
--
| ==24717== ERROR: libFuzzer: deadly signal
| #0 0x59033612cf71 in __sanitizer_print_stack_trace /src/llvm-project/compiler-rt/lib/asan/asan_stack.cpp:87:3
| #1 0x59033602fd98 in fuzzer::PrintStackTrace() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerUtil.cpp:210:5
| #2 0x590336012ac3 in fuzzer::Fuzzer::CrashCallback() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:231:3
| #3 0x79353882e41f in libpthread.so.0
| #4 0x79353862800a in __libc_signal_restore_set /build/glibc-SzIz7B/glibc-2.31/sysdeps/unix/sysv/linux/internal-signals.h:86:3
| #5 0x79353862800a in raise /build/glibc-SzIz7B/glibc-2.31/sysdeps/unix/sysv/linux/raise.c:48:3
| #6 0x793538607858 in abort /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:79:7
| #7 0x793538607728 in __assert_fail_base /build/glibc-SzIz7B/glibc-2.31/assert/assert.c:92:3
| #8 0x793538618fd5 in __assert_fail /build/glibc-SzIz7B/glibc-2.31/assert/assert.c:101:3
| #9 0x5903364b52da in compute_code_flags cpython3/Python/compile.c:7482:9
| #10 0x5903364b52da in optimize_and_assemble cpython3/Python/compile.c:7578:22
| #11 0x5903364af709 in compiler_mod cpython3/Python/compile.c:1597:10
| #12 0x5903364af709 in _PyAST_Compile cpython3/Python/compile.c:422:24
| #13 0x59033660c8a0 in Py_CompileStringObject cpython3/Python/pythonrun.c:1450:10
| #14 0x59033660c994 in Py_CompileStringExFlags cpython3/Python/pythonrun.c:1463:10
| #15 0x590336161d2e in fuzz_pycompile cpython3/Modules/_xxtestfuzz/fuzzer.c:551:24
| #16 0x590336161d2e in _run_fuzz cpython3/Modules/_xxtestfuzz/fuzzer.c:570:14
| #17 0x590336161d2e in LLVMFuzzerTestOneInput cpython3/Modules/_xxtestfuzz/fuzzer.c:711:11
| #18 0x590336013fe0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:614:13
| #19 0x590335ffe774 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:327:6
| #20 0x59033600420a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:862:9
| #21 0x590336030602 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
| #22 0x793538609082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
| #23 0x590335ff524d in _start
|
This regressed somewhere in f621618...690b935
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
No response
Linked PRs
Reacted by Gregory P. Smith Activityalex commented on Jul 12, 2024 cc: @iritkatriel -- it looks like you've got a few commits in this file in that range, though I don't see one that's obviously relevant. JelleZijlstra commented on Jul 12, 2024 sobolevn commented on Jul 12, 2024 I will add both asyncio and interpreter-core, because both are related. JelleZijlstra commented on Jul 12, 2024 Irit is on vacation, I'll look into this over the next few days. JelleZijlstra commented on Jul 13, 2024 The problem is arguably this:
% python -c 'assert await 42'
File "<string>", line 1
SyntaxError: 'await' outside function
% python -O -c 'assert await 42'
%
It used to be that we didn't raise a SyntaxError if there is an await but it is optimized away, and the code would work fine.
Now, however, the symtable sets ste_coroutine when it sees the await, but it still doesn't raise an error about the await, leading to the crash reported here.
I can see two solutions:
The latter seems like the more intuitive behavior for users.
It's also consistent with other cases where optimized-away code leads to a compile error:
% python -O
Python 3.12.1 (main, Apr 22 2024, 00:22:15) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def f():
... assert (x := 3)
... global x
...
File "<stdin>", line 3
SyntaxError: name 'x' is assigned to before global declaration
So I'll send a PR to make python -O -c 'assert await 1' a SyntaxError. alex commented on Jul 13, 2024 Always doing a syntax error, even if the assert is optimized out, feels like the right behavior to me. JelleZijlstra commented on Jul 13, 2024 I realized there's a few more similar cases (essentially, every place we call compiler_error in compile.c while inside an expression):
% python -c 'assert (yield)'
File "<string>", line 1
SyntaxError: 'yield' outside function
% python -O -c 'assert (yield)'
% python -O -c 'assert (__debug__ := True)'
% python -c 'assert (__debug__ := True)'
File "<string>", line 1
SyntaxError: cannot assign to __debug__
% python -O -c 'assert (__debug__ := True)'
% python -c 'assert [x async for x in ()]'
File "<string>", line 1
SyntaxError: asynchronous comprehension outside of an asynchronous function
% python -O -c 'assert [x async for x in ()]'
The latter also crashes on current main:
% ./python.exe -O -c 'assert [x async for x in ()]'
Assertion failed: (IS_TOP_LEVEL_AWAIT(c) || _PyST_IsFunctionLike(ste)), function compute_code_flags, file compile.c, line 7482.
zsh: abort ./python.exe -O -c 'assert [x async for x in ()]'
JelleZijlstra commented on Jul 13, 2024 The linked PR makes symtable.c detect incorrect await and async comprehensions, fixing this issue.
I didn't bother detecting other compiler errors (e.g., invalid yields) because they don't cause similar crashes. I wouldn't be surprised though if there's a crash lurking somewhere with e.g. assert (yield) being optimized away, so maybe if you point your fuzzer in that direction it will find something :) gvanrossum commented on Jul 13, 2024
Always doing a syntax error, even if the assert is optimized out, feels like the right behavior to me.
Of course! It must have been an oversight that we didn't do this from the start.
I'm removing the topic-asyncio label -- just because it mentions await doesn't make it an asyncio issue. JelleZijlstra commented on Jul 22, 2024 Thanks for the report! This will now be a SyntaxError.
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|