GeezerVison is showing github.com — Direct unit tests for compiler optimisations · Issue #93678 · python/cpython
Direct unit tests for compiler optimisations #93678DescriptionWe currently do not have a way to directly test compiler optimisations, and the tests we have are not always reliable (more background).
We need to be able to feed a sequence of bytecode instructions into the optimiser from python, and get its output back into python. Then we can write tests. Reacted by Dennis Sweeney and Erlend E. Aasland Activityadded 2 commits that reference this issue on Jun 14, 2022GH-93678: reduce boilerplate and code repetition in the compiler (GH-… GH-93678: refactor compiler so that optimizer does not need the assem… added 6 commits that reference this issue on Jul 19, 2022gh-93678: move normalize_basic_block and extend_block call into optim… pythongh-93678: extract 'struct cfg' from the compiler so that the CF… pythongh-93678: extract 'struct cfg' from the compiler so that the CF… gh-93678: extract 'struct cfg_builder' from the compiler so that the … pythongh-93678: extract 'struct cfg' from the compiler so that the CF… tiran commented on Jul 26, 2022 The PR seems to have increase stack consumption slightly. Some tests are now failing on WASI because the WASM runtime is running out of stack space, https://buildbot.python.org/all/#/builders/1046/builds/278
In the past I have solved similar issues by replacing a hard-coded limit (e.g. 1000) with the current recursion limit. WASI has a lower default recursion limit. iritkatriel commented on Jul 26, 2022
The PR seems to have increase stack consumption slightly. Some tests are now failing on WASI because the WASM runtime is running out of stack space, https://buildbot.python.org/all/#/builders/1046/builds/278
In the past I have solved similar issues by replacing a hard-coded limit (e.g. 1000) with the current recursion limit. WASI has a lower default recursion limit.
Thanks for the ping. I would guess it's because we now have things like compiler_new_block calling cfg_builder_new_block. It basically glue to avoid changing all the calls. I'll inline/replace them in a future PR. Reacted by Christian Heimes tiran commented on Jul 26, 2022 I have added a temporary band-aid in GH-95296. Feel free to remove the skips if you have a solution for the additional stack use. iritkatriel commented on Jul 26, 2022
I have added a temporary band-aid in GH-95296. Feel free to remove the skips if you have a solution for the additional stack use.
Is there a way I can reproduce it? tiran commented on Jul 27, 2022 Here is a full WASM stack trace generated with
WASMTIME_BACKTRACE_DETAILS=1 \
wasmtime run --env PYTHONPATH=/builddir/wasi/$(cat pybuilddir.txt) --mapdir /::../../ -- \
python.wasm -m test -v test_compile -m test_compiler_recursion_limit
wasi_stacktrace.txt
test_compiler_recursion_limit (test.test_compile.TestSpecifics.test_compiler_recursion_limit) ... Error: failed to run main module `python.wasm`
Caused by:
0: failed to invoke command default
1: wasm trap: out of bounds memory access
wasm backtrace:
0: 0x15d8eb - compiler_visit_expr
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:5868:37
1: 0x15ee62 - compiler_call
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:4846:5
- compiler_visit_expr1
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:5802:16
- compiler_visit_expr
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:5870:15
2: 0x15ee62 - compiler_call
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:4846:5
- compiler_visit_expr1
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:5802:16
- compiler_visit_expr
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:5870:15
3: 0x15ee62 - compiler_call
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:4846:5
- compiler_visit_expr1
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:5802:16
- compiler_visit_expr
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:5870:15
4: 0x15ee62 - compiler_call
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:4846:5
- compiler_visit_expr1
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:5802:16
- compiler_visit_expr
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:5870:15
5: 0x15ee62 - compiler_call
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:4846:5
- compiler_visit_expr1
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:5802:16
- compiler_visit_expr
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:5870:15
6: 0x15ee62 - compiler_call
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:4846:5
- compiler_visit_expr1
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:5802:16
- compiler_visit_expr
at /python-wasm/cpython/builddir/wasi/../../Python/compile.c:5870:15
iritkatriel commented on Jul 27, 2022 I created #95335 to track the WASM problem. added a commit that references this issue on Aug 24, 2022gh-93678: add _testinternalcapi.optimize_cfg() and test utils for com…
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|