GeezerVison

$ CC="clang" CXX="clang++" LDFLAGS="${LDFLAGS:+$LDFLAGS }-fuse-ld=lld" PYTHON_CONFIGURE_OPTS="--enable-optimizations --with-lto" PYTHON_CFLAGS="-march=native -mtune=native" pyenv install 3.14.4

$ python -m test -v test_call
== CPython 3.14.4 (main, Apr 9 2026, 07:24:07) [Clang 22.1.3 (++20260402073256+4250a0fc5de9-1~exp1~20260402073413.57)]
== Linux-6.8.0-107-generic-x86_64-with-glibc2.39 little-endian
== Python build: release shared ThinLTO+PGO
== cwd: /tmp/test_python_worker_93691æ
== CPU count: 4
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 1791553283
0:00:00 load avg: 0.00 Run 1 test sequentially in a single process
0:00:00 load avg: 0.00 [1/1] test_call
test_module_not_callable_no_suggestion_0 (test.test_call.CFunctionCallsErrorMessages.test_module_not_callable_no_suggestion_0) ... ok
...
...
test_margin_is_sufficient (test.test_call.TestRecursion.test_margin_is_sufficient) ... FAIL
test_recursion_with_kwargs (test.test_call.TestRecursion.test_recursion_with_kwargs) ... ok
test_super_deep (test.test_call.TestRecursion.test_super_deep) ... Fatal Python error: Segmentation fault

Current thread 0x00007a261a915b80 [python] (most recent call first):
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse

Current thread's C stack trace (most recent call first):
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _Py_DumpStack+0x29 [0x7a261b007919]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x60dd5f [0x7a261b00dd5f]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x60dc2d [0x7a261b00dc2d]
  Binary file "/lib/x86_64-linux-gnu/libc.so.6", at +0x45330 [0x7a261a645330]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x226 [0x7a261ac164b6]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2a7e3c [0x7a261aca7e3c]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7a261ad003a9]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1da74 [0x7a261ac33d04]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2a7e3c [0x7a261aca7e3c]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7a261ad003a9]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1da74 [0x7a261ac33d04]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2a7e3c [0x7a261aca7e3c]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7a261ad003a9]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1da74 [0x7a261ac33d04]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2a7e3c [0x7a261aca7e3c]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7a261ad003a9]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1da74 [0x7a261ac33d04]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2a7e3c [0x7a261aca7e3c]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7a261ad003a9]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1da74 [0x7a261ac33d04]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2a7e3c [0x7a261aca7e3c]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7a261ad003a9]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1da74 [0x7a261ac33d04]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2a7e3c [0x7a261aca7e3c]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7a261ad003a9]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1da74 [0x7a261ac33d04]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2a7e3c [0x7a261aca7e3c]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7a261ad003a9]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1da74 [0x7a261ac33d04]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2a7e3c [0x7a261aca7e3c]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7a261ad003a9]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1da74 [0x7a261ac33d04]
  <truncated rest of calls>

Extension modules: _testcapi, _testlimitedcapi, _testinternalcapi (total: 3)
Segmentation fault (core dumped)

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.4 (main, Apr 9 2026, 07:24:07) [Clang 22.1.3 (++20260402073256+4250a0fc5de9-1~~exp1~~20260402073413.57)]

Linked PRs

Activity

ppigazzini
added
type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump
on Apr 9, 2026
picnixz
added
testsTests in the Lib/test dirTests in the Lib/test dir
on Apr 9, 2026
Fidget-Spinner commented on Apr 9, 2026
Fidget-Spinner
on Apr 9, 2026
Member
More actions
Knowing our luck, there's a chance this might be due to LTO inlining things again that shouldn't be inlined. Can you try passing --with-tail-call-interp do your build and see if it still fails? If it doesn't, then that means it's time to play the game of finding the function to outline.

ppigazzini commented on Apr 9, 2026

ppigazzini

on Apr 9, 2026

Author

More actions

Passed that test using --with-tail-call-interp, now I'm running the whole test suit.

CC="clang" CXX="clang++" LDFLAGS="${LDFLAGS:+$LDFLAGS }-fuse-ld=lld" PYTHON_CONFIGURE_OPTS="--enable-optimizations --with-lto --with-tail-call-interp" PYTHON_CFLAGS="-march=native -mtune=native" pyenv install 3.14.4

python -m test test_call
Using random seed: 1273119896
0:00:00 load avg: 1.48 Run 1 test sequentially in a single process
0:00:00 load avg: 1.48 [1/1] test_call
0:00:00 load avg: 1.48 [1/1] test_call passed

== Tests result: SUCCESS ==

1 test OK.

Total duration: 208 ms
Total tests: run=186
Total test files: run=1/1
Result: SUCCESS

To add more context, it seems an issue highlighted by clang 22.
python 3.14.3 shows the same issue with clang 22 but it passed the tests with clang 21 (but for https://github.com/python/cpython/issues/142414d).

EDIT:

python 3.14.4 built with clang22 and --with-tail-call-interp completes the whole test (but the open issue with test_struct)

== Tests result: FAILURE ==

22 tests skipped:
    test.test_asyncio.test_windows_events
    test.test_asyncio.test_windows_utils test.test_gdb.test_backtrace
    test.test_gdb.test_cfunction test.test_gdb.test_cfunction_full
    test.test_gdb.test_misc test.test_gdb.test_pretty_print
    test_android test_apple test_asdl_parser test_clinic test_devpoll
    test_free_threading test_generated_cases test_kqueue test_launcher
    test_msvcrt test_startfile test_winapi test_winconsoleio
    test_winreg test_wmi

11 tests skipped (resource denied):
    test_curses test_peg_generator test_smtpnet test_socketserver
    test_tkinter test_ttk test_urllib2net test_urllibnet test_winsound
    test_xpickle test_zipfile64

1 test failed:
    test_struct

458 tests OK.

Total duration: 18 min 18 sec
Total tests: run=47,323 skipped=2,331
Total test files: run=481/492 failed=1 skipped=22 resource_denied=11
Result: FAILURE

Fidget-Spinner commented on Apr 9, 2026

Fidget-Spinner

on Apr 9, 2026

Member

More actions

Just to make sure I'm diagnosing the issue correctly, please try this fix on 3.14 and tell me if it works:

Fix 1:

diff --git a/Python/ceval_macros.h b/Python/ceval_macros.h
index 4a878d6dff4..5b9762866a2 100644
--- a/Python/ceval_macros.h
+++ b/Python/ceval_macros.h
@@ -425,7 +425,7 @@ do { \
 /* Stackref macros */
 
 /* How much scratch space to give stackref to PyObject* conversion. */
-#define MAX_STACKREF_SCRATCH 10
+#define MAX_STACKREF_SCRATCH 1
 
 #define STACKREFS_TO_PYOBJECTS(ARGS, ARG_COUNT, NAME) \
     /* +1 because vectorcall might use -1 to write self */ \

Reacted by ppigazzini

ppigazzini commented on Apr 9, 2026

ppigazzini

on Apr 9, 2026

Author

More actions

The patch looks good.

usr00@pp-lap-00:~$ CC="clang" CXX="clang++" \
LDFLAGS="${LDFLAGS:+$LDFLAGS }-fuse-ld=lld" \
PYTHON_CONFIGURE_OPTS="--enable-optimizations --with-lto" \
PYTHON_CFLAGS="-march=native -mtune=native" \
pyenv install --patch 3.14.4 < ~/patches/stackref.patch
Downloading Python-3.14.4.tar.xz...
-> https://www.python.org/ftp/python/3.14.4/Python-3.14.4.tar.xz
Installing Python-3.14.4...
patching file Python/ceval_macros.h
Installed Python-3.14.4 to /home/usr00/.pyenv/versions/3.14.4

usr00@pp-lap-00:~$ pyenv shell 3.14.4

usr00@pp-lap-00:~$ python -m test test_call
Using random seed: 2420094728
0:00:00 load avg: 3.52 Run 1 test sequentially in a single process
0:00:00 load avg: 3.52 [1/1] test_call
0:00:00 load avg: 3.52 [1/1] test_call passed

== Tests result: SUCCESS ==

1 test OK.

Total duration: 217 ms
Total tests: run=186
Total test files: run=1/1
Result: SUCCESS

usr00@pp-lap-00:~$ python -VV
Python 3.14.4 (main, Apr  9 2026, 17:52:49) [Clang 22.1.3 (++20260402073256+4250a0fc5de9-1~exp1~20260402073413.57)]

Fidget-Spinner commented on Apr 9, 2026
Fidget-Spinner
on Apr 9, 2026
Member
More actions
Thanks, it's confirmed then, what's happening is the stack usage is too high with the stackref buffer. We need to move the it to the heap. I earlier suggested this, but it seems now we need it for correctness #138115
Reacted by ppigazzini
Fidget-Spinner
added
3.14bugs and security fixesbugs and security fixes

3.15bugs and security fixesbugs and security fixes
on Apr 9, 2026
Fidget-Spinner commented on Apr 9, 2026
Fidget-Spinner
on Apr 9, 2026
Member
More actions
I can't reproduce this locally with LLVM 22.1.3 and Python 3.14.4+. However, I believe this is probably real. Do you know if pyenv does anything special to their builds? I'm not using pyenv to test this out.

Also I just dissasembled the stack consumption (rsp) and compared a non-shared and shared version. It's rsp-1736 vs rsp-1784. So it seems stack consumption is higher on shared builds.
ppigazzini commented on Apr 9, 2026
ppigazzini
on Apr 9, 2026
Author
More actions
I'm not an expert of pyenv internals, I start sharing the log of a build with the issue python-build.20260409071620.66319.zip

Fidget-Spinner commented on Apr 9, 2026

Fidget-Spinner

on Apr 9, 2026

Member

More actions

@ppigazzini ok, last time debugging please :). If this works, I can fix the issue and mark this as release blocker

please apply the following patch, and let me know if things work:

diff --git a/Python/ceval.c b/Python/ceval.c
index 377b4644edd..b7867c72772 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -1238,6 +1238,8 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int
 #endif
     }
 
+    PyObject *STACKREF_SCRATCH[MAX_STACKREF_SCRATCH+1];
+
 #if defined(_Py_TIER2) && !defined(_Py_JIT)
     /* Tier 2 interpreter state */
     _PyExecutorObject *current_executor = NULL;
diff --git a/Python/ceval_macros.h b/Python/ceval_macros.h
index 4a878d6dff4..3b39ea91ec2 100644
--- a/Python/ceval_macros.h
+++ b/Python/ceval_macros.h
@@ -429,7 +429,7 @@ do { \
 
 #define STACKREFS_TO_PYOBJECTS(ARGS, ARG_COUNT, NAME) \
     /* +1 because vectorcall might use -1 to write self */ \
-    PyObject *NAME##_temp[MAX_STACKREF_SCRATCH+1]; \
+    PyObject **NAME##_temp = (PyObject **)&STACKREF_SCRATCH; \
     PyObject **NAME = _PyObjectArray_FromStackRefArray(ARGS, ARG_COUNT, NAME##_temp + 1);
 
 #define STACKREFS_TO_PYOBJECTS_CLEANUP(NAME) \

Viewing the dissassembly, the stack usage is cut by 1/3rd with this patch on LLVM 22.

Reacted by ppigazzini

ppigazzini commented on Apr 9, 2026

ppigazzini

on Apr 9, 2026

Author

More actions

Ok, give me 10 minutes.

$ CC="clang" CXX="clang++" \
LDFLAGS="${LDFLAGS:+$LDFLAGS }-fuse-ld=lld" \
PYTHON_CONFIGURE_OPTS="--enable-optimizations --with-lto" \
PYTHON_CFLAGS="-march=native -mtune=native" \
pyenv install --patch 3.14.4 < ~/patches/stackref_2.patch
Downloading Python-3.14.4.tar.xz...
-> https://www.python.org/ftp/python/3.14.4/Python-3.14.4.tar.xz
Installing Python-3.14.4...
patching file Python/ceval.c
patching file Python/ceval_macros.h

Reacted by Ken Jin

ppigazzini commented on Apr 9, 2026

ppigazzini

on Apr 9, 2026

Author

More actions

Logs of the build python-build.zip

$ python -VV
Python 3.14.4 (main, Apr  9 2026, 20:55:10) [Clang 22.1.3 (++20260402073256+4250a0fc5de9-1~exp1~20260402073413.57)]

python -m test test_call
Using random seed: 2721431693
0:00:00 load avg: 3.63 Run 1 test sequentially in a single process
0:00:00 load avg: 3.63 [1/1] test_call
Fatal Python error: Segmentation fault

Current thread 0x00007eab02994b80 [python] (most recent call first):
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse
  File "/home/usr00/.pyenv/versions/3.14.4/lib/python3.14/test/test_call.py", line 1081 in c_recurse

Current thread's C stack trace (most recent call first):
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _Py_DumpStack+0x29 [0x7eab0280f969]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x615d9f [0x7eab02815d9f]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x615c6d [0x7eab02815c6d]
  Binary file "/lib/x86_64-linux-gnu/libc.so.6", at +0x45330 [0x7eab01e45330]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x226 [0x7eab0241b856]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2ab715 [0x7eab024ab715]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7eab02501749]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1dc76 [0x7eab024392a6]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2ab715 [0x7eab024ab715]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7eab02501749]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1dc76 [0x7eab024392a6]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2ab715 [0x7eab024ab715]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7eab02501749]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1dc76 [0x7eab024392a6]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2ab715 [0x7eab024ab715]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7eab02501749]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1dc76 [0x7eab024392a6]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2ab715 [0x7eab024ab715]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7eab02501749]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1dc76 [0x7eab024392a6]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2ab715 [0x7eab024ab715]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7eab02501749]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1dc76 [0x7eab024392a6]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2ab715 [0x7eab024ab715]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7eab02501749]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1dc76 [0x7eab024392a6]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2ab715 [0x7eab024ab715]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7eab02501749]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1dc76 [0x7eab024392a6]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at +0x2ab715 [0x7eab024ab715]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at PyObject_Vectorcall+0x59 [0x7eab02501749]
  Binary file "/home/usr00/.pyenv/versions/3.14.4/lib/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x1dc76 [0x7eab024392a6]
  <truncated rest of calls>

Extension modules: _testcapi, _testlimitedcapi, _testinternalcapi (total: 3)
Segmentation fault (core dumped)

Fidget-Spinner commented on Apr 9, 2026

Fidget-Spinner

on Apr 9, 2026

Member

More actions

@ppigazzini thanks for reporting back.

I'm really sorry to trouble you again (I can't reproduce this locally, so I have to rely on you for this). Can you please try the following diff applied on its own without the other patches?

diff --git a/Modules/_testcapi/vectorcall.c b/Modules/_testcapi/vectorcall.c
index f89dcb6c4cf..60dd4b423fa 100644
--- a/Modules/_testcapi/vectorcall.c
+++ b/Modules/_testcapi/vectorcall.c
@@ -98,7 +98,15 @@ _testcapi_pyobject_vectorcall_impl(PyObject *module, PyObject *func,
         PyErr_SetString(PyExc_TypeError, "kwnames must be None or a tuple");
         return NULL;
     }
-    return PyObject_Vectorcall(func, stack, nargs, kwnames);
+    PyObject *res;
+    // The CPython interpreter does not guarantee that vectorcalls are
+    // checked for recursion limit. It's thus up to the C extension themselves to check.
+    if (Py_EnterRecursiveCall("in _testcapi.pyobject_vectorcall")) {
+        return NULL;
+    }
+    res = PyObject_Vectorcall(func, stack, nargs, kwnames);
+    Py_LeaveRecursiveCall();
+    return res;
 }
 
 static PyObject *

32 remaining items
Ignyra commented on Apr 10, 2026
Ignyra
on Apr 10, 2026
More actions

Please give it a go at https://patch-diff.githubusercontent.com/raw/python/cpython/pull/148326.diff and let me know!

This worked for me as well, with and without --enable-shared :)
Reacted by Ken Jin, ppigazzini and Chris Eibl
Reacted by Ken Jin
Fidget-Spinner commented on Apr 10, 2026
Fidget-Spinner
on Apr 10, 2026
Member
More actions
FWIW, I can reproduce on current main of CPython (3.15a8+ too).
Reacted by ppigazzini, Ignyra and Victor Stinner
ppigazzini commented on Apr 10, 2026
ppigazzini
on Apr 10, 2026
Author
More actions
@Fidget-Spinner this seems related #145953
Reacted by Ken Jin

ppigazzini commented on Apr 10, 2026

ppigazzini

on Apr 10, 2026

Author

More actions

@Fidget-Spinner your feeling about clang22 inlining issues perhaps are correct:
https://discourse.llvm.org/t/approaches-to-the-inliner-blowing-up/89971
https://llvm.org/doxygen/InlineCost_8cpp.html

I managed to build vanilla 3.14.4 adding -mllvm -inline-threshold=150 -mllvm -inlinehint-threshold=250 using 2/3 of the default values.

$ CC="clang" CXX="clang++" LDFLAGS="${LDFLAGS:+$LDFLAGS }-fuse-ld=lld" PYTHON_CONFIGURE_OPTS="--enable-optimizations --with-lto" PYTHON_CFLAGS="-march=native -mtune=native -mllvm -inline-threshold=150 -mllvm -inlinehint-threshold=250" pyenv install 3.14.4
Downloading Python-3.14.4.tar.xz...
-> https://www.python.org/ftp/python/3.14.4/Python-3.14.4.tar.xz
Installing Python-3.14.4...
Installed Python-3.14.4 to /home/usr00/.pyenv/versions/3.14.4

$ pyenv shell 3.14.4
$ python -m test test_call
Using random seed: 1847535037
0:00:00 load avg: 4.59 Run 1 test sequentially in a single process
0:00:00 load avg: 4.59 [1/1] test_call
0:00:00 load avg: 4.59 [1/1] test_call passed

== Tests result: SUCCESS ==

1 test OK.

Total duration: 207 ms
Total tests: run=186
Total test files: run=1/1
Result: SUCCESS

$ python test_stack.py
stack memory per call: 2.2 kB

Reacted by Ken Jin

Fidget-Spinner
added a commit that references this issue on Apr 10, 2026
gh-148284: Block inlining of gigantic functions in ceval.c for clang …
e007631
miss-islington
added a commit that references this issue on Apr 10, 2026
pythongh-148284: Block inlining of gigantic functions in ceval.c for …
3b60a68
Fidget-Spinner commented on Apr 10, 2026
Fidget-Spinner
on Apr 10, 2026
Member
More actions
@ppigazzini you are right. Hopefully, we can fix that in the next version of 3.14.

Before I merge the 3.14 PR and close the issue, can you please check if https://patch-diff.githubusercontent.com/raw/python/cpython/pull/148349.diff works for you one final time?

@Ignyra Thank you for your help in testing this issue :)
Reacted by ppigazzini
Reacted by Ignyra

ppigazzini commented on Apr 10, 2026

ppigazzini

on Apr 10, 2026

Author

More actions

It looks good to me, stack memory per call: 2.2 kB that I already got with the -mllvm -inline-threshold=150 -mllvm -inlinehint-threshold=250 :)

$ CC="clang" CXX="clang++" LDFLAGS="${LDFLAGS:+$LDFLAGS }-fuse-ld=lld" PYTHON_CONFIGURE_OPTS="--enable-optimizations --with-lto" PYTHON_CFLAGS="-march=native -mtune=native" pyenv install --patch 3.14.4 < 148349.diff
Downloading Python-3.14.4.tar.xz...
-> https://www.python.org/ftp/python/3.14.4/Python-3.14.4.tar.xz
Installing Python-3.14.4...
patching file Doc/using/configure.rst
patching file Makefile.pre.in
patching file Misc/NEWS.d/next/Core_and_Builtins/2026-04-10-14-20-54.gh-issue-148284.HKs-S_.rst
patching file configure
patching file configure.ac
Installed Python-3.14.4 to /home/usr00/.pyenv/versions/3.14.4

$ python -m test test_call
Using random seed: 15611483
0:00:00 load avg: 0.02 Run 1 test sequentially in a single process
0:00:00 load avg: 0.02 [1/1] test_call
0:00:00 load avg: 0.02 [1/1] test_call passed

== Tests result: SUCCESS ==

1 test OK.

Total duration: 194 ms
Total tests: run=186
Total test files: run=1/1
Result: SUCCESS

$ python test_stack.py
stack memory per call: 2.2 kB

Fidget-Spinner
added a commit that references this issue on Apr 10, 2026
[3.14] gh-148284: Block inlining of gigantic functions in ceval.c for…
288cbac
Fidget-Spinner
removed
release-blocker
on Apr 10, 2026
Fidget-Spinner
marked Segmentation fault in test_call.py (c_recurse) on Linux with Clang (LTO+PGO) #145953 as a duplicate of this issue on Apr 10, 2026
Fidget-Spinner commented on Apr 10, 2026
Fidget-Spinner
on Apr 10, 2026
Member
More actions
Awesome. Thanks to both of you for submitting the report and testing. This was an extremely tricky bug to hunt down (properly the second strangest one I've seen this year).
Reacted by Victor Stinner, ppigazzini, Hugo van Kemenade and Ignyra
Fidget-Spinner
closed this as completed on Apr 10, 2026
github-project-automation
moved this from Todo to Done in Release and Deferred blockers 🚫on Apr 10, 2026
ljfp
added a commit that references this issue on Apr 25, 2026
pythongh-148284: Block inlining of gigantic functions in ceval.c for …
35a1785

Metadata

Assignees

No one assigned

Labels

Projects

Release and Deferred blockers 🚫
Status
Done
Show more project fields

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

python -m test -v test_call segfault with clang 22.1.3 build #148284

Description

Crash report

What happened?

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

Activity

Fidget-Spinner commented on Apr 9, 2026

ppigazzini commented on Apr 9, 2026

Fidget-Spinner commented on Apr 9, 2026

ppigazzini commented on Apr 9, 2026

Fidget-Spinner commented on Apr 9, 2026

Fidget-Spinner commented on Apr 9, 2026

ppigazzini commented on Apr 9, 2026

Fidget-Spinner commented on Apr 9, 2026

ppigazzini commented on Apr 9, 2026

ppigazzini commented on Apr 9, 2026

Fidget-Spinner commented on Apr 9, 2026

32 remaining items

Ignyra commented on Apr 10, 2026

Fidget-Spinner commented on Apr 10, 2026

ppigazzini commented on Apr 10, 2026

ppigazzini commented on Apr 10, 2026

Fidget-Spinner commented on Apr 10, 2026

ppigazzini commented on Apr 10, 2026

Fidget-Spinner commented on Apr 10, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`python -m test -v test_call` segfault with clang 22.1.3 build #148284