GeezerVison is showing github.com — GitHub free threading CI failing with SIGTERM · Issue #150114 · python/cpython
GitHub free threading CI failing with SIGTERM #150114DescriptionBug report
Randomly GitHub Ubuntu (free-threading) / build and test (ubuntu-24.04) CI is failing with a SIGTERM (exit code 134).
My best guess is that the VM is running out of memory due to a test using too much memory
0:08:51 load avg: 14.72 [272/505] test.test_asyncio.test_pep492 passed -- running (2): test.test_multiprocessing_spawn.test_processes (3 min 23 sec), test_free_threading (6 min)
make: *** [Makefile:2434: ci] Terminated
Error: Process completed with exit code 143.
Examples:
Linked PRs
Activityvstinner commented on May 21, 2026 I close this issue since test_free_threading memory usage was optimized. Reopen the issue if it comes back on the CI. colesbury commented on May 21, 2026 ContributorAuthor More actions colesbury commented on May 21, 2026 ContributorAuthor More actions Another hypothesis:
When CPU is this saturated, the runner agent process itself struggles to get scheduled. The agent has to send periodic heartbeats to GitHub's control plane. If it misses too many (because the kernel keeps preempting
it for python workers), the control plane decides the runner is unhealthy and terminates the job — which manifests exactly as "The runner has received a shutdown signal", with no kernel OOM in dmesg, no memory
exhaustion, and no warning to the test process.
vstinner commented on May 22, 2026 I created PR gh-150255 to track the memory usage in regrtest. It should help to understand if the "exit code 143" error is related to the memory usage or not.
On my PR gh-150248, "Ubuntu (free-threading) / build and test (ubuntu-24.04)" failed twice with:
...
0:15:41 load avg: 6.82 [503/505] test_multiprocessing_main_handling passed (34.6 sec) -- running (2): test_profiling (6 min 58 sec), test_free_threading (1 min 11 sec)
0:16:11 load avg: 5.71 running (2): test_profiling (7 min 28 sec), test_free_threading (1 min 41 sec)
0:16:41 load avg: 4.96 running (2): test_profiling (7 min 58 sec), test_free_threading (2 min 11 sec)
0:17:11 load avg: 4.58 running (2): test_profiling (8 min 28 sec), test_free_threading (2 min 41 sec)
0:17:41 load avg: 4.41 running (2): test_profiling (8 min 58 sec), test_free_threading (3 min 11 sec)
0:18:11 load avg: 4.28 running (2): test_profiling (9 min 29 sec), test_free_threading (3 min 41 sec)
make: *** [Makefile:2434: ci] Terminated
Error: Process completed with exit code 143.
I don't think that the issue here is the CPU usage, since the CPU usage was way higher before: system load of 31.25 at 8 min versus a system load of 4.28 at 18 min when the error occurred.
0:08:03 load avg: 31.25 [238/505] test_cmd_line_script passed (48.9 sec) -- running (5): test_capi (2 min 14 sec), test.test_multiprocessing_fork.test_manager (42.0 sec), test_set (42.6 sec), test.test_concurrent_futures.test_process_pool (2 min 48 sec), test.test_multiprocessing_forkserver.test_processes (2 min 11 sec)
vstinner commented on May 22, 2026 Reacted by Cody Maloney and Chris Eibl vstinner commented on May 22, 2026
Only the main branch is affected: test_free_threading.test_pickle only exists in the main branch.
Oh. The root cause is the change #150025 which was backported to 3.14 and 3.15 branches. So these branches are affected as well, even if they don't have the new test. vstinner commented on May 25, 2026 I modified regrtest (Python test runner) to log the memory usage. It's now supported on Linux, Windows, FreeBSD and macOS.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|