GeezerVison is showing github.com — test_interpreters Failing Intermittently Under Py_DISABLE_GIL · Issue #118731 · python/cpython
test_interpreters Failing Intermittently Under Py_DISABLE_GIL #118731DescriptionBug report
Bug description:
On one of my recent PRs, gh-118157, I was getting consistent failures on CI in test_interpreters. Different tests would fail, but the error was always the same:
(expand)
0:08:24 load avg: 4.18 Re-running 1 failed tests in verbose mode in subprocesses
0:08:24 load avg: 4.18 Run 1 test in parallel using 1 worker process (timeout: 10 min, worker timeout: 15 min)
0:08:26 load avg: 3.93 [1/1/1] test_interpreters failed (6 failures)
Re-running test_interpreters in verbose mode (matching: test_display_preserved_exception, test_gh_109793, test_sys_path_0, test_sys_path_0, test_sys_path_0, test_sys_path_0)
test_display_preserved_exception (test.test_interpreters.test_api.TestInterpreterExec.test_display_preserved_exception) ... FAIL
test_gh_109793 (test.test_interpreters.test_lifecycle.FinalizationTests.test_gh_109793) ... FAIL
test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) ...
### start subtest debug ###
# python3 script.py failed:
--- stdout ---
{
"main": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4",
"sub": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4"
}
------
--- stderr ---
Exception ignored in: <function Interpreter.__del__ at 0x2000119aed0>
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 157, in __del__
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 173, in _decref
TypeError: catching classes that do not inherit from BaseException is not allowed
Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
Python runtime state: finalizing (tstate=0x00005639aed8ea10)
------
### end subtest debug ###
test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 script.py] ... FAIL
### start subtest debug ###
# python3 -m script failed:
--- stdout ---
{
"main": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4",
"sub": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4"
}
------
--- stderr ---
Exception ignored in: <function Interpreter.__del__ at 0x2000117c0d0>
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 157, in __del__
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 173, in _decref
TypeError: catching classes that do not inherit from BaseException is not allowed
Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
Python runtime state: finalizing (tstate=0x0000564591b58a10)
------
### end subtest debug ###
test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 -m script] ... FAIL
### start subtest debug ###
# python3 -m pkg failed:
--- stdout ---
{
"main": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4",
"sub": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4"
}
------
--- stderr ---
Exception ignored in: <function Interpreter.__del__ at 0x2000118c0d0>
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 157, in __del__
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 173, in _decref
TypeError: catching classes that do not inherit from BaseException is not allowed
Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
Python runtime state: finalizing (tstate=0x000055aaa11eaa10)
------
### end subtest debug ###
test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 -m pkg] ... FAIL
### start subtest debug ###
# python3 -m pkg.script failed:
--- stdout ---
{
"main": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4",
"sub": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4"
}
------
--- stderr ---
Exception ignored in: <function Interpreter.__del__ at 0x2000118c0d0>
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 157, in __del__
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 173, in _decref
TypeError: catching classes that do not inherit from BaseException is not allowed
Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
Python runtime state: finalizing (tstate=0x00005637ca7cfa10)
------
### end subtest debug ###
test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 -m pkg.script] ... FAIL
======================================================================
FAIL: test_display_preserved_exception (test.test_interpreters.test_api.TestInterpreterExec.test_display_preserved_exception)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/__init__.py", line 2603, in wrapper
return func(*args, **kwargs)
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_api.py", line 764, in test_display_preserved_exception
self.assertEqual(stderr, dedent(f"""\
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
Traceback (most recent call last):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<18 lines>...
RuntimeError: uh-oh!
^^^^^^^^^^^^^^^^^^^^
"""))
^^^^^
AssertionError: 'Trac[778 chars]oh!\nException ignored in: <function Interpret[519 chars]\n\n' != 'Trac[778 chars]oh!\n'
Traceback (most recent call last):
File "/tmp/test_python_tk7ha1xh/tmpkqxiakqz/script.py", line 9, in <module>
interp.exec(script)
~~~~~~~~~~~^^^^^^^^
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 227, in exec
raise ExecutionFailed(excinfo)
test.support.interpreters.ExecutionFailed: RuntimeError: uh-oh!
Uncaught in the interpreter:
Traceback (most recent call last):
File "/tmp/test_python_tk7ha1xh/tmpkqxiakqz/script.py", line 6, in script
spam.eggs()
~~~~~~~~~^^
File "/tmp/test_python_tk7ha1xh/tmpkqxiakqz/spam.py", line 6, in eggs
ham()
~~~^^
File "/tmp/test_python_tk7ha1xh/tmpkqxiakqz/spam.py", line 3, in ham
raise RuntimeError('uh-oh!')
RuntimeError: uh-oh!
- Exception ignored in: <function Interpreter.__del__ at 0x2000119aed0>
- Traceback (most recent call last):
- File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 157, in __del__
- File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 173, in _decref
- TypeError: catching classes that do not inherit from BaseException is not allowed
- Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
- Python runtime state: finalizing (tstate=0x000055b06f0a1a10)
-
======================================================================
FAIL: test_gh_10[979](https://github.com/python/cpython/actions/runs/8978599353/job/24660038142#step:19:980)3 (test.test_interpreters.test_lifecycle.FinalizationTests.test_gh_109793)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 185, in test_gh_109793
self.assertEqual(proc.returncode, 1)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
AssertionError: -6 != 1
======================================================================
FAIL: test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 script.py]
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 29, in subTest
yield ctx
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 157, in test_sys_path_0
out = self.run_python(argv, cwd=cwd)
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 114, in run_python
self.assertEqual(proc.returncode, 0)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
AssertionError: -6 != 0
======================================================================
FAIL: test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 -m script]
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 29, in subTest
yield ctx
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 157, in test_sys_path_0
out = self.run_python(argv, cwd=cwd)
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 114, in run_python
self.assertEqual(proc.returncode, 0)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
AssertionError: -6 != 0
======================================================================
FAIL: test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 -m pkg]
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 29, in subTest
yield ctx
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 157, in test_sys_path_0
out = self.run_python(argv, cwd=cwd)
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 114, in run_python
self.assertEqual(proc.returncode, 0)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
AssertionError: -6 != 0
======================================================================
FAIL: test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 -m pkg.script]
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 29, in subTest
yield ctx
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 157, in test_sys_path_0
out = self.run_python(argv, cwd=cwd)
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 114, in run_python
self.assertEqual(proc.returncode, 0)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
AssertionError: -6 != 0
----------------------------------------------------------------------
Ran 3 tests in 1.720s
FAILED (failures=6)
test test_interpreters failed
1 test failed again:
test_interpreters
The error looks like this:
Exception ignored in: <function Interpreter.__del__ at 0x2000117c0d0>
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 157, in __del__
File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 173, in _decref
TypeError: catching classes that do not inherit from BaseException is not allowed
Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
Python runtime state: finalizing (tstate=0x0000564591b58a10)
I'm fairly sure that means one of the 3 calls in _interpreters._decref() is raising a bogus exception.
It isn't clear to me how gh-118157 would have caused this to happen. As a very temporary solution, I've skipped test_interpreters on free-threading builds.
CC @colesbury
CPython versions tested on:
CPython main branch
Operating systems tested on:
No response Activitycolesbury commented on May 7, 2024 I think what's happening is:
By the time Interpreter.__del__ is called, a lot of global variables including _interpreters have already been deleted from the module's dictionary so you get a bunch of weird errors, and the interpreter is not deleted. ericsnowcurrently commented on May 7, 2024 That makes sense. You have any thoughts already on how to resolve this? colesbury commented on May 7, 2024 I'm not sure. We could call the finalizers on all GC-enabled objects during shutdown in the free-threaded build (before we start clearing module's dictionaries). I think that makes logical sense: everything is going away, why not call their finalizers? That seems to fix test_interpreters, but a few other test cases would need tweaking. kumaraditya303 commented on Jul 14, 2025 On current main, test_interpreters passes under free-threading as deferred reference counting is implemented now, the tests can be enabled back now.
== CPython 3.15.0a0 free-threading build (heads/main-dirty:75e2c5dd346, Jul 14 2025, 17:20:43) [Clang 17.0.0 (clang-1700.0.13.5)]
== macOS-15.5-arm64-arm-64bit-Mach-O little-endian
== Python build: free_threading debug
== cwd: /Users/kumaraditya/work/cpython/build/test_python_worker_53278æ
== CPU count: 10
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests
Using random seed: 2517149026
0:00:00 load avg: 8.09 Run 1 test sequentially in a single process
0:00:00 load avg: 8.09 [1/1] test_interpreters
0:00:07 load avg: 7.60 [1/1] test_interpreters passed
== Tests result: SUCCESS ==
1 test OK.
Total duration: 7.1 sec
Total tests: run=165 skipped=8
Total test files: run=1/1
Result: SUCCESS
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|