Description

ZeroIntensity

opened on Oct 26, 2024

Issue body actions

Crash report

What happened?

(Found while investigating a fix for #125920.)

Subinterpreters that are running in a different thread can still be running upon the finalization of the main interpreter, depending on what the thread was doing. Take the following script:

from threading import Thread
import _interpreters


def silly():
    interp = _interpreters.create()
    _interpreters.run_string(interp, "import time; time.sleep(100)")


if __name__ == "__main__":
    Thread(target=silly).start()

On 3.13 and 3.14, killing the process with CTRL+C results in an assertion failure:

python: Python/pylifecycle.c:2480: finalize_subinterpreters: Assertion `!_PyInterpreterState_IsRunningMain(interp)' failed.

However, if the thread is daemon, then the interpreter fully segfaults:

from threading import Thread
import _interpreters


def silly():
    interp = _interpreters.create()
    _interpreters.run_string(interp, "import time; time.sleep(100)")


if __name__ == "__main__":
    Thread(target=silly, daemon=True).start()

I'm going to investigate possible fixes.

cc @ericsnowcurrently

CPython versions tested on:

3.13, CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

No response

Linked PRs

• gh-126016: Force crossinterpreter thread states to get cleaned up #126026
• gh-126016: Remove bad assertion in PyThreadState_Clear #139158
• gh-126016: Fix flaky test by allowing the SIGINT return code #139219

Activity

1. ZeroIntensity

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

   topic-subinterpreters

   3.13only security fixesonly security fixes

   3.14bugs and security fixesbugs and security fixes

   on Oct 26, 2024
2. ZeroIntensity

   self-assigned this

   on Oct 26, 2024
3. github-project-automation

   added this to Subinterpreterson Oct 26, 2024
4. github-project-automation

   moved this to Todo in Subinterpreterson Oct 26, 2024
5. rruuaanng commented on Oct 27, 2024

   

   rruuaanng

   on Oct 27, 2024

   Contributor

   More actions

   Using .join() will not trigger an assertion error. For the reason, please see the PR submitted by Eric and me.

   >>> def silly():
   ...     interp = _interpreters.create()
   ...     _interpreters.run_string(interp, "import time; time.sleep(100)")
   ...
   >>> if __name__ == "__main__":
   ...     Thread(target=silly).start()
   ...
   >>>
   >>> Thread(target=silly).start()
   >>> Thread(target=silly).start()
   >>> quit
   
   
   ^CTraceback (most recent call last):
     File "/root/cpython/Lib/threading.py", line 1540, in _shutdown
       _thread_shutdown()
   KeyboardInterrupt:
   python: Python/pylifecycle.c:2463: finalize_subinterpreters: Assertion `!_PyInterpreterState_IsRunningMain(interp)' failed.
   

   >>> def silly():
   ...     interp = _interpreters.create()
   ...     _interpreters.run_string(interp, "import time; time.sleep(100)")
   ...
   >>> t = Thread(target=silly)
   >>> t.start()
   >>> t.join()
   ^CTraceback (most recent call last):
     File "<python-input-14>", line 1, in <module>
       t.join()
       ~~~~~~^^
     File "/root/cpython/Lib/threading.py", line 1092, in join
       self._handle.join(timeout)
       ~~~~~~~~~~~~~~~~~^^^^^^^^^
   KeyboardInterrupt
   
6. ZeroIntensity commented on Oct 27, 2024

   

   ZeroIntensity

   on Oct 27, 2024

   MemberAuthor

   More actions

   Yes, I know. The issue is that CTRL+C'ing causes the thread to not finish joining, meaning that the thread state is still marked as running. I'm working on a fix to properly clean up thread states before the interpreter shuts down using pending calls, but daemon threads will probably have to get addressed elsewhere (there's a data race, I think).
7. rruuaanng commented on Oct 27, 2024

   

   rruuaanng

   on Oct 27, 2024

   Contributor

   More actions

   If you want, you can insert a signal catcher to catch SIGINT and do error handling on it. And in the error handling iterate over the list of threads and handle them

   Reacted by RUANG (James Roy)
8. ZeroIntensity commented on Oct 27, 2024

   

   ZeroIntensity

   on Oct 27, 2024

   MemberAuthor

   More actions

   That would result in a thread safety nightmare if the thread was still running :)

   Reacted by RUANG (James Roy)
9. ZeroIntensity commented on Oct 27, 2024

   

   ZeroIntensity

   on Oct 27, 2024

   MemberAuthor

   More actions

   #126026 addresses non-daemon threads. It's a bit hacky, though, because I have to manually mess with the status of the thread state.
10. ZeroIntensity commented on Jan 11, 2025

    

    ZeroIntensity

    on Jan 11, 2025

    MemberAuthor

    More actions

    I think #128640 will fix this as well.
11. ZeroIntensity

    added a commit that references this issue on Sep 19, 2025

    gh-126016: Remove bad assertion in `PyThreadState_Clear` (GH-139158)

    9243a4b
12. ZeroIntensity commented on Sep 19, 2025

    

    ZeroIntensity

    on Sep 19, 2025

    MemberAuthor

    More actions

    Closing this for now. Ideally we'll get to a point where we can add the assertion back, but that can be tracked elsewhere.
13. ZeroIntensity

    closed this as completedon Sep 19, 2025
14. github-project-automation

    moved this from Todo to Done in Subinterpreterson Sep 19, 2025
15. encukou

    added a commit that references this issue on Sep 23, 2025

    gh-126016: Fix flaky test by allowing the SIGINT return code (GH-139219)

    6b4e3fe

Issue actions