Description

dgrisby

opened on Aug 16, 2023

Issue body actions

Checklist

• I am confident this is a bug in CPython, not a bug in a third-party project
• I have searched the CPython issue tracker,
  and am confident this bug has not been reported before

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.13.0a0 (main, Jul 29 2023, 00:25:05) [GCC 13.1.1 20230614 (Red Hat 13.1.1-4)]

A clear and concise description of the bug:

Threads that call PyEval_RestoreThread while the runtime is finalizing are terminated. The documented way to avoid this is to call _Py_IsFinalizing() to check. https://docs.python.org/3.13/c-api/init.html#c.PyEval_RestoreThread

_Py_IsFinalizing() has been removed from Python 3.13. Please provide a public API function that has the equivalent behavior, or some other way for a non-Python thread to safely attempt to call into Python code when the interpreter may be finalizing.

Rather than the calling code checking for finalization, an alternative solution would be to provide a variant of PyEval_RestoreThread that returns success or failure, so the calling code can know that it was unable to acquire the GIL, and react accordingly.

Linked PRs

• gh-108014: Add Py_IsFinalizing() function #108032

Activity

1. dgrisby

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Aug 16, 2023
2. vstinner commented on Aug 16, 2023

   

   vstinner

   on Aug 16, 2023

   Member

   More actions

   Since sys.is_finalizing() was added to Python 3.5, IMO it's fine to add a public C API function for that: https://docs.python.org/dev/library/sys.html#sys.is_finalizing

   cc @ericsnowcurrently

   Reacted by Erlend E. Aasland
3. Eclips4

   added

   topic-C-API

   on Aug 16, 2023
4. vstinner

   added a commit that references this issue on Aug 16, 2023

   pythongh-108014: Add Py_IsFinalizing() function

   e03b437
5. ericsnowcurrently commented on Aug 16, 2023

   

   ericsnowcurrently

   on Aug 16, 2023

   Member

   More actions

   FYI, there is some work to hang remaining threads rather than killing them: gh-105805. I expect that would land in 3.13.
   Presumably a Py_IsFinalizing() would still be helpful though.
6. erlend-aasland commented on Aug 17, 2023

   

   erlend-aasland

   on Aug 17, 2023

   Contributor

   More actions

   I think it would make sense to also make _Py_IsInterpreterFinalizing public:

   cpython/Python/pylifecycle.c

   Lines 2181 to 2191 in 1344cfa

   	int
   	_Py_IsInterpreterFinalizing(PyInterpreterState *interp)
   	{
   	/* We check the runtime first since, in a daemon thread,
   	interp might be dangling pointer. */
   	PyThreadState *finalizing = _PyRuntimeState_GetFinalizing(&_PyRuntime);
   	if (finalizing == NULL) {
   	finalizing = _PyInterpreterState_GetFinalizing(interp);
   	}
   	return finalizing != NULL;
   	}

   Reacted by Eric Snow
7. vstinner

   added a commit that references this issue on Aug 18, 2023

   gh-108014: Add Py_IsFinalizing() function (#108032)

   3ff5ef2
8. vstinner commented on Aug 18, 2023

   

   vstinner

   on Aug 18, 2023

   Member

   More actions

   Function added by commit 3ff5ef2.
9. vstinner

   closed this as completedon Aug 18, 2023
10. vstinner commented on Aug 18, 2023

    

    vstinner

    on Aug 18, 2023

    Member

    More actions

    Function added to pythoncapi-compat by commit python/pythoncapi-compat@1bb4e16.
11. vstinner commented on Sep 25, 2023

    

    vstinner

    on Sep 25, 2023

    Member

    More actions

    I proposed adding PythonFinalizationError exception in PR gh-109809.
12. vstinner commented on Oct 5, 2023

    

    vstinner

    on Oct 5, 2023

    Member

    More actions

    Follow-up: @wjakob asks to add this new function to the limited C API version 3.13.

    Reacted by Erlend E. Aasland and Dustin Spicuzza

Issue actions