Description

tacaswell

opened on Feb 9, 2024

Issue body actions

Bug report

Bug description:

In #104826 (backported to 3.12 in #105277) to fix #104690 disallowed creating new threads during interpreter finalization. This poses a problem on windows because the Popen._communicate uses a thread (see matplotlib/matplotlib#27437 (comment) for discussion).

In Matplotlib we are holding onto a latex process and triggering this via a weakref finalize, but a minimal reproducer is:

from subprocess import Popen, PIPE
import atexit

# anything that holds stdin and stdout should work
proc = Popen(['powershell'], stdin=PIPE, stdout=PIPE)

def cleanup():
    print('hi bob')
    proc.kill()
    proc.communicate()

atexit.register(cleanup)

xref matplotlib/matplotlib#27437

CPython versions tested on:

3.12

Operating systems tested on:

Windows

Reacted by Michael Droettboom and Gregory P. Smith

Activity

1. tacaswell

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Feb 9, 2024
2. serhiy-storchaka

   added this to Multiprocessing issueson Feb 9, 2024
3. serhiy-storchaka

   added

   3.12only security fixesonly security fixes

   3.13only security fixesonly security fixes

   on Feb 9, 2024
4. serhiy-storchaka commented on Feb 9, 2024

   

   serhiy-storchaka

   on Feb 9, 2024

   Member

   More actions

   cc @chgnrdv, @gpshead
5. AlexWaygood

   added

   OS-windows

   on Feb 10, 2024
6. mdboom

   added

   topic-multiprocessing

   on Feb 12, 2024
7. zooba commented on Feb 12, 2024

   

   zooba

   on Feb 12, 2024

   Member

   More actions

   (cc @ericsnowcurrently)

   One possibility is to transform the Windows code here into a purely native thread, rather than a Python thread, but that is likely to break people who are passing file objects to stdout=/stderr= besides our constants. That's probably unlikely, but we also shouldn't break it.

   What might be better, and why I pinged Eric, is to find a finalization ordering where we can run atexit callbacks and then require all threads to be exited. That may not help with the similar GC issues, but it seems like atexit ought to be okay with running a bit earlier.
8. ericsnowcurrently

   added

   topic-subinterpreters

   on Feb 27, 2024
9. github-project-automation

   added this to Subinterpreterson Feb 27, 2024
10. github-project-automation

    moved this to Todo in Subinterpreterson Feb 27, 2024
11. ericsnowcurrently commented on Mar 18, 2024

    

    ericsnowcurrently

    on Mar 18, 2024

    Member

    More actions

    See gh-116677.
12. gpshead

    removed this from Subinterpreterson Mar 19, 2024
13. gpshead

    moved this to In Progress in Multiprocessing issueson Mar 19, 2024
14. gpshead

    self-assigned this

    on Mar 19, 2024
15. gpshead

    removed

    topic-subinterpreters

    topic-multiprocessing

    on Mar 19, 2024
16. gpshead

    removed this from Multiprocessing issueson Mar 19, 2024
17. gpshead commented on Mar 19, 2024

    

    gpshead

    on Mar 19, 2024

    Member

    More actions

    This subprocess on Windows atexit or from a non-daemon thread use case is a good one. I'm going to mark this a duplicate of our more general issue #113964 on the topic where we've got a PR and likely a second one in the works to address this.

    Reacted by Thomas A Caswell, Steve Dower and Felix Kolbe
18. gpshead

    closed this as not plannedon Mar 19, 2024

Issue actions