GeezerVison is showing github.com — [macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe? · Issue #75999 · python/cpython
[macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe? #75999DescriptionMirkoFriedenhagen mannequin
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2017-10-19.10:30:35.581>
labels = ['OS-mac', '3.10', '3.8', '3.9', 'type-crash']
title = '[macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?'
updated_at = <Date 2020-10-27.04:18:24.213>
user = 'https://bugs.python.org/MirkoFriedenhagen' bugs.python.org fields:
activity = <Date 2020-10-27.04:18:24.213>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['macOS']
creation = <Date 2017-10-19.10:30:35.581>
creator = 'Mirko Friedenhagen'
dependencies = []
files = ['47227']
hgrepos = []
issue_num = 31818
keywords = []
message_count = 14.0
messages = ['304614', '304615', '304616', '304617', '304621', '304635', '304648', '304649', '378947', '378966', '378968', '378977', '379021', '379106']
nosy_count = 4.0
nosy_names = ['barry', 'ronaldoussoren', 'ned.deily', 'Mirko Friedenhagen']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue31818'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10'] ActivityMirkoFriedenhagen commented on Oct 19, 2017 MirkoFriedenhagenmannequin MannequinAuthor More actions vstinner commented on Oct 19, 2017 Hi, can you please explain how to reproduce your issue?
According to the crash report, it seems like you are running Ansible on macOS and that the Python function _scproxy.get_proxies() was called.
get_proxies() calls CFPreferencesCopyAppValue() which calls indirectly performForkChildInitialize(). It seems like Ansible forked the process or something like that. Finally, performForkChildInitialize() calls _objc_fatal() which kills the process with abort().
The parent process is also Python ("Parent Process: python2.7 [4305]") which confirms that the application used fork().
See also:
- bpo-9405: Similar but old (2010) crash caused by SCDynamicStoreCopyProxies in a small Python application using multiprocessing and so using fork
- bpo-27126: "Apple-supplied libsqlite3 on OS X is not fork safe; can cause crashes"
- "fork() without exec() is dangerous in large programs" article by Evan Jones (2016-August-16): http://www.evanjones.ca/fork-is-dangerous.html -- this article mentions bpo-27126
Ned Deily's advice from bpo-9405: "A quick workaround is to make a [get_proxies()] call from the main process."
IMHO the safest fix is to not run any Python program after fork(). Using use subprocess to use fork() immmediately followed by exec(). It's not safe to execute code after fork(), many functions are not "fork safe". But I know that many applications don't care, since there is also a lot of functions which are fork safe... changed the title [-]macOS HighSierra final - Python Crash because of _scproxy[/-] [+][macOS] _scproxy.get_proxies() crash -- get_proxies() is not fork-safe?[/+] on Oct 19, 2017vstinner commented on Oct 19, 2017 Confirmation from Apple:
https://developer.apple.com/library/content/technotes/tn2083/_index.html#//apple_ref/doc/uid/DTS10003794-CH1-SUBSECTION52
"""
Many Mac OS X frameworks do not work reliably if you call fork but do not call exec. The only exception is the System framework and, even there, the POSIX standard places severe constraints on what you can do between a fork and an exec.
(...)
Listing 13 Core Foundation complaining about fork-without-exec
The process has forked and you cannot use this CoreFoundation \
functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_\
COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.
""" vstinner commented on Oct 19, 2017 ned-deily commented on Oct 19, 2017 Ronald is the expert on this but, from what I understand, I don't think there is any reason to spend time on trying to further analyze this. This issue has been around since day one of _scproxy and affects all versions of Python on macOS. There is nothing we can do to fix it, and, after all these years, it isn't likely that Apple is going to change the underlying framework. What we could do is: (1) better document the restriction; (2) find another way to access the system's network proxy configuration (not likely), or (3) change how we use the System Configuration framework, i.e. either don't call it at all or don't call it by default (but that seems like overfill for an edge case for which there already is a fairly simple workaround). Ronald, what do you think? vstinner commented on Oct 19, 2017 Another workaround is to call get_proxies() in a fresh subprocess, and use
a pipe to retrieve the result. ronaldoussoren commented on Oct 20, 2017 Calling get_proxies() in a subprocess would also work, although I'd then prefer to use a small daemon proces to avoid the startup cost of a new process for every call to _scproxy functions.
There is a conflict between two goals w.r.t. the macOS port:
-
Integrate nicely with the platform
-
Be like other unixy platforms
The former requires the use of Apple specific APIs, like those used in _scproxy, but those cause problems when using fork without calling exec.
The latter is technically an issue for all processing using threads on POSIX systems (see <http://pubs.opengroup.org/onlinepubs/009695399/functions/fork.html\>), AFAIK users get bitten by this more on macOS because Apple appears to use threading in their implementation (making processes multi-threaded without explicitly using threading in user code), and because Apple explicitly checks for the "fork without exec" case and crashes the child proces.
This can of course also be seen as a qualify of implementation issue on macOS, as in "Apple can't be bothered to do the work to support this use case" ;-/
Anyways: As Ned writes this is unlikely to change on Apple's side and we have to life with that.
There's three options going forward:
- Remove _scproxy
I'm -1 on that because I'm in favour of having good platform integration, Python's URL fetching APIs should transparently use the system proxy config configuration.
-
Document this problem and move on
-
Find a workaround (such as calling the APIs used by _scproxy in a clean supprocess).
The 3th option is probably the most useful in the long run, but requires someone willing to do the work. I'm in principle willing to do the work, but haven't had enough free time to work on CPython for quite a while now (which saddens me, but that's off topic). vstinner commented on Oct 20, 2017
- Find a workaround (such as calling the APIs used by _scproxy in a clean supprocess).
I dislike the idea of *always* spawning a child process, I prefer to leave it as it is, but add a recipe in the doc, or add a new helper function doing that.
Spawning a subprocess can have side effects as well, whereas the subprocess is only need if you call the function after forking which is not the most common pattern in Python. ronaldoussoren commented on Oct 19, 2020 I'm in favour of closing this issue.
If anything needs to be done its adding a warning to the documentation of os.fork() to the effect that doing substantial work in the child can be problematic on macOS. Maybe something like:
.. warning:: On macOS the child proces might crash if the parent
proces uses higher level system APIs before calling this function. This
includes, but is not limited to, using :mod:`urllib.request` in the parent
process. warsaw commented on Oct 19, 2020 I agree that we should close this issue, possibly with a documentation fix. I wrote a blog posting about my investigations:
https://wefearchange.org/2018/11/forkmacos.rst.html
I don't think there's really much Python itself can do, and developers will just have to know that fork-without-exec is pretty much unsafe on macOS. ned-deily commented on Oct 19, 2020 I wish there was a better solution but, as long as we want to continue to integrate with the system-provided proxy configuration, I guess we're kind of stuck. I'll try to expand Ronald's wording into a doc PR. ronaldoussoren commented on Oct 19, 2020 I'm in favour of keeping integration with system proxy settings, that's more user friendly.
Note that _scproxy is an example of triggering this using the standard library, the same may happen when using 3th-party libraries on PyPI if those happen to use a higher-level API from Apple. Even the system provided copy of sqlite is not safe (or at least wasn't, I haven't checked that lately).
I expect that similar problems might crop up on other platforms, especially when using threading. warsaw commented on Oct 19, 2020 My understanding is that this is specifically a problem with the Objective-C runtime that _scproxy.c accesses. The runtime is not thread safe and whereas in earlier versions of macOS, it silently failed, now macOS is explicitly aborting the process. ronaldoussoren commented on Oct 20, 2020
My understanding is that this is specifically a problem with the Objective-C
runtime that _scproxy.c accesses. The runtime is not thread safe and
whereas in earlier versions of macOS, it silently failed, now macOS is
explicitly aborting the process.
Note quite, the ObjC runtime is thread safe (and so are most high-level APIs) but a number of APIs use threads or process-level state that needs to be reset in a child proces and isn't. To phrase this in CPython API: Apple doesn't use the equivalent of PyOS_AfterFork_Child in a number of places where this would be needed to be able to use APIs in child processes. Instead of that recentish version of macOS just detect that the process-id changed and bail out.
This is a quality of implementation issue, and could also happen on other platforms (locks held across fork(), ...), especially when using threads.
"A process shall be created with a single thread. If a multi-threaded process calls fork(), the new process shall contain a replica of the calling thread and its entire address space, possibly including the states of mutexes and other resources. Consequently, to avoid errors, the child process may only execute async-signal-safe operations until such time as one of the exec functions is called."
<https://pubs.opengroup.org/onlinepubs/9699919799/\> ronaldoussoren commented on Nov 15, 2022 I propose closing this issue as there's nothing we can do here: _scproxy uses APIs that don't work in a child created with fork-without-exec. The only way to "fix" the crash is to drop the _scproxy extension, but that affects users that proxy servers (at least some business users, including myself at the time of writing this extension). warsaw commented on Nov 15, 2022
I propose closing this issue as there's nothing we can do here
I concur.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|