GeezerVison is showing github.com — multiprocessing's default posix start method of `'fork'` is broken: change to ``'forkserver' || 'spawn'` · Issue #84559 · python/cpython
multiprocessing's default posix start method of 'fork' is broken: change to `'forkserver' || 'spawn' #84559Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2020-04-24.18:22:23.389>
labels = ['3.8', 'type-bug', '3.7', '3.9']
title = "multiprocessing's default start method of fork()-without-exec() is broken"
updated_at = <Date 2022-02-11.16:13:53.872>
user = 'https://bugs.python.org/itamarst' bugs.python.org fields:
activity = <Date 2022-02-11.16:13:53.872>
actor = 'mgorny'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2020-04-24.18:22:23.389>
creator = 'itamarst'
dependencies = []
files = []
hgrepos = []
issue_num = 40379
keywords = []
message_count = 11.0
messages = ['367210', '367211', '368173', '380478', '392358', '392501', '392503', '392506', '392507', '392508', '413081']
nosy_count = 8.0
nosy_names = ['pitrou', 'mgorny', 'Julian', 'wim.glenn', 'itamarst', 'davin', 'itamarst2', 'aduncan']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40379'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9'] Linked PRs
Reacted by Hash, Hafidh Rendyanto, Gregory P. Smith, Giangpro89, heimalne, Jonah George, Will Thompson, Philipp Moritz, Aidan and Ningú Activityitamarst commented on Apr 24, 2020 MannequinAuthor More actions itamarst commented on Apr 24, 2020 MannequinAuthor More actions Looks like as of 3.8 this only impacts Linux/non-macOS-POSIX, so I'll amend the above to say this will also make it consistent with macOS. Reacted by jakirkham, Diony Rosa and Ningú itamarst commented on May 5, 2020 MannequinAuthor More actions Just got an email from someone for whom switching to "spawn" fixed a problem. Earlier this week someone tweeted about this fixing things. This keeps hitting people in the real world. itamarst commented on Nov 6, 2020 MannequinAuthor More actions aduncan commented on Apr 29, 2021 I just ran into and fixed (thanks to itamarst's blog post) a problem likely related to this.
Multiprocessing workers performing work and sending a logging message back with success/fail info. I had a few intermittent deadlocks that became a recurring problem when I sped up the process that skipped tasks which had previously completed (I think this shortened the time between forking and attempting to send messages causing the third process to deadlock). After changing that it deadlocked *every time*.
Switching to "spawn" at the top of the main function has fixed it. pitrou commented on Apr 30, 2021 The problem with changing the default is that this will break any application that depends on passing non-picklable data to the child process (in addition to the potentially unexpected performance impact).
The docs already contain a significant elaboration on the matter, but feel free to submit a PR that would make the various caveats more explicit:
https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods itamarst commented on Apr 30, 2021 MannequinAuthor More actions This change was made on macOS at some point, so why not Linux? "spawn" is already the default on macOS and Windows. pitrou commented on Apr 30, 2021 The macOS change was required before "fork" simply ceased to work.
Windows has always used "spawn", because no other method can be implemented on Windows. itamarst commented on Apr 30, 2021 MannequinAuthor More actions Given people's general experience, I would not say that "fork" works on Linux either. More like "99% of the time it works, 1% it randomly breaks in mysterious way". pitrou commented on Apr 30, 2021 Agreed, but again, changing will break some applications.
We could switch to forkserver, but we should have a transition period where a FutureWarning will be displayed if people didn't explicitly set a start method. mgorny commented on Feb 11, 2022 After updating PyPy3 to use Python 3.9's stdlib, we hit very bad hangs because of this — literally compiling a single file with "parallel" compileall could hang. In the end, we had to revert the change in how Python 3.9 starts workers because otherwise multiprocessing would be impossible to use:
https://foss.heptapod.net/pypy/pypy/-/commit/c594b6c48a48386e8ac1f3f52d4b82f9c3e34784
This is a very bad default and what's even worse is that it often causes deadlocks that are hard to reproduce or debug. Furthermore, since "fork" is the default, people are unintentionally relying on its support for passing non-pickleable projects and are creating non-portable code. The code often becomes complex and hard to change before they discover the problem.
Before we managed to figure out how to workaround the deadlocks in PyPy3, we were experimenting with switching the default to "spawn". Unfortunately, we've hit multiple projects that didn't work with this method, precisely because of pickling problems. Furthermore, they were surprised to learn that their code wouldn't work on macOS (in the end, many people perceive Python as a language for writing portable software).
Finally, back in 2018 I've made one of my projects do parallel work using multiprocessing. It gave its users great speedup but for some it caused deadlocks that I couldn't reproduce nor debug. In the end, I had to revert it. Now that I've learned about this problem, I'm wondering if this wasn't precisely because of "fork" method. tasleson commented on Feb 4, 2026 @asottile
I doubt it's more than a fraction of a percent of python users or pypi packages -- anything that people actually use has already been updated or is abandoned
I ran into this yesterday when code that has been running reliably since 2015 broke after upgrading to Python 3.14.
Calling the impact “a fraction of a percent” misses the point at Python’s scale. Even small percentages represent a large amount of deployed, working code, much of which is intentionally unchanged because it is correct.
Equating infrequent updates with lack of use conflates maintenance with usage. The success of any programming language depends on expectations of long-term stability, and changes like this undermine that.
This kind of breakage also discourages writing new Python code, knowing it will have to be revisited later for no good reason. locked as resolved and limited conversation to collaborators on Feb 4, 2026
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|