GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-84559: multiprocessing: detect if forkserver cannot work due to missing hmac-sha256 by gpshead · Pull Request #127467 · python/cpython

Conversation

@gpshead

@gpshead gpshead commented Dec 1, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Default to the spawn start method in that scenario.

Mostly I want to see if this meaningfully lets the odd configuration FIPS buildbot make further progress or not.

This PR is now stacked on top of #127492.

  • Craft a unittest for this scenario. Probably by mocking out hmac.new and raising ValueError in a child process before the multiprocessing.context import.
  • merge the parent PR gh-127298: Refactor test_hashlib for better usedforsecurity & openssl fips mode env support. #127492, merge main & unstack.
  • Issue: multiprocessing's default posix start method of 'fork' is broken: change to `'forkserver' || 'spawn' #84559

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@gpshead
pythongh-84559: multiprocessing detect of forkserver cannot work due …
7474d73
…to missing hmac-sha256

Default to the spawn start method in that scenario.
@bedevere-app bedevere-app Bot mentioned this pull request Dec 1, 2024
multiprocessing's default posix start method of 'fork' is broken: change to `'forkserver' || 'spawn' #84559
Closed
@gpshead

gpshead commented Dec 1, 2024

Copy link
Copy Markdown
Member Author

!buildbot FIPS

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Dec 1, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @gpshead for commit 7474d73 🤖

The command will test the builders whose names match following regular expression: FIPS

The builders matched are:

  • AMD64 CentOS9 FIPS Only Blake2 Builtin Hash PR
  • AMD64 RHEL8 FIPS No Builtin Hashes PR
  • AMD64 RHEL8 FIPS Only Blake2 Builtin Hash PR
  • AMD64 CentOS9 FIPS No Builtin Hashes PR
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gpshead gpshead changed the title gh-84559: multiprocessing detect of forkserver cannot work due to missing hmac-sha256 gh-84559: multiprocessing: detect if forkserver cannot work due to missing hmac-sha256 Dec 1, 2024
gpshead added 5 commits December 1, 2024 21:42
@gpshead
Refactor test_hashlib for better usedforsecurity & openssl fips mode …
59d9a85
…environment support.
@gpshead
More hashlib robustness in strange "FIPS mode" environments.
bd46651
@gpshead
pythongh-84559: multiprocessing detect of forkserver cannot work due …
7a5ab6c
…to missing hmac-sha256

Default to the spawn start method in that scenario.
@gpshead
More multiprocessing & concurrent.futures disables when forkserver ca…
adfd6c7
…nnot work
@gpshead
Merge branch 'hashlib-fips-oddmode' into multiprocessing-forkserver-n…
740cb40
…o-hmac-disable
@gpshead

gpshead commented Dec 2, 2024

Copy link
Copy Markdown
Member Author

!buildbot FIPS

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Dec 2, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @gpshead for commit 740cb40 🤖

The command will test the builders whose names match following regular expression: FIPS

The builders matched are:

  • AMD64 CentOS9 FIPS Only Blake2 Builtin Hash PR
  • AMD64 RHEL8 FIPS No Builtin Hashes PR
  • AMD64 RHEL8 FIPS Only Blake2 Builtin Hash PR
  • AMD64 CentOS9 FIPS No Builtin Hashes PR
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Dec 2, 2024
gh-127298: When in FIPS mode ensure builtin hashes check for usedforsecurity=False #127301
Open
gh-127298: Refactor test_hashlib for better usedforsecurity & openssl fips mode env support. #127492
Closed
@xnox

xnox commented Dec 10, 2024

Copy link
Copy Markdown

On hosts without the hmac-sha256 algorithm available (rare)

Is this artificially created host, or something realistic? Without sha256 there would not be any realistic TLS working. And HMAC construction should also be always available.

Would it be a host with tampered/missing openssl & no built-in fallbacks? But then like nothing else is working either (hashlib, ssl)

I'm trying to understand if this is really more "support multiprocessing without openssl & without fallback crypto algorithms" as in no cryptography whats-so-ever, rather than "missing sha256" which I would have thought is a broken, and hopefully impossible, situation.

hexagonrecursion reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gpshead

gpshead commented Dec 11, 2024

Copy link
Copy Markdown
Member Author

This is a with a dumb "openssl fips mode" config that blocks all ciphers, running a python also configured not to have a fallback builtin sha256 module. configure --without-builtin-hashlib-hashes, and set the OPENSSL_CONF= environment variable to point to a file like https://github.com/python/cpython/blob/905ba7f06c12f7ef9985ccaa2bf24229f759a2e1/Lib/test/hashlibdata/openssl.cnf and you can repro this kind of setup.

this is primarily a "get the test suite to work in this situation" as we have a buildbot running with that type of config to better understand odd config failure modes.

i don't expect anyone to actually run their system with such a borked config. I'm not spending a lot of time on this, but I do consider it a "nice to have" for the redistributors who make strange "fips mode" envrionments for customers. they should wind up with less pain this way. Even though it isn't how any CPython binary release we ship would ever be configured. ie, it should unblock #127298.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gpshead

gpshead commented Dec 11, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

IIRC, anything involving a (non-fork start method?) multiprocessing Connection will fail without the relevant hmac auth algorithm. But not all multiprocessing features rely on those. I do not intend to document what does and doesn't work in this situation - we don't make any guarantees here.

xnox reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@xnox

xnox commented Dec 11, 2024

Copy link
Copy Markdown

This is a with a dumb "openssl fips mode" config that blocks all ciphers, running a python also configured not to have a fallback builtin sha256 module. configure --without-builtin-hashlib-hashes, and set the OPENSSL_CONF= environment variable to point to a file like https://github.com/python/cpython/blob/905ba7f06c12f7ef9985ccaa2bf24229f759a2e1/Lib/test/hashlibdata/openssl.cnf and you can repro this kind of setup.

this is primarily a "get the test suite to work in this situation" as we have a buildbot running with that type of config to better understand odd config failure modes.

i don't expect anyone to actually run their system with such a borked config. I'm not spending a lot of time on this, but I do consider it a "nice to have" for the redistributors who make strange "fips mode" envrionments for customers. they should wind up with less pain this way. Even though it isn't how any CPython binary release we ship would ever be configured. ie, it should unblock #127298.

Ack, whilst SHA-2 is still valid for post-2035 as per https://csrc.nist.gov/pubs/ir/8547/ipd it could be the case that people move on to just SHA-3 & SHAKE, at which point SHA-2 hmac may become irrelevant in favour of KMAC-256. And these changes will make things limp along.

For pre-2035 horizon, the relevant changes that we do need from this pr is the test/ changes and the multiprocessing is nice-to-have (or borderline theoretical / making it easier to reason about by using absolutely crypto-less config AKA zero-knowledge as to what future holds).

So can this be marked not-draft? what is missing to land the testsuite changes of this PR, or even this whole PR to unblock #127298?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@xnox

xnox commented Dec 11, 2024

Copy link
Copy Markdown

Oh I see this PR is in draft, because it is based on #127492 and that one needs to land first, which indeed has the required perquisites. Ok I am catching up to all the things now.

gpshead reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@xnox

xnox commented Apr 28, 2025

Copy link
Copy Markdown

@gpshead hi, my PRs had comments that you are taking over with these changes instead, but they now seem to have stalled. Are you still actively working on this and related PRs?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions

github-actions Bot commented Apr 20, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 20, 2026
@ambv

ambv commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

This PR is attached to gh-84559, which is already completed: the fix for that issue landed through other changes, and this PR's approach was not taken or has since become obsolete. Closing to cull the number of open pull requests. Do not reopen this pull request without getting a go-ahead from a triager or core developer.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ambv ambv closed this Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@picnixz picnixz Awaiting requested review from picnixz picnixz will be requested when the pull request is marked ready for review picnixz is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

stale Stale PR or inactive for long period of time.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@gpshead @bedevere-bot @xnox @ambv
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.