GeezerVison

The code here is ready, but I'll keep it as a draft until gh-116749 and gh-116329 are done, since this doesn't do anything when the GIL is enabled by default. I changed the default locally to test this as it works as intended.

I think it does make sense to land this now after all. The code won't function as intended until the GIL is disabled by default, but it will be nice to have the module slot available while I work on gh-116738, so I can tag modules as they're ready.

This is structured such that there shouldn't be any behavioral differences until the GIL is disabled by default, except when a --disable-gil build is run with -v, when it will print a single line for every non-free-threading-safe module that's loaded.

I'll take a look on Monday.

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@ericsnowcurrently I don't think there should be any code changes as a result of our big sub-thread. Please let me know if that's wrong!

Before this is ready for another review, I need to make some other changes. It looks like the consensus in Discord is that we should enable the GIL when calling a module init function, because we don't know before the call if it's a single-phase init module that needs the GIL. Then if it's a multi-phase init module that provides Py_GIL_NOT_USED, we can disable the GIL and continue, otherwise we leave the GIL on for the rest of the interpreter's lifetime. I'll make an issue soon to nail down how we handle single-phase init modules.

The only nontrivial (but not too complicated) part of this will be adding a function to disable the GIL safely. It should also move the GIL-related logic up out of PyModule_FromDefAndSpec2(), which will be a nice side-effect.

I don't think there should be any code changes as a result of our big sub-thread.

Yeah, I think you're right.

I have made the requested changes; please review again.

This should be ready for review again. It grew in scope a bit, and now includes support for single-phase init modules (gh-117526) (some details are in the commit message). With the infrastructure needed to properly support multi-phase init modules, supporting single-phase init modules is just a few extra lines, but I can pull it out into a separate PR if anyone wants.

One specific thing I'd like input on is the timing of the _PyImport_CheckGILForModule() call in PyModule_FromDefAndSpec2(): I put it before module creation to disable the GIL as soon as possible for modules that don't need the GIL. The downside of this is that if the module does need the GIL and a subsequent part of the import process fails, the GIL will remain enabled even though we didn't actually load the module. Maybe this should be rare enough that it's not worth worrying about.

!buildbot nogil

🤖 New build scheduled with the buildbot fleet by @colesbury for commit 77d1652 🤖

The command will test the builders whose names match following regular expression: nogil

The builders matched are:

• x86-64 MacOS Intel ASAN NoGIL PR
• AMD64 Ubuntu NoGIL Refleaks PR
• ARM64 MacOS M1 NoGIL PR
• AMD64 Windows Server 2022 NoGIL PR
• AMD64 Ubuntu NoGIL PR
• ARM64 MacOS M1 Refleaks NoGIL PR
• x86-64 MacOS Intel NoGIL PR

I found a few more modules that I missed during some testing, and will push a commit to add those shortly.

!buildbot nogil

🤖 New build scheduled with the buildbot fleet by @swtaarrs for commit d1fe0cc 🤖

The command will test the builders whose names match following regular expression: nogil

The builders matched are:

• x86-64 MacOS Intel ASAN NoGIL PR
• AMD64 Ubuntu NoGIL Refleaks PR
• ARM64 MacOS M1 NoGIL PR
• AMD64 Windows Server 2022 NoGIL PR
• AMD64 Ubuntu NoGIL PR
• ARM64 MacOS M1 Refleaks NoGIL PR
• x86-64 MacOS Intel NoGIL PR