GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-98627: Add an Optional Check for Extension Module Subinterpreter Compatibility by ericsnowcurrently · Pull Request #99040 · python/cpython

Conversation

@ericsnowcurrently

@ericsnowcurrently ericsnowcurrently commented Nov 3, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Enforcing (optionally) the restriction set by PEP 489 makes sense. Furthermore, this sets the stage for a potential restriction related to a per-interpreter GIL.

This change includes the following:

  • add tests for extension module subinterpreter compatibility
  • add _PyInterpreterConfig.check_multi_interp_extensions
  • add Py_RTFLAGS_MULTI_INTERP_EXTENSIONS
  • add _PyImport_CheckSubinterpIncompatibleExtensionAllowed()
  • fail iff the module does not implement multi-phase init and the current interpreter is configured to check
  • Issue: Add a Check for Extensions that Support Running in Subinterpreters #98627

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

vasily-v-ryabov reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@ericsnowcurrently
ericsnowcurrently requested a review from encukou November 3, 2022 02:40
@ericsnowcurrently
ericsnowcurrently requested review from brettcannon, ncoghlan and warsaw as code owners November 3, 2022 02:40
@bedevere-bot bedevere-bot mentioned this pull request Nov 3, 2022
Add a Check for Extensions that Support Running in Subinterpreters #98627
Open
@bedevere-bot bedevere-bot added the awaiting core review label Nov 3, 2022
@ericsnowcurrently
ericsnowcurrently marked this pull request as draft November 3, 2022 02:41
@ericsnowcurrently
ericsnowcurrently removed request for brettcannon, ncoghlan and warsaw November 3, 2022 02:41
ericsnowcurrently added 10 commits January 11, 2023 10:35
@ericsnowcurrently
Add tests for extension module subinterpreter compatibility.
eb9d241
@ericsnowcurrently
Add _PyInterpreterConfig.check_multi_interp_extensions and Py_RTFLAGS…
49b4895
…_MULTI_INTERP_EXTENSIONS.
@ericsnowcurrently
Add _PyImport_CheckSubinterpIncompatibleExtensionAllowed().
ed505a6
@ericsnowcurrently
Raise ImportError in subinterpreters for incompatible single-phase in…
64e1dc8
…it extensions.
@ericsnowcurrently
Add a NEWS entry.
72ab9b6
@ericsnowcurrently
Add PyInterpreterState.override_multi_interp_extensions_check.
9c24b34
@ericsnowcurrently
Add check_multi_interp_extensions().
3c084eb
@ericsnowcurrently
Add _imp._override_multi_interp_extensions_check().
a3d3a65
@ericsnowcurrently
Add test.support.import_helper.multi_interp_extensions_check().
ad3fe36
@ericsnowcurrently
Add a test.
1defec3
@ericsnowcurrently
ericsnowcurrently force-pushed the interpreter-multi-interp-extensions-check branch from 93ea024 to 1defec3 Compare January 11, 2023 17:36
@erlend-aasland

erlend-aasland commented Jan 11, 2023

Copy link
Copy Markdown
Contributor

FYI, converting _testinternalcapi.c to multi-phase init resolves four of the test failures.

test_configured_settings (test.test_capi.test_misc.SubinterpreterTest.test_configured_settings)
The config with which an interpreter is created corresponds ... Traceback (most recent call last):
  File "<string>", line 2, in <module>
ImportError: module _testinternalcapi does not support loading in subinterpreters
[...]

👇🏻

test_configured_settings (test.test_capi.test_misc.SubinterpreterTest.test_configured_settings)
The config with which an interpreter is created corresponds ... ok
[...]
ericsnowcurrently reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ericsnowcurrently added 8 commits January 12, 2023 13:45
@ericsnowcurrently
Merge branch 'main' into HEAD
99f3371
@ericsnowcurrently
Fix a typo.
3c3ed2b
@ericsnowcurrently
Add some extra diagnostic info.
de6c791
@ericsnowcurrently
Clarify various names (e.g. data keys) in the test.
af114f2
@ericsnowcurrently
Allow long test output.
282e6d3
@ericsnowcurrently
Do not show the noop values unless different.
3cb8645
@ericsnowcurrently
Add comments to the expected values.
81abbfb
@ericsnowcurrently
Tweak the subtest labels.
db5d35a
ericsnowcurrently added 5 commits January 12, 2023 15:24
@ericsnowcurrently
Fix the expected results.
e0c55ad
@ericsnowcurrently
Add a test just for how the setting is used.
3b2dd6d
@ericsnowcurrently
Revert "Add a test just for how the setting is used."
d648a7b
This reverts commit 3b2dd6d.
@ericsnowcurrently
Add a test for the various settings and overrides for a singlephase e…
dc8d877
…xtension.
@ericsnowcurrently
Fix check_config.py.
5fba674
@ericsnowcurrently ericsnowcurrently mentioned this pull request Feb 6, 2023
gh-59956: Partial Fix for GILState API Compatibility with Subinterpreters #101431
Merged
ericsnowcurrently added a commit that referenced this pull request Feb 6, 2023
@ericsnowcurrently
gh-59956: Partial Fix for GILState API Compatibility with Subinterpre…
132b3f8
…ters (gh-101431)

The GILState API (PEP 311) implementation from 2003 made the assumption that only one thread state would ever be used for any given OS thread, explicitly disregarding the case of subinterpreters.  However, PyThreadState_Swap() still facilitated switching between subinterpreters, meaning the "current" thread state (holding the GIL), and the GILState thread state could end up out of sync, causing problems (including crashes).

This change addresses the issue by keeping the two in sync in PyThreadState_Swap().  I verified the fix against gh-99040.

Note that the other GILState-subinterpreter incompatibility (with autoInterpreterState) is not resolved here.

#59956
@ericsnowcurrently
Merge branch 'main' into interpreter-multi-interp-extensions-check
35d322d
@ericsnowcurrently
ericsnowcurrently marked this pull request as ready for review February 6, 2023 22:28
ericsnowcurrently added 2 commits February 15, 2023 16:23
@ericsnowcurrently
Merge branch 'main' into interpreter-multi-interp-extensions-check
ddf01fb
@ericsnowcurrently
Merge branch 'main' into interpreter-multi-interp-extensions-check
ee2cd3c
@ericsnowcurrently
ericsnowcurrently merged commit 89ac665 into python:main Feb 16, 2023
@bedevere-bot bedevere-bot removed the awaiting core review label Feb 16, 2023
@ericsnowcurrently
ericsnowcurrently deleted the interpreter-multi-interp-extensions-check branch February 16, 2023 01:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@encukou encukou Awaiting requested review from encukou

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@ericsnowcurrently @erlend-aasland @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.