GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-100227: Make the Global Interned Dict Safe for Isolated Interpreters by ericsnowcurrently · Pull Request #102925 · python/cpython

Conversation

@ericsnowcurrently

@ericsnowcurrently ericsnowcurrently commented Mar 22, 2023 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This is effectively two changes. The first (the bulk of the change) is where we add _Py_AddToGlobalDict() (and _PyRuntime.cached_objects.main_tstate, etc.). The second (much smaller) change is where we update PyUnicode_InternInPlace() to use _Py_AddToGlobalDict() instead of calling PyDict_SetDefault() directly.

Basically, _Py_AddToGlobalDict() is a wrapper around PyDict_SetDefault() that should be used whenever we need to add a value to a runtime-global dict object (in the few cases where we are leaving the container global rather than moving it to PyInterpreterState, e.g. the interned strings dict). _Py_AddToGlobalDict() does all the necessary work to make sure the target global dict is shared safely between isolated interpreters. This is especially important as we move the obmalloc state to each interpreter (gh-101660), as well as, potentially, the GIL (PEP 684).

(This is an alternative to #102339, where we keep the interned dict global instead of making it per-interpreter.)

  • Issue: Improve Interpreter Isolation #100227

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
ericsnowcurrently added 17 commits March 22, 2023 11:59
@ericsnowcurrently
Factor out add_threadstate().
546b934
@ericsnowcurrently
Add _PyThreadState_InitDetached().
0bcd136
@ericsnowcurrently
Add _PyThreadState_ClearDetached().
35d5310
@ericsnowcurrently
Add _PyRuntime.cached_objects.main_tstate.
cba9e34
@ericsnowcurrently
Add _Py_AcquireGlobalObjectsState() and _Py_ReleaseGlobalObjectsState().
3b1bb8b
@ericsnowcurrently
Add _Py_AddToGlobalDict().
eb42aa1
@ericsnowcurrently
Drop _Py_AcquireGlobalObjectsState() and _Py_ReleaseGlobalObjectsStat…
4e9da2d
…e().
@ericsnowcurrently
Add acquire_global_objects_lock() and release_global_objects_lock().
6216207
@ericsnowcurrently
Add some TODO comments.
3c007c0
@ericsnowcurrently
Factor out store_interned().
7d95514
@ericsnowcurrently
Store a thread state to use just for interned strings.
5c20b84
@ericsnowcurrently
Always use the main interpreter when possibly resizing the interned d…
a3ae02a
…ict.
@ericsnowcurrently
Use _PyRuntime.cached_objects.main_tstate instead.
d5fbc37
@ericsnowcurrently
Add _PyThreadState_IsBound() and _PyThreadState_Unbind().
459325f
@ericsnowcurrently
Make sure the one-off tstate is bound before using it.
4f25244
@ericsnowcurrently
Use _Py_AcquireGlobalObjectsState() in store_interned().
e68535a
@ericsnowcurrently
Use _Py_AddToGlobalDict().
22753b3
@ericsnowcurrently ericsnowcurrently added the skip news label Mar 22, 2023
@ericsnowcurrently
ericsnowcurrently requested a review from kumaraditya303 March 22, 2023 18:17
@bedevere-bot bedevere-bot mentioned this pull request Mar 22, 2023
Improve Interpreter Isolation #100227
Closed
@bedevere-bot bedevere-bot added the awaiting core review label Mar 22, 2023
This was referenced Mar 22, 2023
gh-100227: Make the Global Interned Dict Safe for Isolated Interpreters #102858
Closed
gh-84436: Make Interned Strings Immortal #102887
Closed
@ericsnowcurrently
ericsnowcurrently marked this pull request as ready for review March 22, 2023 20:21
@ericsnowcurrently ericsnowcurrently added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Mar 22, 2023
@bedevere-bot

bedevere-bot commented Mar 22, 2023

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @ericsnowcurrently for commit 22753b3 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Mar 22, 2023
This was referenced Mar 22, 2023
gh-100227: Make the Global PyModuleDef Cache Safe for Isolated Interpreters #102938
Closed
gh-100227: Move the Dict of Interned Strings to PyInterpreterState #102339
Merged
@ericsnowcurrently
ericsnowcurrently merged commit 87be8d9 into python:main Mar 23, 2023
@bedevere-bot bedevere-bot removed the awaiting core review label Mar 23, 2023
@ericsnowcurrently
ericsnowcurrently deleted the interned-dict-owned-by-main-interpreter branch March 23, 2023 00:30
Fidget-Spinner pushed a commit to Fidget-Spinner/cpython that referenced this pull request Mar 27, 2023
@ericsnowcurrently @Fidget-Spinner
pythongh-100227: Make the Global Interned Dict Safe for Isolated Inte…
828e887
…rpreters (pythongh-102925)

This is effectively two changes.  The first (the bulk of the change) is where we add _Py_AddToGlobalDict() (and _PyRuntime.cached_objects.main_tstate, etc.).  The second (much smaller) change is where we update PyUnicode_InternInPlace() to use _Py_AddToGlobalDict() instead of calling PyDict_SetDefault() directly.

Basically, _Py_AddToGlobalDict() is a wrapper around PyDict_SetDefault() that should be used whenever we need to add a value to a runtime-global dict object (in the few cases where we are leaving the container global rather than moving it to PyInterpreterState, e.g. the interned strings dict).  _Py_AddToGlobalDict() does all the necessary work to make sure the target global dict is shared safely between isolated interpreters.  This is especially important as we move the obmalloc state to each interpreter (pythongh-101660), as well as, potentially, the GIL (PEP 684).

python#100227
ericsnowcurrently added a commit to ericsnowcurrently/cpython that referenced this pull request Mar 27, 2023
@ericsnowcurrently
Revert "pythongh-100227: Make the Global Interned Dict Safe for Isola…
2e8773b
…ted Interpreters (pythongh-102925)"

This reverts commit 87be8d9.
ericsnowcurrently added a commit that referenced this pull request Mar 27, 2023
@ericsnowcurrently
gh-100227: Revert gh-102925 "gh-100227: Make the Global Interned Dict…
89e67ad
… Safe for Isolated Interpreters" (gh-103063)

This reverts commit 87be8d9.

This approach to keeping the interned strings safe is turning out to be too complex for my taste (due to obmalloc isolation). For now I'm going with the simpler solution, making the dict per-interpreter. We can revisit that later if we want a sharing solution.
@ericsnowcurrently ericsnowcurrently mentioned this pull request Mar 28, 2023
gh-100227: Make the Global PyModuleDef Cache Safe for Isolated Interpreters #103084
Merged
warsaw pushed a commit to warsaw/cpython that referenced this pull request Apr 11, 2023
@ericsnowcurrently @warsaw
pythongh-100227: Make the Global Interned Dict Safe for Isolated Inte…
6cffe84
…rpreters (pythongh-102925)

This is effectively two changes.  The first (the bulk of the change) is where we add _Py_AddToGlobalDict() (and _PyRuntime.cached_objects.main_tstate, etc.).  The second (much smaller) change is where we update PyUnicode_InternInPlace() to use _Py_AddToGlobalDict() instead of calling PyDict_SetDefault() directly.

Basically, _Py_AddToGlobalDict() is a wrapper around PyDict_SetDefault() that should be used whenever we need to add a value to a runtime-global dict object (in the few cases where we are leaving the container global rather than moving it to PyInterpreterState, e.g. the interned strings dict).  _Py_AddToGlobalDict() does all the necessary work to make sure the target global dict is shared safely between isolated interpreters.  This is especially important as we move the obmalloc state to each interpreter (pythongh-101660), as well as, potentially, the GIL (PEP 684).

python#100227
warsaw pushed a commit to warsaw/cpython that referenced this pull request Apr 11, 2023
@ericsnowcurrently @warsaw
pythongh-100227: Revert pythongh-102925 "pythongh-100227: Make the Gl…
cf66c45
…obal Interned Dict Safe for Isolated Interpreters" (pythongh-103063)

This reverts commit 87be8d9.

This approach to keeping the interned strings safe is turning out to be too complex for my taste (due to obmalloc isolation). For now I'm going with the simpler solution, making the dict per-interpreter. We can revisit that later if we want a sharing solution.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kumaraditya303 kumaraditya303 Awaiting requested review from kumaraditya303

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@ericsnowcurrently @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.