GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-40521: Per-interpreter interned strings by vstinner · Pull Request #20085 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented May 14, 2020 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Make the Unicode dictionary of interned strings per-interpreter to
make it compatible with subinterpreters.

https://bugs.python.org/issue40521

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label May 14, 2020
@bedevere-bot bedevere-bot added the awaiting core review label May 14, 2020
@vstinner

vstinner commented May 14, 2020

Copy link
Copy Markdown
Member Author

This change requires https://bugs.python.org/issue39465 to be fixed, and also to make the type method cache compatible with subinterpreters.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner vstinner mentioned this pull request Jun 7, 2020
bpo-40881: Fix unicode_release_interned() #20699
Merged
@vstinner

vstinner commented Dec 23, 2020

Copy link
Copy Markdown
Member Author

This change requires https://bugs.python.org/issue39465 to be fixed, and also to make the type method cache compatible with subinterpreters.

With this PR, tests using subinterpreters like test_ast.test_subinterpreter() does crash since the method cache is built by default, whereas it shares interned strings between multiple interpreters. MCACHE must not be defined to test this PR, or the method cache must be fixed for subinterpreters.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner

vstinner commented Dec 23, 2020

Copy link
Copy Markdown
Member Author

I rebased my PR on master. It is still a draft: PR #20058 must be merged first, and then the method cache must be fixed for subinterpreter, before we can consider to merge this PR.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
This was referenced Dec 23, 2020
bpo-39465: Fix _PyUnicode_FromId() for subinterpreters #20058
Merged
bpo-42745: Make the type cache per-interpreter #23947
Merged
@vstinner
vstinner marked this pull request as ready for review December 26, 2020 00:48
@vstinner
vstinner requested a review from markshannon as a code owner December 26, 2020 00:48
@vstinner vstinner changed the title [WIP] bpo-40521: Per-interpreter interned strings bpo-40521: Per-interpreter interned strings Dec 26, 2020
@vstinner

vstinner commented Dec 26, 2020

Copy link
Copy Markdown
Member Author

And I pushed a second fix for test_repl:

_PyUnicode_ClearInterned() now uses PyDict_Next() to no longer allocate memory, to ensure that the interned dictionary is cleared.

It's more an enhancement than a fix. Previously, the code also failed. It's just that previously, we didn't check if interned is NULL at exit, whereas my PR adds such assertion.
shihai1991 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner
bpo-40521: Per-interpreter interned strings
8cb4246 
Make the Unicode dictionary of interned strings compatible with
subinterpreters.

Remove the INTERN_NAME_STRINGS macro in typeobject.c: names are
always now interned (even if EXPERIMENTAL_ISOLATED_SUBINTERPRETERS
macro is defined).

_PyUnicode_ClearInterned() now uses PyDict_Next() to no longer
allocate memory, to ensure that the interned dictionary is cleared.
@vstinner
vstinner merged commit ea25180 into python:master Dec 26, 2020
@bedevere-bot bedevere-bot removed the awaiting core review label Dec 26, 2020
@vstinner
vstinner deleted the unicode_interned_subinterp branch December 26, 2020 01:58
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 13, 2021
@vstinner @adorilson
bpo-40521: Per-interpreter interned strings (pythonGH-20085)
68dcee2 
Make the Unicode dictionary of interned strings compatible with
subinterpreters.

Remove the INTERN_NAME_STRINGS macro in typeobject.c: names are
always now interned (even if EXPERIMENTAL_ISOLATED_SUBINTERPRETERS
macro is defined).

_PyUnicode_ClearInterned() now uses PyDict_Next() to no longer
allocate memory, to ensure that the interned dictionary is cleared.
vstinner added a commit that referenced this pull request Jan 6, 2022
@vstinner
bpo-46006: Revert "bpo-40521: Per-interpreter interned strings (GH-20085
35d6540 
)" (GH-30422)

This reverts commit ea25180.

Keep "assert(interned == NULL);" in _PyUnicode_Fini(), but only for
the main interpreter.

Keep _PyUnicode_ClearInterned() changes avoiding the creation of a
temporary Python list object.
This was referenced Jan 6, 2022
bpo-46006: Fix _PyUnicode_EqualToASCIIId() for interned strings #30123
Closed
bpo-46006: Move the interned strings and identifiers to _PyRuntimeState. #30131
Closed
vstinner added a commit that referenced this pull request Jan 6, 2022
@vstinner
[3.10] bpo-46006: Revert "bpo-40521: Per-interpreter interned strings (…
72c260c 
…GH-20085)" (GH-30422) (GH-30425)

This reverts commit ea25180.

Keep "assert(interned == NULL);" in _PyUnicode_Fini(), but only for
the main interpreter.

Keep _PyUnicode_ClearInterned() changes avoiding the creation of a
temporary Python list object.

Leave the PyInterpreterState structure unchanged to keep the ABI
backward compatibility with Python 3.10.0: rename the "interned"
member to "unused_interned".

(cherry picked from commit 35d6540)
ericsnowcurrently added a commit to ericsnowcurrently/cpython that referenced this pull request Jan 13, 2022
@ericsnowcurrently
Un-revert "bpo-40521: Per-interpreter interned strings (pythonGH-20085)…
a1ef98c 
…".

This reverts commit 35d6540.
@vstinner vstinner mentioned this pull request Apr 10, 2022
[subinterpreters] Make free lists and unicode caches per-interpreter #84701
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@markshannon markshannon Awaiting requested review from markshannon

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@vstinner @the-knights-who-say-ni @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.