GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-116510: Fix a Crash Due to Shared Immortal Interned Strings by ericsnowcurrently · Pull Request #124865 · python/cpython

Conversation

@ericsnowcurrently

@ericsnowcurrently ericsnowcurrently commented Oct 1, 2024 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Fix a crash caused by immortal interned strings being shared between
sub-interpreters that use basic single-phase init. In that case, the string
can be used by an interpreter that outlives the interpreter that created and
interned it. For interpreters that share obmalloc state, also share the
interned dict with the main interpreter.

This is an un-revert of gh-124646 that addresses the Py_TRACE_REFS failures identified by gh-124785.

The 3.12 backport will be slightly different and will need to be done manually.

  • Issue: CPython 3.12 embedded in WeeChat causes segfault on subsequent calls to Py_EndInterpreter #116510

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
ericsnowcurrently added 2 commits October 1, 2024 12:24
@ericsnowcurrently
Revert "pythongh-124785: Revert "pythongh-116510: Fix crash due to sh…
884f398
…ared immortal interned strings (pythongh-124646)" (pythongh-124807)"

This reverts commit 7bdfabe.
@ericsnowcurrently
Move non-isolated immortal objects to the right ref chain.
623fb87
@ericsnowcurrently ericsnowcurrently added needs backport to 3.12 only security fixes needs backport to 3.13 only security fixes labels Oct 1, 2024
@ericsnowcurrently
ericsnowcurrently requested a review from nascheme October 1, 2024 18:36
@bedevere-app bedevere-app Bot mentioned this pull request Oct 1, 2024
CPython 3.12 embedded in WeeChat causes segfault on subsequent calls to Py_EndInterpreter #116510
Closed
@bedevere-app bedevere-app Bot added the awaiting core review label Oct 1, 2024
@ericsnowcurrently ericsnowcurrently mentioned this pull request Oct 1, 2024
[TraceRefs build] Tests fail: _PyRefchain_Remove: Assertion `value == REFCHAIN_VALUE' failed #124785
Closed
@ericsnowcurrently

ericsnowcurrently commented Oct 1, 2024

Copy link
Copy Markdown
Member Author

!buildbot AMD64 Arch Linux TraceRefs

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Oct 1, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @ericsnowcurrently for commit 623fb87 🤖

The command will test the builders whose names match following regular expression: AMD64 Arch Linux TraceRefs

The builders matched are:

  • AMD64 Arch Linux TraceRefs PR
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nascheme nascheme mentioned this pull request Oct 1, 2024
gh-124785: re-work fix so tracerefs test passes #124808
Closed
nascheme
nascheme reviewed Oct 1, 2024
View reviewed changes
Comment thread Objects/object.c

#ifdef Py_TRACE_REFS
void
_Py_NormalizeImmortalReference(PyObject *op)

@nascheme nascheme Oct 1, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having some comment here would be helpful. Otherwise this function looks quite mysterious.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@ericsnowcurrently ericsnowcurrently Oct 1, 2024

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

nascheme and gpshead reacted with thumbs up emoji
All reactions
  • 👍 2 reactions
@nascheme

nascheme commented Oct 1, 2024

Copy link
Copy Markdown
Member

LGTM other than adding a bit more comments.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ericsnowcurrently
Add some comments.
fcc6a06
@ericsnowcurrently

ericsnowcurrently commented Oct 1, 2024

Copy link
Copy Markdown
Member Author

The buildbot passed.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nascheme

nascheme commented Oct 8, 2024

Copy link
Copy Markdown
Member

Any reason not to merge this now?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ericsnowcurrently

ericsnowcurrently commented Oct 8, 2024

Copy link
Copy Markdown
Member Author

I can merge it first thing tomorrow if no one objects.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ericsnowcurrently
ericsnowcurrently merged commit f2cb399 into python:main Oct 9, 2024
@miss-islington-app

miss-islington-app Bot commented Oct 9, 2024

Copy link
Copy Markdown

Thanks @ericsnowcurrently for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting core review label Oct 9, 2024
@ericsnowcurrently
ericsnowcurrently deleted the fix-tracerefs branch October 9, 2024 17:32
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 9, 2024
@ericsnowcurrently @miss-islington
pythongh-116510: Fix a Crash Due to Shared Immortal Interned Strings (p…
a5b6574
…ythongh-124865)

Fix a crash caused by immortal interned strings being shared between
sub-interpreters that use basic single-phase init. In that case, the string
can be used by an interpreter that outlives the interpreter that created and
interned it. For interpreters that share obmalloc state, also share the
interned dict with the main interpreter.

This is an un-revert of pythongh-124646 that then addresses the Py_TRACE_REFS
failures identified by pythongh-124785.
(cherry picked from commit f2cb399)

Co-authored-by: Eric Snow <[email protected]>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 9, 2024
@ericsnowcurrently @miss-islington
pythongh-116510: Fix a Crash Due to Shared Immortal Interned Strings (p…
762cb25
…ythongh-124865)

Fix a crash caused by immortal interned strings being shared between
sub-interpreters that use basic single-phase init. In that case, the string
can be used by an interpreter that outlives the interpreter that created and
interned it. For interpreters that share obmalloc state, also share the
interned dict with the main interpreter.

This is an un-revert of pythongh-124646 that then addresses the Py_TRACE_REFS
failures identified by pythongh-124785.
(cherry picked from commit f2cb399)

Co-authored-by: Eric Snow <[email protected]>
@bedevere-app

bedevere-app Bot commented Oct 9, 2024

Copy link
Copy Markdown

GH-125204 is a backport of this pull request to the 3.13 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 only security fixes label Oct 9, 2024
@bedevere-app

bedevere-app Bot commented Oct 9, 2024

Copy link
Copy Markdown

GH-125205 is a backport of this pull request to the 3.12 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.12 only security fixes label Oct 9, 2024
@encukou

encukou commented Oct 10, 2024

Copy link
Copy Markdown
Member

Due to either this change or #125199, the AMD64 Arch Linux TraceRefs 3.x buildbot is failing: https://buildbot.python.org/#/builders/484/builds/6022/steps/6/logs/stdio

ericsnowcurrently reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@encukou encukou mentioned this pull request Oct 10, 2024
gh-125196: Use PyUnicodeWriter in symtable.c #125199
Merged
@vstinner

vstinner commented Oct 10, 2024

Copy link
Copy Markdown
Member

Due to either this change or #124865, the AMD64 Arch Linux TraceRefs 3.x buildbot is failing: https://buildbot.python.org/#/builders/484/builds/6022/steps/6/logs/stdio

I checked with git bisect and the regression was introduced by commit f2cb399:

commit f2cb39947093feda3ff85b8dc820922cc5e5f954
Author: Eric Snow <[email protected]>
Date:   Wed Oct 9 11:32:16 2024 -0600

    gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (gh-124865)
    
    Fix a crash caused by immortal interned strings being shared between
    sub-interpreters that use basic single-phase init. In that case, the string
    can be used by an interpreter that outlives the interpreter that created and
    interned it. For interpreters that share obmalloc state, also share the
    interned dict with the main interpreter.
    
    This is an un-revert of gh-124646 that then addresses the Py_TRACE_REFS
    failures identified by gh-124785.
encukou reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Oct 10, 2024
[3.13] gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (gh-124865) #125204
Merged
[3.12] gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (gh-124865) #125205
Merged
@ericsnowcurrently

ericsnowcurrently commented Oct 10, 2024

Copy link
Copy Markdown
Member Author

I'll take a look.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Oct 10, 2024
test_audit_subinterpreter crashes with tracerefs enabled #125286
Closed
Memory leak on executables embedded with 3.13 #113055
Closed
encukou pushed a commit that referenced this pull request Nov 12, 2024
@miss-islington @ericsnowcurrently
[3.13] gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (…
02cd3ce
…gh-124865) (gh-125709) (GH-125204)

* gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (gh-124865)

Fix a crash caused by immortal interned strings being shared between
sub-interpreters that use basic single-phase init. In that case, the string
can be used by an interpreter that outlives the interpreter that created and
interned it. For interpreters that share obmalloc state, also share the
interned dict with the main interpreter.

This is an un-revert of gh-124646 that then addresses the Py_TRACE_REFS
failures identified by gh-124785.
(cherry picked from commit f2cb399)

Co-authored-by: Eric Snow <[email protected]>

* [3.13] gh-125286: Share the Main Refchain With Legacy Interpreters (gh-125709)

They used to be shared, before 3.12.  Returning to sharing them resolves a failure on Py_TRACE_REFS builds.

---------

Co-authored-by: Eric Snow <[email protected]>
ericsnowcurrently added a commit that referenced this pull request Dec 3, 2024
@miss-islington @ericsnowcurrently
[3.12] gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (…
49da170
…gh-125205)

Fix a crash caused by immortal interned strings being shared between
sub-interpreters that use basic single-phase init. In that case, the string
can be used by an interpreter that outlives the interpreter that created and
interned it. For interpreters that share obmalloc state, also share the
interned dict with the main interpreter.

This is an un-revert of gh-124646 that then addresses the Py_TRACE_REFS
failures identified by gh-124785 (i.e. backporting gh-125709 too).

(cherry picked from commit f2cb399, AKA gh-124865)

Co-authored-by: Eric Snow <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nascheme nascheme nascheme left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@ericsnowcurrently @bedevere-bot @nascheme @encukou @vstinner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.