GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-117783: Immortalize objects that use deferred reference counting by colesbury · Pull Request #118112 · python/cpython

Conversation

@colesbury

@colesbury colesbury commented Apr 19, 2024 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Deferred reference counting is not fully implemented yet. As a temporary measure, we immortalize objects that would use deferred reference counting to avoid multi-threaded scaling bottlenecks.

This is only performed in the free-threaded build once the first non-main thread is started. Additionally, some tests, including refleak tests, suppress this behavior.

  • Issue: Temporarily immortalize objects that use deferred reference counting #117783

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@colesbury
pythongh-117783: Immortalize objects that use deferred reference coun…
30a4133 
…ting

Deferred reference counting is not fully implemented yet. As a temporary
measure, we immortalize objects that would use deferred reference
counting to avoid multi-threaded scaling bottlenecks.

This is only performed in the free-threaded build once the first
non-main thread is started. Additionally, some tests, including refleak
tests, suppress this behavior.
@colesbury colesbury added the topic-free-threading label Apr 19, 2024
@bedevere-app bedevere-app Bot mentioned this pull request Apr 19, 2024
Temporarily immortalize objects that use deferred reference counting #117783
Closed
@colesbury colesbury added the skip news label Apr 19, 2024
@colesbury
Fix set_immortalize_deferred
0cab82c
@colesbury colesbury mentioned this pull request Apr 19, 2024
Disable GIL by default in free-threaded build #116749
Closed
@colesbury colesbury added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Apr 19, 2024
@bedevere-bot

bedevere-bot commented Apr 19, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @colesbury for commit 0cab82c 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Apr 19, 2024
@colesbury
colesbury marked this pull request as ready for review April 19, 2024 21:37
@colesbury
colesbury requested review from ericsnowcurrently and rhettinger as code owners April 19, 2024 21:37
@bedevere-app bedevere-app Bot added the awaiting core review label Apr 19, 2024
@colesbury
colesbury requested a review from DinoV April 19, 2024 21:37
@rhettinger
rhettinger removed their request for review April 21, 2024 19:11
DinoV
DinoV reviewed Apr 22, 2024
View reviewed changes
Comment thread Include/internal/pycore_gc.h Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Python/pylifecycle.c Outdated
// run much later than typical leading to attribute errors due to
// partially cleared modules. To avoid this, we copy the module dict
// if it was immortalized.
PyObject *copy = PyDict_Copy(mod->md_dict);

@DinoV DinoV Apr 22, 2024

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to clear the original dict here too? Otherwise it seems like the immortal dict will just hang around and keep everything alive anyway?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@colesbury colesbury Apr 26, 2024

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My intention was to keep the fields alive, but maybe that's not the best way to address this.

Basically, test_concurrent_futures.test_interpreter_shutdown was failing because a weakref was cleared later than usual, after the module dictionary was partially cleared. I was concerned that the problem might not be limited to that test, but I don't really have evidence for that one way or the other.

I'll get rid of this swapping and make the weakref callback more robust.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
colesbury added 4 commits April 25, 2024 16:10
@colesbury
Rename field to enable_on_thread_created
c6216ab
@colesbury
Remove module dict swapping
588255e
@colesbury
Adjust formatting
ea81a4c
@colesbury
Remove whitespace change
0c9a155
@colesbury
colesbury requested a review from DinoV April 26, 2024 17:19
@colesbury colesbury mentioned this pull request Apr 29, 2024
Dramatic slowdown with random.random on free-threading build. #118392
Open
DinoV
DinoV approved these changes Apr 29, 2024
View reviewed changes

@DinoV DinoV left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Apr 29, 2024
@colesbury
colesbury merged commit 7ccacb2 into python:main Apr 29, 2024
@colesbury
colesbury deleted the gh-117783-immortalize branch April 29, 2024 18:36
@bedevere-app bedevere-app Bot removed the awaiting merge label Apr 29, 2024
@vasiliyk vasiliyk mentioned this pull request Sep 24, 2026
concurrent/futures/process.py: mp.util.debug is called, but mp.util is never imported #99368
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@DinoV DinoV DinoV approved these changes
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@colesbury @bedevere-bot @DinoV
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.