GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-36854: Move GC runtime state from _PyRuntimeState to PyInterpreterState. by ericsnowcurrently · Pull Request #13219 · python/cpython

Conversation

@ericsnowcurrently

@ericsnowcurrently ericsnowcurrently commented May 9, 2019 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Note: as part of this PR, I've made sure that Python threads (created by the threading module) are marked as "deleted" earlier in finalization. That marker is how joining threads get unblocked. It was happening in PyThreadState_Delete() but now it will happen in PyThreadState_Clear(). This is necessary to ensure that the callback gets called before much interpreter/runtime finalization happens. (It could impact daemon threads, but we already can't guarantee behavior for those once finalization starts.)

https://bugs.python.org/issue36854

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@ericsnowcurrently
ericsnowcurrently requested review from brettcannon, encukou, ncoghlan and warsaw as code owners May 9, 2019 15:38
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label May 9, 2019
@bedevere-bot bedevere-bot added the awaiting core review label May 9, 2019
@pablogsal
pablogsal requested review from pablogsal and vstinner May 10, 2019 17:33
ericsnowcurrently added 3 commits May 10, 2019 11:44
@ericsnowcurrently
Move GC state from _PyRuntimeState to PyInterpreterState.
e3debdb
@ericsnowcurrently
Mark threads as "done" *before* GC is finalized.
fff3356
@ericsnowcurrently
Add a NEWS entry.
1607996
@ericsnowcurrently
ericsnowcurrently force-pushed the runtime-interp-gc branch from 4bdd07f to 1607996 Compare May 10, 2019 17:45
@ericsnowcurrently ericsnowcurrently mentioned this pull request May 10, 2019
bpo-36818: Add PyInterpreterState.runtime field. #13129
Merged
vstinner
vstinner requested changes May 10, 2019
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like what I see: -) But I have some comments on the actual implementation.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Python/pystate.c
// This will unblock any joining threads.
tstate->on_delete(tstate->on_delete_data);
tstate->on_delete = NULL;
tstate->on_delete_data = NULL;

@vstinner vstinner May 10, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change looks to be unrelated. If it's related, I would prefer to see it as a separated PR to prepare that one.

Same comment for other thread changes:

    // All threading module threads are marked as "done" later
    // in PyThreadState_Clear().

and

        // This will unblock any joining threads.
        // We also do this in PyThreadState_Clear(), but do it here to be sure.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Include/internal/pycore_object.h
filename, lineno, "_PyObject_GC_TRACK");

PyGC_Head *last = (PyGC_Head*)(_PyRuntime.gc.generation0->_gc_prev);
PyInterpreterState *interp = _PyInterpreterState_GET_UNSAFE();

@vstinner vstinner May 10, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to use a "struct _gc_runtime_state *state" variable instead?

struct _gc_runtime_state *state = &_PyInterpreterState_GET_UNSAFE()->gc;

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Modules/gcmodule.c
/*[clinic end generated code: output=45a427e9dce9155c input=81ac4940ca579707]*/
{
_PyRuntime.gc.enabled = 1;
PyInterpreterState *interp = _PyInterpreterState_GET_UNSAFE();

@vstinner vstinner May 10, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to use a "struct _gc_runtime_state *state" variable instead?

struct _gc_runtime_state *state = &_PyInterpreterState_GET_UNSAFE()->gc;

If the line is too lone, maybe add a GET_STATE() macro which returns &_PyInterpreterState_GET_UNSAFE()->gc.

Same comment for the whole file. I would prefer to avoid introducing "PyInterpreterState *interp = _PyInterpreterState_GET_UNSAFE(); " lines if it's only used to get the GC state.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Objects/object.c
{
while (_PyRuntime.gc.trash_delete_later) {
PyObject *op = _PyRuntime.gc.trash_delete_later;
PyInterpreterState *interp = _PyInterpreterState_GET_UNSAFE();

@vstinner vstinner May 10, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, a "state" variable would be more appropriate here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Python/pylifecycle.c
}
*interp_p = interp;

_PyGC_Initialize(&interp->gc);

@vstinner vstinner May 10, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hum, maybe _PyGC_Initialize() should take interp argument instead, to be consistent with other _PyGC functions.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-bot bedevere-bot added awaiting changes and removed awaiting core review labels May 10, 2019
@bedevere-bot

bedevere-bot commented May 10, 2019

Copy link
Copy Markdown

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@csabella

csabella commented May 31, 2019

Copy link
Copy Markdown
Contributor

@ericsnowcurrently, were you wanting to land this for 3.8?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@brettcannon
brettcannon removed their request for review May 31, 2019 19:08
@encukou
encukou removed their request for review September 12, 2019 14:04
@vstinner

vstinner commented Nov 20, 2019

Copy link
Copy Markdown
Member

I wrote a different change, PR #17287, that I just merged. So I close this old PR. Thanks anyway Eric for working on this, my work is partially based on yours ;-)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner vstinner closed this Nov 20, 2019
@ericsnowcurrently

ericsnowcurrently commented Nov 22, 2019

Copy link
Copy Markdown
Member Author

Thanks for doing it!!!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@ericsnowcurrently
ericsnowcurrently deleted the runtime-interp-gc branch November 22, 2019 22:17
@vstinner

vstinner commented Nov 22, 2019

Copy link
Copy Markdown
Member

Thanks for doing it!!!

You're welcome.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner vstinner mentioned this pull request Jan 31, 2020
bpo-39511: PyThreadState_Clear() calls on_delete #18296
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner requested changes
@ncoghlan ncoghlan Awaiting requested review from ncoghlan
@warsaw warsaw Awaiting requested review from warsaw
@pablogsal pablogsal Awaiting requested review from pablogsal

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting changes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@ericsnowcurrently @bedevere-bot @csabella @vstinner @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.