GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GH-95245: Move weakreflist into the pre-header. by markshannon · Pull Request #95657 · python/cpython

Conversation

@markshannon

@markshannon markshannon commented Aug 4, 2022 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Part 2 of #95245

Stores the weakreflist pointer in the now vacant space before the dict/values pointer.

  • Issue: Reduce the size of plain Python objects from 8 to 6 machine words. #95245

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
markshannon added 2 commits August 4, 2022 15:21
@markshannon
Move weakreflist into the pre-header.
16746d9
@markshannon
Add news
bcf23ce
@bedevere-bot bedevere-bot added the awaiting core review label Aug 4, 2022
@markshannon

markshannon commented Aug 4, 2022

Copy link
Copy Markdown
Member Author

@ericsnowcurrently You've been playing with weakreflists recently.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon
markshannon requested a review from ericsnowcurrently August 4, 2022 15:56
ericsnowcurrently
ericsnowcurrently requested changes Aug 4, 2022
View reviewed changes

@ericsnowcurrently ericsnowcurrently left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM

I'd just like some clarity on what Py_TPFLAGS_MANAGED_DICT means now and if we need a Py_TPFLAGS_MANAGED_WEAKLIST too.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Objects/typeobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Objects/typeobject.c
assert(!ctx->base->tp_itemsize);
type->tp_weaklistoffset = slotoffset;
slotoffset += sizeof(PyObject *);
if (type->tp_flags & Py_TPFLAGS_MANAGED_DICT) {

@ericsnowcurrently ericsnowcurrently Aug 4, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could Py_TPFLAGS_MANAGED_DICT be set even if wasn't set on line 3089? In other words, could the flag have been set before type_new_descriptors() was called?

If so, that implies that Py_TPFLAGS_MANAGED_DICT also implies a managed weaklist, which I'm not sure is correct.

Otherwise the assert on line 3088 should probably move up to before line 3082.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@ericsnowcurrently ericsnowcurrently Aug 4, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Further, if it's always either both-are-managed or neither then it would be helpful if the flag were more clear (e.g. Py_TPFLAGS_MANAGED_DICT_AND_WEAKLIST).

If either can be managed separately then there should probably be a separate Py_TPFLAGS_MANAGED_WEAKLIST flag.

That might be necessary anyway since Py_TPFLAGS_MANAGED_DICT is already part of 3.11.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@markshannon markshannon Aug 5, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want both to be managed, or neither, but our hand might be forced in the case of multiple inheritance.
One base might have a managed dict, and the other an explicit weakreflist.
I expect this case to be rare.

Currently Py_TPFLAGS_MANAGED_DICT is internal, it is set by the VM, but shouldn't be set by third-party code.

When we make it public, as we should do for 3.12, then it would make sense to add Py_TPFLAGS_MANAGED_WEAKLIST.

I'd like some sort of decision on #95589 first, though.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ericsnowcurrently reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@markshannon markshannon Aug 5, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about this a bit more, we can ensure both are explicit or both managed.
If one is explicit, the we make both explicit.

Moving from an explicitly declare offset to a managed one is unsafe, so we can't do that.
But we can do it the other way around.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ericsnowcurrently reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@ericsnowcurrently ericsnowcurrently Aug 5, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it only matters in the context of type_new_descriptors(). If we can guarantee the both-or-neither aspect in this "limited" context then that is sufficient for this PR (assuming we leave a comment to that effect).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@ericsnowcurrently ericsnowcurrently Aug 5, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regardless, my original question stands: could Py_TPFLAGS_MANAGED_DICT already be set when type_new_descriptors() is called? If so, the branch on 3095 is now problematic, no?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Objects/typeobject.c
type->tp_weaklistoffset = slotoffset;
slotoffset += sizeof(PyObject *);
if (type->tp_flags & Py_TPFLAGS_MANAGED_DICT) {
type->tp_weaklistoffset = -4 * sizeof(PyObject *);

@ericsnowcurrently ericsnowcurrently Aug 4, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the pre-header layout documented, either in a comment, the docs, or an adjacent .txt file? Either way, I don't see the weaklist's place in the pre-header added to such documentation in this PR. That information would be super helpful, since I can only guess what -4 * sizeof(PyObject *) means. A short comment here pointing to that documentation would be nice too.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@markshannon markshannon Aug 5, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documenting the object layout is on the to-do list.
Early draft here: https://github.com/faster-cpython/cpython/blob/document-object-layout/Objects/object_layout.md

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ericsnowcurrently reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@ericsnowcurrently ericsnowcurrently Aug 5, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What? No SVG?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Objects/typeobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Objects/typeobject.c
Comment on lines -3092 to +3103
assert((type->tp_flags & Py_TPFLAGS_MANAGED_DICT) == 0);
type->tp_flags |= Py_TPFLAGS_MANAGED_DICT;
if (type->tp_flags & Py_TPFLAGS_MANAGED_DICT) {

@ericsnowcurrently ericsnowcurrently Aug 4, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise about Py_TPFLAGS_MANAGED_DICT implying something it didn't before (but only if the flag could possibly be set before type_new_descriptors() gets called).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot removed the awaiting core review label Aug 4, 2022
@bedevere-bot

bedevere-bot commented Aug 4, 2022

Copy link
Copy Markdown

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added the awaiting changes label Aug 4, 2022
@markshannon markshannon mentioned this pull request Aug 15, 2022
GH-95245: Move weakreflist into the pre-header. #95996
Merged
@markshannon

markshannon commented Aug 15, 2022

Copy link
Copy Markdown
Member Author

I'm going with a slightly different approach, now that #95854 is merged.
Closing in favor of #95996

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon markshannon closed this Aug 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ericsnowcurrently ericsnowcurrently ericsnowcurrently requested changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting changes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@markshannon @bedevere-bot @ericsnowcurrently
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.