GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-95756: Lazily created cached co_* attrs by Fidget-Spinner · Pull Request #97791 · python/cpython

Conversation

@Fidget-Spinner

@Fidget-Spinner Fidget-Spinner commented Oct 3, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

As long as these aren't accessed, they don't actually consume additional space over 3.11.

  • Issue: Better performance for code getters in C API #95756

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@Fidget-Spinner
Cached co_* attrs
2e28fb7
@bedevere-bot bedevere-bot added the awaiting core review label Oct 3, 2022
@blurb-it
📜🤖 Added by blurb_it.
4e85e31
@Fidget-Spinner
Fidget-Spinner marked this pull request as ready for review October 3, 2022 20:33
@Fidget-Spinner
Fidget-Spinner requested a review from markshannon as a code owner October 3, 2022 20:33
@Fidget-Spinner
Fidget-Spinner requested a review from brandtbucher October 3, 2022 20:33
@Fidget-Spinner Fidget-Spinner added the sprint label Oct 3, 2022
@Fidget-Spinner

Fidget-Spinner commented Oct 3, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Note that these were all O(1) in 3.10. This restores their behavior. They also previously returned just incref of a cache.
https://github.com/python/cpython/blame/08eb754d840696914928355014c2d424131f8835/Objects/codeobject.c
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@markshannon

markshannon commented Oct 3, 2022

Copy link
Copy Markdown
Member

I'd like to reduce the size of code objects for a few reasons. PEP 649 will create lots of small code objects and we want multiple interpreters to share as much data as possible.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@markshannon

markshannon commented Oct 7, 2022

Copy link
Copy Markdown
Member

I'm merging this, as we can add the line array, and any other tables needed in unusual cases, to the _co_cached structs.
In other words, this will save space.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
markshannon
markshannon requested changes Oct 7, 2022
View reviewed changes
Comment thread Doc/c-api/code.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Doc/c-api/code.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Doc/c-api/code.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Objects/codeobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Objects/codeobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the awaiting core review label Oct 7, 2022
@bedevere-bot

bedevere-bot commented Oct 7, 2022

Copy link
Copy Markdown

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot added the awaiting changes label Oct 7, 2022
Fidget-Spinner added 5 commits October 7, 2022 12:17
@Fidget-Spinner
apply suggestions
6cfa7ae
@Fidget-Spinner
fix segfault
ce5f0e8
@Fidget-Spinner
yes
d998aa1
@Fidget-Spinner
Update 2022-10-03-20-33-24.gh-issue-95756.SSmXlG.rst
d321cfb
@Fidget-Spinner
Merge branch 'main' of https://github.com/python/cpython into code_at…
3e227fa 
…trs_cached
markshannon
markshannon reviewed Oct 10, 2022
View reviewed changes
Comment thread Objects/codeobject.c
if (*cached_field != NULL) {
return Py_NewRef(*cached_field);
}
assert(*cached_field == NULL);

@markshannon markshannon Oct 10, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to assert this. The test two lines above ensures that it is true.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
markshannon
markshannon approved these changes Oct 10, 2022
View reviewed changes

@markshannon markshannon left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One assert that could be removed, but it's harmless.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting changes labels Oct 10, 2022
@Fidget-Spinner

Fidget-Spinner commented Oct 11, 2022

Copy link
Copy Markdown
Member Author

I'll leave the assert in since it communicates some info should the code change in the future. And like you mentioned it's harmless anyways.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@Fidget-Spinner
Fidget-Spinner merged commit b399115 into python:main Oct 11, 2022
@bedevere-bot bedevere-bot removed the awaiting merge label Oct 11, 2022
@Fidget-Spinner
Fidget-Spinner deleted the code_attrs_cached branch October 11, 2022 03:26
@bedevere-bot

bedevere-bot commented Oct 11, 2022

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM64 macOS 3.x has failed when building commit b399115.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/725/builds/2786) and take a look at the build logs.
  4. Check if the failure is related to this commit (b399115) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/725/builds/2786

Failed tests:

  • test_syslog

Summary of the results of the build (if available):

==

Click to see traceback logs 
remote: Enumerating objects: 30, done.        
remote: Counting objects:   3% (1/30)        
remote: Counting objects:   6% (2/30)        
remote: Counting objects:  10% (3/30)        
remote: Counting objects:  13% (4/30)        
remote: Counting objects:  16% (5/30)        
remote: Counting objects:  20% (6/30)        
remote: Counting objects:  23% (7/30)        
remote: Counting objects:  26% (8/30)        
remote: Counting objects:  30% (9/30)        
remote: Counting objects:  33% (10/30)        
remote: Counting objects:  36% (11/30)        
remote: Counting objects:  40% (12/30)        
remote: Counting objects:  43% (13/30)        
remote: Counting objects:  46% (14/30)        
remote: Counting objects:  50% (15/30)        
remote: Counting objects:  53% (16/30)        
remote: Counting objects:  56% (17/30)        
remote: Counting objects:  60% (18/30)        
remote: Counting objects:  63% (19/30)        
remote: Counting objects:  66% (20/30)        
remote: Counting objects:  70% (21/30)        
remote: Counting objects:  73% (22/30)        
remote: Counting objects:  76% (23/30)        
remote: Counting objects:  80% (24/30)        
remote: Counting objects:  83% (25/30)        
remote: Counting objects:  86% (26/30)        
remote: Counting objects:  90% (27/30)        
remote: Counting objects:  93% (28/30)        
remote: Counting objects:  96% (29/30)        
remote: Counting objects: 100% (30/30)        
remote: Counting objects: 100% (30/30), done.        
remote: Compressing objects:   6% (1/15)        
remote: Compressing objects:  13% (2/15)        
remote: Compressing objects:  20% (3/15)        
remote: Compressing objects:  26% (4/15)        
remote: Compressing objects:  33% (5/15)        
remote: Compressing objects:  40% (6/15)        
remote: Compressing objects:  46% (7/15)        
remote: Compressing objects:  53% (8/15)        
remote: Compressing objects:  60% (9/15)        
remote: Compressing objects:  66% (10/15)        
remote: Compressing objects:  73% (11/15)        
remote: Compressing objects:  80% (12/15)        
remote: Compressing objects:  86% (13/15)        
remote: Compressing objects:  93% (14/15)        
remote: Compressing objects: 100% (15/15)        
remote: Compressing objects: 100% (15/15), done.        
remote: Total 16 (delta 14), reused 2 (delta 1), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to 'b399115ef18945f7526492225d72a512048ad20d'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at b399115ef1 gh-95756: Lazily created cached co_* attrs (GH-97791)
Switched to and reset branch 'main'

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
configure: WARNING: ffi.h: accepted by the compiler, rejected by the preprocessor!
configure: WARNING: ffi.h: proceeding with the compiler's result
configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.

Fatal Python error: Fatal Python error: Segmentation faultSegmentation fault



Thread 0x000000028301f000 (most recent call first):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_syslog.py", line 67 in logger
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/threading.py", line 986 in run
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/threading.py", line 1049 in _bootstrap_inner
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/threading.py", line 1006 in _bootstrap

Thread 0x0000000282013000make: *** [buildbottest] Segmentation fault: 11

Cannot open file '/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/test-results.xml' for upload
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@Fidget-Spinner

Fidget-Spinner commented Oct 11, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

I'm confused. Why is this the only buildbot failing? In any case I might revert this within 24 hours.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner

vstinner commented Oct 11, 2022

Copy link
Copy Markdown
Member

I'm confused. Why is this the only buildbot failing? In any case I might revert this within 24 hours.

I created #98178 about this crash. It seems to be specific to macOS and unrelated to your change. If your change would introduce a bug, more tests would crash on more platforms.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@markshannon markshannon markshannon approved these changes
@brandtbucher brandtbucher Awaiting requested review from brandtbucher

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

sprint

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@Fidget-Spinner @markshannon @bedevere-bot @vstinner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.