GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-126072: do not add `None` to `co_consts` if no docstring by xuantengh · Pull Request #126101 · python/cpython

Conversation

@xuantengh

@xuantengh xuantengh commented Oct 29, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Hi @markshannon this is my attempt to implement #126072. Initially the value of CO_HAS_DOCSTRING is set as 0x0040, but I found it has been used by the CO_NOFREE in inspect.

cpython/Lib/inspect.py

Lines 409 to 411 in 85799f1

co_flags bitmap: 1=optimized | 2=newlocals | 4=*arg | 8=**arg
| 16=nested | 32=generator | 64=nofree | 128=coroutine
| 256=iterable_coroutine | 512=async_generator

And I've tested it with the following code: See the test cases in changed test files

I've also updated the related documentation, but maybe there is something I missed. Please help me improve this PR, thank you!

  • Issue: The method for storing docstrings in code objects is awkward and prevents optimizations. #126072

📚 Documentation preview 📚: https://cpython-previews--126101.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
xuantengh and others added 9 commits October 28, 2024 21:33
@xuantengh
Merge branch 'python:main' into main
b6d9943
@xuantengh
Merge branch 'python:main' into main
92d4470
@xuantengh
Merge branch 'python:main' into main
b6aee70
@xuantengh
Merge branch 'python:main' into main
7fbd458
@xuantengh
Merge branch 'python:main' into main
f6f52bc
@xuantengh
Merge branch 'python:main' into main
70a1313
@xuantengh
add CO_HAS_DOCSTRING flag
257d58e
@xuantengh
do not add None to co_consts if no docstring
b6edafd
@xuantengh
update doc
7ceaaee
@xuantengh
xuantengh requested review from JelleZijlstra, carljm, iritkatriel, markshannon and willingc as code owners October 29, 2024 02:27
@bedevere-app

bedevere-app Bot commented Oct 29, 2024

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot added the awaiting review label Oct 29, 2024
@bedevere-app bedevere-app Bot mentioned this pull request Oct 29, 2024
The method for storing docstrings in code objects is awkward and prevents optimizations. #126072
Closed
@xuantengh
add news
1674f83
JelleZijlstra
JelleZijlstra reviewed Oct 29, 2024
View reviewed changes

@JelleZijlstra JelleZijlstra left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add some tests, probably to test_code.py.

You'll also probably need to change the __doc__ descriptor on function objects so it continues to work correctly.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Python/codegen.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@xuantengh
xuantengh force-pushed the main branch from 1b0ba7b to 49d3254 Compare October 29, 2024 06:18
iritkatriel
iritkatriel reviewed Oct 29, 2024
View reviewed changes
Comment thread Objects/funcobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Python/codegen.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@xuantengh

xuantengh commented Oct 29, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Maybe the optimization pass should be updated as the hypothesis may not hold:

cpython/Python/flowgraph.c

Lines 2117 to 2119 in 9b14083

// The first constant may be docstring; keep it always.
index_map[0] = 0;

The generated bytecodes for the same function are different before and after this PR.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon

markshannon commented Oct 29, 2024

Copy link
Copy Markdown
Member

FYI, we are in the process of removing small ints from the co_consts tuple: #125972

Can you change the ints in your tests to strings. Doing so will also help test that non-docstring strings do not get interpreted as docstrings.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon

markshannon commented Oct 29, 2024

Copy link
Copy Markdown
Member

Maybe the optimization pass should be updated as the hypothesis may not hold:

cpython/Python/flowgraph.c

Lines 2117 to 2119 in 9b14083

// The first constant may be docstring; keep it always.
index_map[0] = 0;

The generated bytecodes for the same function are different before and after this PR.

There are a few improvements we can make to the bytecode compiler once this PR is done.
It is probably best to do those in other PRs, once this is working.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@xuantengh

xuantengh commented Oct 29, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

It is probably best to do those in other PRs, once this is working.

Yeah, what I concern is whether the constant removal optimization will lead to a unexpected results when the first element in co_consts is no longer None if there is no docstring.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@xuantengh

xuantengh commented Oct 29, 2024

Copy link
Copy Markdown
Contributor Author

Currently, as the const removal optimization unconditionally keeps the first element in consts, None will present in co_consts even if co has no docstring if there is only one const (i.e., None itself).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@xuantengh
xuantengh force-pushed the main branch from bbe1eae to 8d9c6b5 Compare October 29, 2024 11:55
markshannon
markshannon reviewed Oct 29, 2024
View reviewed changes
Comment thread Lib/test/test_compiler_assemble.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_inspect/test_inspect.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

willingc
willingc approved these changes Oct 29, 2024
View reviewed changes

@willingc willingc left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. One small edit for the language reference.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Doc/reference/datamodel.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting review label Oct 29, 2024
@bedevere-app bedevere-app Bot added the awaiting merge label Oct 29, 2024
markshannon
markshannon approved these changes Oct 29, 2024
View reviewed changes

@markshannon markshannon left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this.

This was one of those little things that's been bothering me for a while.
Glad to see it fixed

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

willingc and xuantengh reacted with thumbs up emoji
All reactions
  • 👍 2 reactions
iritkatriel
iritkatriel reviewed Oct 29, 2024
View reviewed changes
Comment thread Python/codegen.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

xuantengh and others added 13 commits October 30, 2024 12:08
@xuantengh
Merge branch 'main' into main
52334c4
@xuantengh
calculate ste_has_docstring in symtable.c
099a765
@xuantengh
update __doc__ for function object
436b2cf
@xuantengh
fix test inspect
f92f180
@xuantengh
fix potential bugs
63a40e0
@xuantengh
fix async func
b34076a
@xuantengh
fix tests
be4cffe
@xuantengh
add doc string co tests
0450f96
@xuantengh
rebase main to resolve conflicts
4fb24e6
@xuantengh
fix test
96085e1
@xuantengh
fix more tests
7b37e1f
@xuantengh
remove issue number comments
67f893e
@xuantengh
take advises
6ebadd6
@xuantengh
xuantengh force-pushed the main branch from 5ff3450 to 6ebadd6 Compare October 30, 2024 04:13
@xuantengh
update inspect doc
0edf4fc
@xuantengh

xuantengh commented Oct 30, 2024

Copy link
Copy Markdown
Contributor Author

Update doc inspect.rst as well.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

willingc
willingc approved these changes Oct 30, 2024
View reviewed changes

@willingc willingc left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small comment on inspect.rst.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Doc/library/inspect.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

xuantengh added 2 commits October 30, 2024 14:34
@xuantengh
update doc
4879a5d
@xuantengh
update inspect.py comment
49944b6
@markshannon

markshannon commented Oct 30, 2024

Copy link
Copy Markdown
Member

The UI shows 2 jobs still in progress. But they have completed successfully 🤷‍♂️

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon
markshannon merged commit 35df4eb into python:main Oct 30, 2024
@bedevere-app bedevere-app Bot removed the awaiting merge label Oct 30, 2024
@xuantengh

xuantengh commented Oct 30, 2024

Copy link
Copy Markdown
Contributor Author

Thanks for your review, it's my first PR merged into CPython! And I'm also interested in the following related optimizations.

iritkatriel and willingc reacted with hooray emoji
All reactions
  • 🎉 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iritkatriel

iritkatriel commented Oct 30, 2024

Copy link
Copy Markdown
Member

Thanks for your review, it's my first PR merged into CPython! And I'm also interested in the following related optimizations.

Congratulations! in your next PR, add your name to Misc/ACKS (we should have done it here, but too late now).

willingc reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@xuantengh xuantengh mentioned this pull request Oct 31, 2024
gh-126072: Set docstring attribute for module and class #126231
Merged
picnixz pushed a commit to picnixz/cpython that referenced this pull request Dec 8, 2024
@xuantengh @picnixz
pythongh-126072: do not add None to co_consts if there is no docs…
6671047
…tring (pythonGH-126101)
ebonnal pushed a commit to ebonnal/cpython that referenced this pull request Jan 12, 2025
@xuantengh @ebonnal
pythongh-126072: do not add None to co_consts if there is no docs…
dcf0b8a
…tring (pythonGH-126101)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JelleZijlstra JelleZijlstra JelleZijlstra left review comments
@iritkatriel iritkatriel iritkatriel left review comments
@willingc willingc willingc approved these changes
@markshannon markshannon markshannon approved these changes
@carljm carljm Awaiting requested review from carljm carljm is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@xuantengh @markshannon @iritkatriel @JelleZijlstra @willingc
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.