GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-42202: Store func annotations as single tuple at bytecode level by uriyyo · Pull Request #23316 · python/cpython

Conversation

@uriyyo

@uriyyo uriyyo commented Nov 16, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Store func annotations as single tuple at bytecode level.

https://bugs.python.org/issue42202

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@uriyyo
uriyyo requested review from a team, markshannon and tiran as code owners November 16, 2020 16:26
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Nov 16, 2020
@bedevere-bot bedevere-bot added the awaiting review label Nov 16, 2020
serhiy-storchaka
serhiy-storchaka requested changes Nov 16, 2020
View reviewed changes

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM in general.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Objects/codeobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Objects/codeobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Objects/funcobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Python/compile.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Python/compile.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Python/compile.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Python/compile.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread PC/launcher.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Include/cpython/code.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added awaiting changes and removed awaiting review labels Nov 16, 2020
@bedevere-bot

bedevere-bot commented Nov 16, 2020

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@uriyyo
uriyyo requested review from serhiy-storchaka and removed request for a team November 16, 2020 22:25
@uriyyo

uriyyo commented Nov 17, 2020

Copy link
Copy Markdown
Member Author

@serhiy-storchaka I have implemented the same logic for module and class annotations, so all annotations will be computed at compilation time.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@uriyyo
uriyyo force-pushed the fix-issue-42202 branch 3 times, most recently from a4d717f to d1f7312 Compare November 17, 2020 15:38
uriyyo
uriyyo commented Nov 17, 2020
View reviewed changes
Comment thread Lib/test/test_opcodes.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@uriyyo

uriyyo commented Nov 17, 2020

Copy link
Copy Markdown
Member Author

I have made the requested changes; please review again

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added awaiting change review and removed awaiting changes labels Nov 17, 2020
@bedevere-bot

bedevere-bot commented Nov 17, 2020

Copy link
Copy Markdown

Thanks for making the requested changes!

@serhiy-storchaka: please review the changes made to this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

uriyyo and others added 9 commits November 17, 2020 22:05
@uriyyo
Add co_annotations field to codeobject
b4937e9
@uriyyo @serhiy-storchaka
Update Objects/codeobject.c
7ff803e
Co-authored-by: Serhiy Storchaka <[email protected]>
@uriyyo @serhiy-storchaka
Update Objects/codeobject.c
b3bca0e
Co-authored-by: Serhiy Storchaka <[email protected]>
@uriyyo @serhiy-storchaka
Update Objects/funcobject.c
ec0e565
Co-authored-by: Serhiy Storchaka <[email protected]>
@uriyyo @serhiy-storchaka
Update Python/compile.c
6abc7e7
Co-authored-by: Serhiy Storchaka <[email protected]>
@uriyyo
Add PyCode_NewWithAnnotations function in order to create CodeObject …
4c09965
…with annotations
@uriyyo
Use co_annotations for module and class annotations
5e03c3e
@uriyyo
Change format of co_annotations to single tuple of string
1fc24f4
@uriyyo
Regenerate importlib
da237df
@uriyyo
uriyyo force-pushed the fix-issue-42202 branch from d1f7312 to da237df Compare November 17, 2020 20:17
@bedevere-bot
bedevere-bot requested a review from methane November 24, 2020 09:25
@uriyyo
Remove unnecessary goto, cleanup code
c12fc1c
@uriyyo
uriyyo force-pushed the fix-issue-42202 branch from 240166c to c12fc1c Compare November 24, 2020 10:15
@markshannon

markshannon commented Nov 24, 2020

Copy link
Copy Markdown
Member

Rather than return the tuple size via a pointer (out) parameter, you can return the size directly as it cannot be negative, leaving -1 to signify an error. So,

int compiler_visit_argannotation(struct compiler *c, identifier id, expr_ty annotation, Py_ssize_t *annotations_len);

becomes

int compiler_visit_argannotation(struct compiler *c, identifier id, expr_ty annotation);

and the use changes from

if (compiler_visit_argannotation(..., &len))
    goto error;

to

len = compiler_visit_argannotation(...);
if (len < 0)
    goto error;
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@uriyyo

uriyyo commented Nov 24, 2020

Copy link
Copy Markdown
Member Author

Rather than return the tuple size via a pointer (out) parameter, you can return the size directly as it cannot be negative,
leaving -1 to signify an error.

I think it will make code more complicated because it will require rewriting all ifs for compiler_visit_annotations function.

It will be great if compiler_visit_argannotations and compiler_visit_argannotation will follow the same conventions - return tuple size through the pointer or as return value.

@markshannon What is your opinion regarding that?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@markshannon

markshannon commented Nov 24, 2020

Copy link
Copy Markdown
Member

I prefer my approach (I wouldn't have suggested it otherwise 🙂 ), but the code is fine as it is.

I'm still happy with merging it.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@uriyyo

uriyyo commented Nov 24, 2020

Copy link
Copy Markdown
Member Author

I totally agree with you, personally, I don't like when the return value passed through the pointer instead of simple return, as zen said explicit is better than implicit.

But otherwise, In my opinion, it will make code less readable.
From this:

if (!compiler_visit_argannotations(c, args->args, &annotations_len))
   return 0;
if (args->vararg && args->vararg->annotation &&
    !compiler_visit_argannotation(c, args->vararg->arg,
                                  args->vararg->annotation, &annotations_len))
   return 0;

It will be rewritten to:

    len = compiler_visit_argannotations(c, args->args);
    if (len < 0)
        return 0;
    annotations_len += len;
    if (args->vararg && args->vararg->annotation) {
        len = compiler_visit_argannotation(c, args->vararg->arg,
                                           args->vararg->annotation);
        if (len < 0)
            return 0;
        annotations_len += len;
    }

This situation is a double-edged sword when we should choose between code readability and code explicity.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Fidget-Spinner
Fidget-Spinner reviewed Nov 24, 2020
View reviewed changes

@Fidget-Spinner Fidget-Spinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have anything valuable to add here, just a small nitpick with the whatsnew.

This optimization makes me really happy :). Thanks Yuri for the PR and Inada-san for the concept.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Doc/whatsnew/3.10.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@uriyyo @Fidget-Spinner
Update Doc/whatsnew/3.10.rst
b233f2c
Co-authored-by: kj <[email protected]>
@uriyyo

uriyyo commented Nov 24, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

@Fidget-Spinner Thank you for making docs more clear and for adding Inada to the list of contributors. It's my bad, I must definitely add Inada to the contributor's list because this PR is fully his idea.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@uriyyo
Remove trailing whitespaces
85b354b
@uriyyo

uriyyo commented Nov 24, 2020

Copy link
Copy Markdown
Member Author

Can someone rerun Azure Pipelines?
I believe it failed not because of my code)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@methane methane closed this Nov 25, 2020
@methane methane reopened this Nov 25, 2020
@methane

methane commented Nov 25, 2020

Copy link
Copy Markdown
Member

Even core committers can not re-run Azure Pipeline. Close and reopen is the most easy way to rebuild, although it starts not only Azure Pipeline.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@uriyyo
Add Inada to list of contributors
5409fad
@uriyyo

uriyyo commented Nov 25, 2020

Copy link
Copy Markdown
Member Author

@methane Looks like everything is done and this PR can be merged

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@methane
Update _bootstrap_external.py
1bc9561
@methane

methane commented Nov 25, 2020

Copy link
Copy Markdown
Member

I did make regen-importlib but I could not git push to your repository. Would you do it?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@uriyyo

uriyyo commented Nov 25, 2020

Copy link
Copy Markdown
Member Author

I did make regen-importlib but I could not git push to your repository. Would you do it?

Sure, no problems.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@uriyyo
Regenerate importlib
957ce6d
@uriyyo

uriyyo commented Nov 25, 2020

Copy link
Copy Markdown
Member Author

@methane I have regenerated import-lib

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@methane
methane merged commit 7301979 into python:master Nov 25, 2020
@bedevere-bot bedevere-bot removed the awaiting change review label Nov 25, 2020
@methane

methane commented Nov 25, 2020

Copy link
Copy Markdown
Member

Congrats, @uriyyo. And thank you reviewers.

uriyyo and Fidget-Spinner reacted with hooray emoji
All reactions
  • 🎉 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@uriyyo

uriyyo commented Nov 25, 2020

Copy link
Copy Markdown
Member Author

Thanks all, I am happy that we merge this PR💫

@methane Should we do smth similar for class/module annotations?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@methane methane mentioned this pull request Jan 24, 2021
Introduce types (replace stubs in typeshed) PyMySQL/PyMySQL#953
Closed
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 13, 2021
@uriyyo @serhiy-storchaka
@methane @adorilson
bpo-42202: Store func annotations as a tuple (pythonGH-23316)
d0fb681
Reduce memory footprint and improve performance of loading modules having many func annotations.

  >>> sys.getsizeof({"a":"int","b":"int","return":"int"})
  232
  >>> sys.getsizeof(("a","int","b","int","return","int"))
  88

The tuple is converted into dict on the fly when `func.__annotations__` is accessed first.

Co-authored-by: Serhiy Storchaka <[email protected]>
Co-authored-by: Inada Naoki <[email protected]>
@pablogsal pablogsal mentioned this pull request Apr 21, 2021
bpo-38605: Revert making 'from __future__ import annotations' the default #25490
Merged
@tekknolagi tekknolagi mentioned this pull request Jun 12, 2023
Use tuples for function annotations tekknolagi/skybison#511
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Fidget-Spinner Fidget-Spinner Fidget-Spinner left review comments
@markshannon markshannon Awaiting requested review from markshannon
@tiran tiran Awaiting requested review from tiran
@serhiy-storchaka serhiy-storchaka Awaiting requested review from serhiy-storchaka
@ambv ambv Awaiting requested review from ambv
@methane methane Awaiting requested review from methane
+1 more reviewer
@ZackerySpytz ZackerySpytz ZackerySpytz left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

8 participants

@uriyyo @bedevere-bot @ZackerySpytz @methane @markshannon @serhiy-storchaka @Fidget-Spinner @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.