GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-42202: Store func annotations as single tuple at bytecode level by uriyyo · Pull Request #23316 · python/cpython

bpo-42202: Store func annotations as single tuple at bytecode level #23316

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
methane merged 33 commits into python:master from uriyyo:fix-issue-42202
Nov 25, 2020

Uh oh!

There was an error while loading. Please reload this page.
Merged

bpo-42202: Store func annotations as single tuple at bytecode level #23316

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
b4937e9
Add co_annotations field to codeobject
uriyyo Nov 16, 2020
7ff803e
Update Objects/codeobject.c
uriyyo Nov 16, 2020
b3bca0e
Update Objects/codeobject.c
uriyyo Nov 16, 2020
ec0e565
Update Objects/funcobject.c
uriyyo Nov 16, 2020
6abc7e7
Update Python/compile.c
uriyyo Nov 16, 2020
4c09965
Add PyCode_NewWithAnnotations function in order to create CodeObject …
uriyyo Nov 16, 2020
5e03c3e
Use co_annotations for module and class annotations
uriyyo Nov 16, 2020
1fc24f4
Change format of co_annotations to single tuple of string
uriyyo Nov 17, 2020
da237df
Regenerate importlib
uriyyo Nov 17, 2020
201a816
Simplify compiler_add_annotation function
uriyyo Nov 17, 2020
9f7a4eb
Merge remote-tracking branch 'upstream/master' into fix-issue-42202
uriyyo Nov 18, 2020
6742c42
Remove co_annotations. Store func annotations as single tuple at byte…
uriyyo Nov 18, 2020
a71ad66
Update Python/compile.c
uriyyo Nov 18, 2020
3ebf232
Update Objects/funcobject.c
uriyyo Nov 18, 2020
fde6f39
Update Objects/funcobject.c
uriyyo Nov 18, 2020
bb21767
Update Objects/funcobject.c
uriyyo Nov 18, 2020
be73760
Update Objects/funcobject.c
uriyyo Nov 18, 2020
b273dc8
Change local variable name from annotations to anns
uriyyo Nov 18, 2020
b54f9ff
Use correct variable name
uriyyo Nov 18, 2020
34b1c4f
Update Objects/funcobject.c
methane Nov 18, 2020
fa67386
Replace PyTuple_Size with PyTuple_GET_SIZE
uriyyo Nov 19, 2020
9e61756
Fix bug with partially inited annotations dict
uriyyo Nov 20, 2020
0a3ddda
Update docs and bytecode MAGIC_NUMBER
uriyyo Nov 23, 2020
52bda16
Merge branch 'master' into fix-issue-42202
uriyyo Nov 23, 2020
97f51f4
Regenerate importlib
uriyyo Nov 23, 2020
f39234a
Update Doc/library/dis.rst
methane Nov 24, 2020
cd12313
Simplify annotations tuple creation
uriyyo Nov 24, 2020
c12fc1c
Remove unnecessary goto, cleanup code
uriyyo Nov 24, 2020
b233f2c
Update Doc/whatsnew/3.10.rst
uriyyo Nov 24, 2020
85b354b
Remove trailing whitespaces
uriyyo Nov 24, 2020
5409fad
Add Inada to list of contributors
uriyyo Nov 25, 2020
1bc9561
Update _bootstrap_external.py
methane Nov 25, 2020
957ce6d
Regenerate importlib
uriyyo Nov 25, 2020
File filter
Filter by extension
Clear filters
Conversations
Failed to load comments.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Diff view
Diff view

There are no files selected for viewing

4 changes: 3 additions & 1 deletion 4 Doc/library/dis.rst
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1144,11 +1144,13 @@ All of the following opcodes use their arguments.
* ``0x01`` a tuple of default values for positional-only and
positional-or-keyword parameters in positional order
* ``0x02`` a dictionary of keyword-only parameters' default values
* ``0x04`` an annotation dictionary
* ``0x04`` a tuple of strings containing parameters' annotations
* ``0x08`` a tuple containing cells for free variables, making a closure
* the code associated with the function (at TOS1)
* the :term:`qualified name` of the function (at TOS)

.. versionchanged:: 3.10
Flag value ``0x04`` is a tuple of strings instead of dictionary

.. opcode:: BUILD_SLICE (argc)

Expand Down
11 changes: 11 additions & 0 deletions 11 Doc/whatsnew/3.10.rst
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -384,6 +384,11 @@ Optimizations
for more details. (Contributed by Victor Stinner and Pablo Galindo in
:issue:`38980`)

* Function parameters and their annotations are no longer computed at runtime,
but rather at compilation time. They are stored as a tuple of strings at the
bytecode level. It is now around 100% faster to create a function with parameter
annotations. (Contributed by Yurii Karabas and Inada Naoki in :issue:`42202`)

Deprecated
==========

Expand Down Expand Up @@ -457,6 +462,12 @@ Changes in the Python API
have been renamed to *exc*.
(Contributed by Zackery Spytz and Matthias Bussonnier in :issue:`26389`.)

CPython bytecode changes
========================

* The ``MAKE_FUNCTION`` instruction accepts tuple of strings as annotations
instead of dictionary.
(Contributed by Yurii Karabas and Inada Naoki in :issue:`42202`)

Build Changes
=============
Expand Down
3 changes: 2 additions & 1 deletion 3 Lib/importlib/_bootstrap_external.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,7 @@ def _write_atomic(path, data, mode=0o666):
# Python 3.9a2 3425 (simplify bytecodes for **value unpacking)
# Python 3.10a1 3430 (Make 'annotations' future by default)
# Python 3.10a1 3431 (New line number table format -- PEP 626)
# Python 3.10a2 3432 (Function annotation for MAKE_FUNCTION is changed from dict to tuple bpo-42202)

#
# MAGIC must change whenever the bytecode emitted by the compiler may no
Expand All @@ -320,7 +321,7 @@ def _write_atomic(path, data, mode=0o666):
# Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array
# in PC/launcher.c must also be updated.

MAGIC_NUMBER = (3431).to_bytes(2, 'little') + b'\r\n'
MAGIC_NUMBER = (3432).to_bytes(2, 'little') + b'\r\n'
_RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c

_PYCACHE = '__pycache__'
Expand Down
2 changes: 2 additions & 0 deletions 2 Misc/NEWS.d/next/Core and Builtins/2020-11-16-18-13-07.bpo-42202.ZxenYD.rst
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Change function parameters annotations internal representation to tuple
of strings. Patch provided by Yurii Karabas.
19 changes: 19 additions & 0 deletions 19 Objects/funcobject.c
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -424,6 +424,25 @@ func_get_annotations(PyFunctionObject *op, void *Py_UNUSED(ignored))
if (op->func_annotations == NULL)
return NULL;
}
if (PyTuple_CheckExact(op->func_annotations)) {
PyObject *ann_tuple = op->func_annotations;
PyObject *ann_dict = PyDict_New();
if (ann_dict == NULL) {
return NULL;
}

assert(PyTuple_GET_SIZE(ann_tuple) % 2 == 0);

for (Py_ssize_t i = 0; i < PyTuple_GET_SIZE(ann_tuple); i += 2) {
int err = PyDict_SetItem(ann_dict,
PyTuple_GET_ITEM(ann_tuple, i),
PyTuple_GET_ITEM(ann_tuple, i + 1));

if (err < 0)
return NULL;
}
Py_SETREF(op->func_annotations, ann_dict);
}
Py_INCREF(op->func_annotations);
return op->func_annotations;
}
Expand Down
2 changes: 1 addition & 1 deletion 2 Python/ceval.c
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3880,7 +3880,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
func ->func_closure = POP();
}
if (oparg & 0x04) {
assert(PyDict_CheckExact(TOP()));
assert(PyTuple_CheckExact(TOP()));
func->func_annotations = POP();
}
if (oparg & 0x02) {
Expand Down
70 changes: 27 additions & 43 deletions 70 Python/compile.c
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2027,26 +2027,24 @@ compiler_visit_annexpr(struct compiler *c, expr_ty annotation)

static int
compiler_visit_argannotation(struct compiler *c, identifier id,
expr_ty annotation, PyObject *names)
expr_ty annotation, Py_ssize_t *annotations_len)
{
if (annotation) {
PyObject *mangled;
VISIT(c, annexpr, annotation);
Comment thread
methane marked this conversation as resolved.
Resolved

Uh oh!

There was an error while loading. Please reload this page.
mangled = _Py_Mangle(c->u->u_private, id);
PyObject *mangled = _Py_Mangle(c->u->u_private, id);
if (!mangled)
return 0;
if (PyList_Append(names, mangled) < 0) {
Py_DECREF(mangled);
return 0;
}

ADDOP_LOAD_CONST(c, mangled);
Py_DECREF(mangled);
VISIT(c, annexpr, annotation);
*annotations_len += 2;
}
return 1;
}

static int
compiler_visit_argannotations(struct compiler *c, asdl_arg_seq* args,
PyObject *names)
Py_ssize_t *annotations_len)
{
int i;
for (i = 0; i < asdl_seq_LEN(args); i++) {
Expand All @@ -2055,7 +2053,7 @@ compiler_visit_argannotations(struct compiler *c, asdl_arg_seq* args,
c,
arg->arg,
arg->annotation,
names))
annotations_len))
return 0;
}
return 1;
Expand All @@ -2065,58 +2063,44 @@ static int
compiler_visit_annotations(struct compiler *c, arguments_ty args,
expr_ty returns)
{
/* Push arg annotation dict.
/* Push arg annotation names and values.
The expressions are evaluated out-of-order wrt the source code.

Return 0 on error, -1 if no dict pushed, 1 if a dict is pushed.
Return 0 on error, -1 if no annotations pushed, 1 if a annotations is pushed.
*/
static identifier return_str;
PyObject *names;
Py_ssize_t len;
names = PyList_New(0);
if (!names)
return 0;
Py_ssize_t annotations_len = 0;

if (!compiler_visit_argannotations(c, args->args, names))
goto error;
if (!compiler_visit_argannotations(c, args->posonlyargs, names))
goto error;
if (!compiler_visit_argannotations(c, args->args, &annotations_len))
return 0;
if (!compiler_visit_argannotations(c, args->posonlyargs, &annotations_len))
return 0;
if (args->vararg && args->vararg->annotation &&
!compiler_visit_argannotation(c, args->vararg->arg,
args->vararg->annotation, names))
goto error;
if (!compiler_visit_argannotations(c, args->kwonlyargs, names))
goto error;
args->vararg->annotation, &annotations_len))
return 0;
if (!compiler_visit_argannotations(c, args->kwonlyargs, &annotations_len))
return 0;
if (args->kwarg && args->kwarg->annotation &&
!compiler_visit_argannotation(c, args->kwarg->arg,
args->kwarg->annotation, names))
goto error;
args->kwarg->annotation, &annotations_len))
return 0;

if (!return_str) {
return_str = PyUnicode_InternFromString("return");
if (!return_str)
goto error;
return 0;
}
if (!compiler_visit_argannotation(c, return_str, returns, names)) {
goto error;
if (!compiler_visit_argannotation(c, return_str, returns, &annotations_len)) {
return 0;
}

len = PyList_GET_SIZE(names);
if (len) {
PyObject *keytuple = PyList_AsTuple(names);
Py_DECREF(names);
ADDOP_LOAD_CONST_NEW(c, keytuple);
ADDOP_I(c, BUILD_CONST_KEY_MAP, len);
if (annotations_len) {
ADDOP_I(c, BUILD_TUPLE, annotations_len);
return 1;
}
else {
Py_DECREF(names);
return -1;
}

error:
Py_DECREF(names);
return 0;
return -1;
}

static int
Expand Down
Loading
Oops, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.