GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-119127: functools.partial placeholders by dg-pb · Pull Request #119827 · python/cpython

Conversation

@dg-pb

@dg-pb dg-pb commented May 31, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

As I already had implementation I though PR might be helpful for others to see and evaluate.

From all the different extensions of functools.partial I think this one is the best. It is relatively simple and exposes all missing functionality. Other partial extensions that I have seen lack functionality and would not provide complete argument ordering capabilities and/or are too complicated in relation to what they offer.

Implementation can be summarised as follows:
a) Trailing placeholders are not allowed. (Makes things simpler)
b) Throws exception if not all placeholders are filled on call
c) retains optimization benefits of application on other partial instances.

Performance penalty compared to current functools.partial is minimal for extension class. + 20-30 ns for initialisation and <4 ns when called with or without placeholders.

To put it simply, new functionality extends functools.partial so that it has flexibility of lambda / def approach (in terms of argument ordering), but call overhead is 2x smaller.

The way I see it is that this could only be justified if this extension provided completeness and no new functionality is going to be needed anywhere near in the future. I have thought about it and tried various alternatives and I think there is a good chance that this is the case. Personally, I don't think I would ever need anything more from partial class.

Current implementation functions reliably.

Benchmark

There is nothing new here in terms of performance. The performance after this PR will be (almost) the same as the performance of partial until now. Placeholders only provide flexibility for taking advantage of performance benefits where it is important.

So far I have identified 2 such cases:

  1. More flexible predicate construction for functions in operator module. This allows for new strategies in making performant iterator recipes.
  2. Partializing input target function. Examples of this are optimizers and similar. I.e. cases where the function will be called over and over within the routine with number of arguments. But the input target function needs partial substitution for positionals and keywords.

Good example of this is scipy.optimize.minimize.

Its signature is: scipy.optimize.minimize(fun, x0, args=(), ...)

Note, it does not have kwds. Why? I don't know. But good reason for it could be:

fun = lambda x: f(x, **kwds)

will need to expand **kwds on every call (even if it is empty), while partial will make the most optimal call. (see benchmarks below). So the minimize function can leave out kwds given there is a good way to source callable with already substituted keywords.

This extension allows pre-substituting both positionals and keywords. This allows optimizer signature to leave out both kwds and args resulting in simpler interface scipy.optimize.minimize(fun, x0, ...) and gaining slightly better performance - function calls are at the center of such problems after all.

Benchmark Results for __call__

Code for Cases

dct = {'a': 1}
kwds = {'c': 1, 'd': 2}
kwds_empty = {}
args1 = (1,)
args3 = (1, 2, 4)

opr_sub = opr.sub
opr_contains = opr.contains

opr_sub_lambda = lambda b: opr_sub(1, b)
opr_sub_partial = ftl.partial(opr_sub, 1)

opr_contains_lambda = lambda b: opr_contains(dct, b)
opr_contains_partial = ftl.partial(opr_contains, dct)


def pos2(a, b):
    pass

def pos6(a, b, c, d, e, f):
    pass

def pos2kw2(a, b, c=1, d=2):
    pass


pos2_lambda = lambda b: pos2(1, b)
pos2_partial = ftl.partial(pos2, 1)

pos6_lambda = lambda b, c, d: pos6(1, 2, 3, b, c, d)
pos6_partial = ftl.partial(pos6, 1, 2, 3)

pos2kw2_kw_lambda = lambda b: pos2kw2(1, b, **kwds)
pos2kw2_kw_partial = ftl.partial(pos2kw2, 1, **kwds)

pos2kw2_kwe_lambda = lambda b: pos2kw2(1, b, **kwds_empty)
pos2kw2_kwe_partial = ftl.partial(pos2kw2, 1, **kwds_empty)

opr_sub_partial_ph = ftl.partial(opr_sub, PH, 1)
opr_contains_partial_ph = ftl.partial(opr_contains, PH, 'a')
pos2_partial_ph = ftl.partial(pos2, PH, 1)
pos6_partial_ph = ftl.partial(pos6, PH, 2, PH, 4, PH, 6)
pos2kw2_kw_partial_ph = ftl.partial(pos2kw2, PH, 1, **kwds)
pos2kw2_kwe_partial_ph = ftl.partial(pos2kw2, PH, 1, **kwds_empty)

# Placeholder versions
from functools import Placeholder as PH
opr_sub_partial_ph = ftl.partial(opr_sub, PH, 1)
opr_contains_partial_ph = ftl.partial(opr_contains, PH, 'a')
pos2_partial_ph = ftl.partial(pos2, PH, 1)
pos6_partial_ph = ftl.partial(pos6, PH, 2, PH, 4, PH, 6)
pos2kw2_kw_partial_ph = ftl.partial(pos2kw2, PH, 1, **kwds)
pos2kw2_kwe_partial_ph = ftl.partial(pos2kw2, PH, 1, **kwds_empty)

CPython Results

C Implementation
----------------
    ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
    ┃   5 repeats, 1,000,000 times     ┃
    ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
    ┃    Units: ns    lambda   partial ┃
    ┃              ┏━━━━━━━━━━━━━━━━━━━┫
    ┃      opr_sub ┃  50 ± 4    40 ± 2 ┃
    ┃ opr_contains ┃  53 ± 3    43 ± 3 ┃
    ┃         pos2 ┃  50 ± 1    64 ± 1 ┃
    ┃ pos2(*args1) ┃  69 ± 5    73 ± 5 ┃
    ┃         pos6 ┃  58 ± 1   103 ± 5 ┃
    ┃ pos6(*args3) ┃  77 ± 3    99 ± 5 ┃
    ┃   pos2kw2_kw ┃ 240 ± 4   259 ± 7 ┃
    ┃  pos2kw2_kwe ┃ 134 ± 6    69 ± 3 ┃
    ┗━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━┛
    With Placeholders
    ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
    ┃           5 repeats, 1,000,000 times              ┃
    ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
    ┃    Units: ns     lambda    partial   Placeholders ┃
    ┃              ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
    ┃      opr_sub ┃  50 ±  2    39 ±  1        44 ±  4 ┃
    ┃ opr_contains ┃  61 ±  2    44 ±  2        49 ±  2 ┃
    ┃         pos2 ┃  54 ±  2    58 ±  3        64 ±  2 ┃
    ┃ pos2(*args1) ┃  67 ±  3    72 ±  9        69 ±  3 ┃
    ┃         pos6 ┃  63 ±  3   102 ±  3        99 ±  2 ┃
    ┃ pos6(*args3) ┃  75 ±  3   101 ±  2        94 ±  4 ┃
    ┃   pos2kw2_kw ┃ 242 ±  7   259 ± 10       260 ±  7 ┃
    ┃  pos2kw2_kwe ┃ 131 ±  4    64 ±  1        69 ±  2 ┃
    ┗━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛


Python Implementation
---------------------
    Current
    ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
    ┃    5 repeats, 1,000,000 times      ┃
    ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
    ┃    Units: ns     lambda    partial ┃
    ┃              ┏━━━━━━━━━━━━━━━━━━━━━┫
    ┃      opr_sub ┃  48 ±  1   373 ± 13 ┃
    ┃ opr_contains ┃  51 ±  1   377 ± 12 ┃
    ┃         pos2 ┃  51 ±  4   378 ±  5 ┃
    ┃ pos2(*args1) ┃  63 ±  5   354 ±  7 ┃
    ┃         pos6 ┃  59 ±  1   437 ±  5 ┃
    ┃ pos6(*args3) ┃  75 ±  2   410 ±  7 ┃
    ┃   pos2kw2_kw ┃ 239 ±  4   517 ±  5 ┃
    ┃  pos2kw2_kwe ┃ 133 ±  3   408 ± 49 ┃
    ┗━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━┛
    With Placeholders
    ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
    ┃           5 repeats, 1,000,000 times              ┃
    ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
    ┃    Units: ns     lambda    partial   Placeholders ┃
    ┃              ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
    ┃      opr_sub ┃  49 ±  1   392 ± 13       547 ± 12 ┃
    ┃ opr_contains ┃  54 ±  2   393 ±  9       605 ± 78 ┃
    ┃         pos2 ┃  55 ±  9   398 ±  7       544 ±  5 ┃
    ┃ pos2(*args1) ┃  66 ±  2   373 ±  5       533 ±  8 ┃
    ┃         pos6 ┃  58 ±  5   462 ±  4       652 ±  3 ┃
    ┃ pos6(*args3) ┃  74 ±  2   428 ± 11       635 ±  9 ┃
    ┃   pos2kw2_kw ┃ 240 ±  5   533 ±  4       696 ± 10 ┃
    ┃  pos2kw2_kwe ┃ 134 ±  2   406 ±  4       555 ±  3 ┃
    ┗━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

PyPy Results

PyPy
----
    ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
    ┃     5 repeats, 10,000 times        ┃
    ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
    ┃    Units: ns     lambda    partial ┃
    ┃              ┏━━━━━━━━━━━━━━━━━━━━━┫
    ┃      opr_sub ┃ 122 ± 15   266 ± 70 ┃
    ┃ opr_contains ┃ 147 ±  7   248 ± 64 ┃
    ┃         pos2 ┃ 114 ± 17   204 ± 49 ┃
    ┃ pos2(*args1) ┃ 156 ± 24   202 ± 28 ┃
    ┃         pos6 ┃ 124 ± 14   268 ± 39 ┃
    ┃ pos6(*args3) ┃ 147 ± 36   225 ± 21 ┃
    ┃   pos2kw2_kw ┃ 259 ± 17   436 ± 66 ┃
    ┃  pos2kw2_kwe ┃ 180 ± 14   243 ± 43 ┃
    ┗━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━┛
    ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
    ┃   5 repeats, 1,000,000 times     ┃
    ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
    ┃     Units: ns   lambda   partial ┃
    ┃               ┏━━━━━━━━━━━━━━━━━━┫
    ┃       opr_sub ┃  1 ± 0     3 ± 1 ┃
    ┃  opr_contains ┃ 13 ± 0    16 ± 2 ┃
    ┃          pos2 ┃  1 ± 0     3 ± 1 ┃
    ┃  pos2(*args1) ┃  2 ± 0     2 ± 0 ┃
    ┃          pos6 ┃  1 ± 0     2 ± 0 ┃
    ┃  pos6(*args3) ┃  2 ± 0     2 ± 0 ┃
    ┃    pos2kw2_kw ┃ 42 ± 1    72 ± 2 ┃
    ┃   pos2kw2_kwe ┃  2 ± 0     2 ± 0 ┃
    ┗━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━┛

PyPy Placeholder
----------------
    ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
    ┃             5 repeats, 10,000 times                 ┃
    ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
    ┃    Units: ns      lambda     partial   Placeholders ┃
    ┃              ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
    ┃      opr_sub ┃ 114 ±   5   256 ±  82      719 ± 170 ┃
    ┃ opr_contains ┃ 142 ±   7   538 ± 536      787 ± 145 ┃
    ┃         pos2 ┃ 125 ±  19   239 ±  54      679 ± 116 ┃
    ┃ pos2(*args1) ┃ 130 ±  30   199 ±  17      638 ±  48 ┃
    ┃         pos6 ┃ 115 ±  16   237 ±  43      785 ± 176 ┃
    ┃ pos6(*args3) ┃ 138 ±  25   214 ±  14      703 ±  19 ┃
    ┃   pos2kw2_kw ┃ 260 ±  24   382 ±  67      850 ±  92 ┃
    ┃  pos2kw2_kwe ┃ 179 ±  28   223 ±  44      661 ±  32 ┃
    ┗━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
    ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
    ┃          5 repeats, 1,000,000 times             ┃
    ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
    ┃    Units: ns    lambda   partial   Placeholders ┃
    ┃              ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
    ┃      opr_sub ┃  1 ±  0    3 ±  1       156 ±  4 ┃
    ┃ opr_contains ┃ 13 ±  0   15 ±  1       173 ±  3 ┃
    ┃         pos2 ┃  2 ±  0    3 ±  1       154 ±  7 ┃
    ┃ pos2(*args1) ┃  2 ±  0    2 ±  0       148 ±  3 ┃
    ┃         pos6 ┃  2 ±  0    3 ±  1       200 ±  2 ┃
    ┃ pos6(*args3) ┃  2 ±  0    3 ±  0       217 ± 39 ┃
    ┃   pos2kw2_kw ┃ 43 ±  1   71 ±  1       240 ±  2 ┃
    ┃  pos2kw2_kwe ┃  2 ±  0    2 ±  0       149 ±  2 ┃
    ┗━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

Setup:

  • First 2 columns are identical calls - one using lambda other partial.
  • 3rd column is using placeholder to expose 1st argument as opposed to 2nd (or different places for 6-arg case).

CPython:

  • There is negligible impact on __call__. Run times are very close of current and new version with Placeholders.
  • It can be seen that run times are not impacted by placeholder usage in any significant way.
  • pos2kw2_kwe (empty kwds) is much faster of partial call. pos2kw2_kw (non-empty kwds) is currently slower, however gh-119109: functool.partial vectorcall supports pto->kw & fallback to tp_call removed #120783 will likely to improve its speed so that it outperforms lambda.

PyPy:

  • Usage of Placeholders results in very poor performance. However, this has no material implication as lambda is more performant than partial in all cases and is an optimal choice.

Benchmark Results for __new__

INIT='import functools as ftl; g = lambda a, b, c, d, e, f: (a, b, c, d, e, f);'

# CURRENT
$PY_MAIN -m timeit -s $INIT 'ftl.partial(g, 0, 1, 2)'   # 160

# PLACEHOLDERS
INIT2="$INIT PH=ftl.Placeholder;"
$PY_MAIN -m timeit -s $INIT2 'ftl.partial(g, 0, 1, 2)'              # 170
$PY_MAIN -m timeit -s $INIT2 'ftl.partial(g, 0, 1, 2, 3, 4, 5)'     # 190
$PY_MAIN -m timeit -s $INIT2 'ftl.partial(g, PH, 1, PH, 3, PH, 5)'  # 200
  • There is small performance decrease for initialization without placeholders.
  • Initializing it with placeholders is slower for the same number of arguments (excluding placeholders).
  • But it is not much slower if placeholders are counted as arguments.

To sum up

This extension:

  1. allows extracting current performance benefits of partial to few more important (at least from my POV) cases.
  2. seems to allow for certain simplifications to happen by bringing it more in line with lambda/def behaviour. Thus, allowing partial to be used for partialmethod application which allows for some simplifications in handling these in other parts of the library - i.e. inspect.
  • Issue: functools.partial placeholders #119127

📚 Documentation preview 📚: https://cpython-previews--119827.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

nineteendo and jab reacted with thumbs up emoji
All reactions
  • 👍 2 reactions
dg-pb and others added 17 commits May 21, 2024 13:50
@dg-pb
Initial Implementation
ee7333c
@dg-pb
serialization fix
8bcc462
@dg-pb
bug fix
c67c9b4
@dg-pb
Bug 2 fix
680d900
@dg-pb
Py_TPFLAGS_IMMUTABLETYPE added
9591ff5
@dg-pb
placeholder added to state as opposed to being used as global constant
067e938
@dg-pb
static removed
8af20b3
@dg-pb
creating sentinel via PyType_Spec
607a0b1
@dg-pb
more accurate variable name
f55801e
@dg-pb
trailing trim bug and tests
5894145
@dg-pb
Updated docs
3722e07
@dg-pb
blurb
a79c2af
d.grigonis
Merge branch 'main' into implement-119127
12aaa72
@dg-pb
minor edit
92c767b
@dg-pb
doc fix
496a9d2
@dg-pb
better variable names and mini corrections
38d9c11
@dg-pb
Merge branch 'implement-119127' into implement-119127-again
707b957
@dg-pb
dg-pb requested a review from rhettinger as a code owner May 31, 2024 07:54
@bedevere-app bedevere-app Bot added the awaiting review label May 31, 2024
@bedevere-app bedevere-app Bot mentioned this pull request May 31, 2024
functools.partial placeholders #119127
Closed
rhettinger
rhettinger reviewed Jun 4, 2024
View reviewed changes
Comment thread Doc/library/functools.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

rhettinger
rhettinger reviewed Jun 4, 2024
View reviewed changes
Comment thread Lib/functools.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

rhettinger
rhettinger reviewed Jun 4, 2024
View reviewed changes
Comment thread Lib/functools.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@rhettinger rhettinger self-assigned this Jun 6, 2024
erlend-aasland
erlend-aasland reviewed Jun 6, 2024
View reviewed changes
Comment thread Doc/library/functools.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/library/functools.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/library/functools.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_functoolsmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_functoolsmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_functoolsmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_functoolsmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_functoolsmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_functoolsmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_functoolsmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

dg-pb added 4 commits June 8, 2024 09:19
@dg-pb
review comments mostly
14b38ca
@dg-pb
singleton sentinel and reduce
32bca19
@dg-pb
python module sentinel better mimics the one of the extension
8576493
@dg-pb
Emulated None behaviour, but using PyType_FromModuleAndSpec
a3fd2d6
picnixz
picnixz reviewed Jun 8, 2024
View reviewed changes

@picnixz picnixz left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally would advocate for PLACEHOLDER instead of Placeholder to stress that 1) this is not a global constant, just a module constant, 2) this is not a class, 3) this is named similarly to dataclasses.KW_ONLY. (see #119827 (comment))

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Doc/library/functools.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/library/functools.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/library/functools.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/functools.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_functoolsmodule.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/library/functools.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

dg-pb added 2 commits August 13, 2024 20:46
@dg-pb
better doc example and small test changes
6e3d282
@dg-pb
assertRaisesRegex exact match
66c305d
@dg-pb

dg-pb commented Aug 14, 2024

Copy link
Copy Markdown
Contributor Author

Not very convenient to match exact exception message string, which, at least from my experience, is more common case. #123013

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

zware
zware reviewed Aug 14, 2024
View reviewed changes
Comment thread Doc/library/functools.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/library/functools.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

dg-pb and others added 6 commits August 14, 2024 23:09
@dg-pb
doc nits
14bf68c
@rhettinger
Add Placeholder to __all__
ee642d5
@rhettinger
Update copyright span
8d6c28e
@rhettinger
Minor doc edits and add doctests
8744bcb
@rhettinger
Merge branch 'main' into implement-119127-again
b896470
@rhettinger
Merge branch 'main' into implement-119127-again
4881ae6
serhiy-storchaka
serhiy-storchaka reviewed Sep 25, 2024
View reviewed changes
Comment thread Doc/library/functools.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@rhettinger rhettinger added the sprint label Sep 26, 2024
@rhettinger
Having the separate dict was necessary to eliminate duplicate keywords
c3ad7d9
@rhettinger

rhettinger commented Sep 26, 2024

Copy link
Copy Markdown
Contributor

I just looked at the Placeholder singleton logic and think it is fine because it mirrors the None singleton. It is a little complicated but not in a way that will impair maintenance.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@rhettinger
Merge branch 'main' into implement-119127-again
5e5d484
@rhettinger
rhettinger enabled auto-merge (squash) September 26, 2024 00:52
@rhettinger
rhettinger merged commit d929652 into python:main Sep 26, 2024
@bedevere-app bedevere-app Bot removed the awaiting core review label Sep 26, 2024
@bedevere-bot

bedevere-bot commented Sep 26, 2024

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 CentOS9 NoGIL Refleaks 3.x has failed when building commit d929652.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1610/builds/60) and take a look at the build logs.
  4. Check if the failure is related to this commit (d929652) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1610/builds/60

Failed tests:

  • test__interpchannels
  • test_capi
  • test__interpreters
  • test_ast
  • test_importlib

Test leaking resources:

  • test_importlib: references
  • test_ast: references
  • test__interpchannels: references
  • test__interpreters: memory blocks
  • test_capi: memory blocks
  • test_capi: references
  • test__interpreters: references
  • test_importlib: memory blocks
  • test_ast: memory blocks
  • test__interpchannels: memory blocks

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "<string>", line 10, in <module>
  File "<frozen importlib._bootstrap>", line 813, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1046, in create_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
ImportError: module _test_module_state_shared does not support loading in subinterpreters
xpected failure
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Sep 26, 2024

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM64 MacOS M1 Refleaks NoGIL 3.x has failed when building commit d929652.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1368/builds/1855) and take a look at the build logs.
  4. Check if the failure is related to this commit (d929652) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1368/builds/1855

Failed tests:

  • test__interpchannels
  • test_capi
  • test__interpreters
  • test_ast
  • test_importlib

Test leaking resources:

  • test_importlib: references
  • test_ast: references
  • test__interpchannels: references
  • test__interpreters: memory blocks
  • test_capi: memory blocks
  • test_capi: references
  • test__interpreters: references
  • test_importlib: memory blocks
  • test_ast: memory blocks
  • test__interpchannels: memory blocks

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "<string>", line 10, in <module>
  File "<frozen importlib._bootstrap>", line 813, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1046, in create_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
ImportError: module _test_module_state_shared does not support loading in subinterpreters
xpected failure
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@rhettinger

rhettinger commented Sep 26, 2024

Copy link
Copy Markdown
Contributor

@dg-pb Congratulations on landing a new feature.

tim-one reacted with thumbs up emoji dg-pb and JelleZijlstra reacted with hooray emoji
All reactions
  • 👍 1 reaction
  • 🎉 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@dg-pb

dg-pb commented Sep 26, 2024

Copy link
Copy Markdown
Contributor Author

Thank you @rhettinger.

And thank you everyone for your help. Especially @serhiy-storchaka.

I hope this will prove to be a useful feature.

Was pleasure working with you all.

tim-one reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@dg-pb
dg-pb deleted the implement-119127-again branch September 26, 2024 05:02
@Eclips4 Eclips4 mentioned this pull request Sep 26, 2024
Reference leaks on current main #124586
Closed
@vstinner

vstinner commented Sep 26, 2024

Copy link
Copy Markdown
Member

The deallocator which marks the Placeholder as an immortal object is causing memory leaks: #124586. I propose to use a regular singleton by just storing a strong reference in the module state: #124601. My PR fix the leak. I don't think that immortal object is needed here.

rhettinger reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@dg-pb dg-pb mentioned this pull request Sep 27, 2024
functools.partialmethod simplification #124652
Open
@johnslavik

johnslavik commented Oct 3, 2024

Copy link
Copy Markdown
Member

I'm a very often user of partial. This looks so useful, thank you!

dg-pb reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@dg-pb dg-pb mentioned this pull request Oct 9, 2024
gh-125028: Prohibit placeholders in partial keywords #125197
Closed
donbarbos added a commit to donbarbos/typeshed that referenced this pull request Feb 25, 2026
@donbarbos
[functools] Add __slots__
3fc1eeb
Python 3.10: https://github.com/python/cpython/blob/3.10/Lib/functools.py
Changed PR: python/cpython#119827
Python 3.14: https://github.com/python/cpython/blob/3.14/Lib/functools.py
@donbarbos donbarbos mentioned this pull request Feb 25, 2026
[functools] Add __slots__ python/typeshed#15464
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rhettinger rhettinger rhettinger left review comments
@serhiy-storchaka serhiy-storchaka serhiy-storchaka left review comments
@zware zware zware left review comments
@picnixz picnixz picnixz left review comments
@erlend-aasland erlend-aasland erlend-aasland left review comments
@1st1 1st1 Awaiting requested review from 1st1
@asvetlov asvetlov Awaiting requested review from asvetlov
@gvanrossum gvanrossum Awaiting requested review from gvanrossum
@willingc willingc Awaiting requested review from willingc
+1 more reviewer
@XuehaiPan XuehaiPan XuehaiPan requested changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@rhettinger rhettinger

Labels

sprint

Projects

Sprint
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

10 participants

@dg-pb @picnixz @rhettinger @serhiy-storchaka @bedevere-bot @vstinner @johnslavik @zware @erlend-aasland @XuehaiPan
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.