GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||
GeezerVison is showing github.com — gh-85160: Reduce memory usage of `singledispatchmethod` when owner instances cannot be weakref'd by AlexWaygood · Pull Request #107706 · python/cpython Conversation
|
|
Alternatively, you can get rid of diff --git a/Lib/functools.py b/Lib/functools.py index 2a8a69b3c5..5622eb6467 100644 --- a/Lib/functools.py +++ b/Lib/functools.py @@ -935,7 +935,6 @@ def __init__(self, func): self.dispatcher = singledispatch(func) self.func = func self._method_cache = weakref.WeakKeyDictionary() - self._all_weakrefable_instances = True def register(self, cls, method=None): """generic_method.register(cls, func) -> func @@ -945,11 +944,11 @@ def register(self, cls, method=None): return self.dispatcher.register(cls, func=method) def __get__(self, obj, cls=None): - if self._all_weakrefable_instances: + if self._method_cache: try: _method = self._method_cache[obj] except TypeError: - self._all_weakrefable_instances = False + self._method_cache = None except KeyError: pass else: @@ -963,7 +962,7 @@ def _method(*args, **kwargs): _method.register = self.register update_wrapper(_method, self.func) - if self._all_weakrefable_instances: + if self._method_cache is not None: self._method_cache[obj] = _method return _method It reduces memory even more, and in all cases. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Very nice. And the performance seems to be the same. The first line of def __get__(self, obj, cls=None): if self._method_cache is not None: |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Co-authored-by: Serhiy Storchaka <[email protected]>
951a87e to
3d8d4ba
Compare
|
Or you need to catch TypeError when save in the cache: if self._method_cache is not None: try: self._method_cache[obj] = _method except TypeError: self._method_cache = None |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
With this diff applied to my PR branch: Diff--- a/Lib/functools.py +++ b/Lib/functools.py @@ -945,7 +945,7 @@ def register(self, cls, method=None): return self.dispatcher.register(cls, func=method) def __get__(self, obj, cls=None): - if self._method_cache is not None: + if self._method_cache: try: _method = self._method_cache[obj] except TypeError: @@ -963,8 +963,10 @@ def _method(*args, **kwargs): _method.register = self.register update_wrapper(_method, self.func) - if self._method_cache is not None: + try: self._method_cache[obj] = _method + except TypeError: + pass return _method Benchmark scriptimport pyperf setup=""" from functools import singledispatch, singledispatchmethod class Test: @singledispatchmethod def go(self, item, arg): pass @go.register def _(self, item: int, arg): return item + arg class Slot: __slots__ = ('a', 'b') @singledispatchmethod def go(self, item, arg): pass @go.register def _(self, item: int, arg): return item + arg t = Test() s= Slot() """ runner = pyperf.Runner() runner.timeit(name="bench singledispatchmethod", stmt="""_ = t.go(1, 1)""", setup=setup ) runner.timeit(name="bench singledispatchmethod slots", stmt="""_ = s.go(1, 1)""", setup=setup ) |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Simpler |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
I mean that you keep |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
I still measure about a 14% overhead from this approach compared to my current PR. (Tested on a PGO-optimised non-debug build on Windows, with a quiet machine.) I'm inclined to stick with how it is currently -- it also feels like slightly cleaner code if we have only one |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
With the test for Anyway, your current code is better. I just explained why I intentionally did not use |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Yep. I'm also surprised there was that difference. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Thanks for the review, Serhiy! |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.