GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||
GeezerVison is showing github.com — gh-150942: Optimize PicklerMemoProxy.copy() with _PyDict_SetItem_Take2 by corona10 · Pull Request #151608 · python/cpython Conversation
|
|
cc @eendebakpt 1-2% consistenatly faster with simple replace logic, I think that it's worth to do this. import io import pickle import pyperf def make_proxy(n): buf = io.BytesIO() pk = pickle.Pickler(buf, protocol=pickle.HIGHEST_PROTOCOL) data = [object() for _ in range(n)] nested = {i: [object(), object()] for i in range(n // 2)} pk.dump(data) pk.dump(nested) proxy = pk.memo # PicklerMemoProxy # Keep a reference to the source pickler alive via closure. return pk, proxy def main(): runner = pyperf.Runner() for n in (1000, 20000): pk, proxy = make_proxy(n) entries = len(proxy.copy()) runner.bench_func( f"PicklerMemoProxy.copy/{entries}entries", proxy.copy, ) if __name__ == "__main__": main() |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
The change itself is ok (the dict is fresh, so no risk of concurrent threads mutating). But is the proxy used in any computations? I thought the proxy was only for human inspection. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Yeah, I also not sure how it will be practical |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
I am fine with merging though. The code looks cleaner (although this is subjective), and if we do not merge we risk someone else making a similar PR. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.