GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-150942: Optimize PicklerMemoProxy.copy() with _PyDict_SetItem_Take2 by corona10 · Pull Request #151608 · python/cpython

Conversation

@corona10

@corona10 corona10 commented Jun 17, 2026 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member
  • Issue: Improve performance by using reference stealing methods #150942

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@corona10
pythongh-150942: Optimize PicklerMemoProxy.copy() with _PyDict_SetIte…
147e099
…m_Take2
@corona10 corona10 added the skip news label Jun 17, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Jun 17, 2026
Improve performance by using reference stealing methods #150942
Open
@bedevere-app bedevere-app Bot added the awaiting core review label Jun 17, 2026
@corona10

corona10 commented Jun 17, 2026

Copy link
Copy Markdown
Member Author

cc @eendebakpt

1-2% consistenatly faster with simple replace logic, I think that it's worth to do this.

PicklerMemoProxy.copy/2505entries: Mean +- std dev: [base] 136 us +- 4 us -> [opt] 133 us +- 5 us: 1.02x faster
PicklerMemoProxy.copy/50005entries: Mean +- std dev: [base] 4.57 ms +- 0.06 ms -> [opt] 4.52 ms +- 0.05 ms: 1.01x faster

Geometric mean: 1.02x faster
import io
import pickle
import pyperf


def make_proxy(n):
    buf = io.BytesIO()
    pk = pickle.Pickler(buf, protocol=pickle.HIGHEST_PROTOCOL)
    data = [object() for _ in range(n)]
    nested = {i: [object(), object()] for i in range(n // 2)}
    pk.dump(data)
    pk.dump(nested)
    proxy = pk.memo  # PicklerMemoProxy
    # Keep a reference to the source pickler alive via closure.
    return pk, proxy


def main():
    runner = pyperf.Runner()

    for n in (1000, 20000):
        pk, proxy = make_proxy(n)
        entries = len(proxy.copy())
        runner.bench_func(
            f"PicklerMemoProxy.copy/{entries}entries",
            proxy.copy,
        )


if __name__ == "__main__":
    main()
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@corona10
corona10 requested a review from vstinner June 17, 2026 16:48
@eendebakpt

eendebakpt commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

The change itself is ok (the dict is fresh, so no risk of concurrent threads mutating).

But is the proxy used in any computations? I thought the proxy was only for human inspection.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@corona10

corona10 commented Jun 17, 2026

Copy link
Copy Markdown
Member Author

Yeah, I also not sure how it will be practical

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@eendebakpt

eendebakpt commented Jun 17, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Yeah, I also not sure how it will be practical

I am fine with merging though. The code looks cleaner (although this is subjective), and if we do not merge we risk someone else making a similar PR.

corona10 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@corona10
corona10 merged commit 8d7c6dc into python:main Jun 18, 2026
55 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting core review label Jun 18, 2026
pyctrl pushed a commit to pyctrl/cpython that referenced this pull request Jul 5, 2026
@corona10 @pyctrl
pythongh-150942: Optimize PicklerMemoProxy.copy() with _PyDict_SetIte…
bb90f37
…m_Take2 (pythongh-151608)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner Awaiting requested review from vstinner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@corona10 @eendebakpt
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.