GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-154594: Fix deepcopy memo lookup when __deepcopy__ returns None by tonghuaroot · Pull Request #154595 · python/cpython

Conversation

@tonghuaroot

@tonghuaroot tonghuaroot commented Jul 24, 2026 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

PR #138429 (gh-132657) replaced the memo-miss sentinel from _nil = [] to None for free-threading performance. But None is a valid deepcopy result, so memo.get(d, None) can't distinguish "not found" from "found None". This skips memoization for None-valued results, causing redundant __deepcopy__ calls.

Fix: use a private _MEMO_MISS = object() sentinel. Immortalized objects have no refcount contention, so the FT benefit is preserved.

  • Issue: copy.deepcopy memo lookup fails when __deepcopy__ returns None #154594

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@tonghuaroot
pythongh-154594: Fix deepcopy memo lookup when __deepcopy__ returns None
17c0eb9
@bedevere-app bedevere-app Bot mentioned this pull request Jul 24, 2026
copy.deepcopy memo lookup fails when __deepcopy__ returns None #154594
Closed
@bedevere-app bedevere-app Bot added the awaiting review label Jul 24, 2026
eendebakpt
eendebakpt reviewed Jul 29, 2026
View reviewed changes

@eendebakpt eendebakpt left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small comments, but I am leaning more towards the approach suggested in the corresponding issue.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/copy.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_copy.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@tonghuaroot
Use 'd in memo' lookup instead of a sentinel
abc037e
Addresses review: clearer, no sentinel, and the test treats memo as opaque.
@tonghuaroot

tonghuaroot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Switched to if d in memo: return memo[d] as suggested. Drops the sentinel, and the test no longer touches memo.

Microbenchmark (5M lookups): d in memo is ~6 ns faster on a hit and ~22 ns on a miss than memo.get(d, None); full deepcopy is within noise.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pochmann

pochmann commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

d in memo is ~6 ns faster on a hit and ~22 ns on a miss than memo.get(d, None)

Faster in both cases? Did you measure purely d in memo and memo.get(d, None)? Or did you include the other differences as well?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tonghuaroot

tonghuaroot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

The earlier numbers included the surrounding statements. Measuring just the lookup, each returning the value (best of 5 runs of 10M):

                                  hit (present)   miss (absent)
memo.get(d, None)                    51 ns           60 ns
memo[d] if d in memo else None       55 ns           49 ns

So d in memo is about 4 ns slower on a hit and 11 ns faster on a miss. In deepcopy each object is looked up once, so misses are the common case, and full deepcopy timings are within noise either way.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pochmann

pochmann commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Ok if the surrounding stuff was properly included, then that's good. This new comparison for "just the lookup" is not, it's missing the is not None check in an if-context and the store/load of y.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pochmann

pochmann commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Anyway, I agree that misses are probably the common case, and I'm not surprised if full deepcopy timings are within noise either way, as deepcopy is rather slow.

tonghuaroot reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

eendebakpt
eendebakpt reviewed Jul 30, 2026
View reviewed changes

@eendebakpt eendebakpt left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small nits, but this looks good to me.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Misc/NEWS.d/next/Library/2026-07-24-00-02-00.gh-issue-154594.Xk7mQ2.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/test/test_copy.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@tonghuaroot
Apply review suggestions: shorten NEWS, use class attribute in test
37b18f0
sreehariannam added a commit to sreehariannam/cpython that referenced this pull request Aug 4, 2026
@sreehariannam
Switch deepcopy memo check to d in memo
6a7529d
pochmann benchmarked this issue's candidate fixes on pythongh-154594 and found
the try/except form is ~3.5x slower on a memo miss (the common case in
real deepcopy workloads) despite being fastest on a hit. `d in memo` ties
or beats the current `is not None` check on both hit and miss, and
matches where the competing PR (pythongh-154595) landed after the same
discussion.
eendebakpt
eendebakpt approved these changes Aug 4, 2026
View reviewed changes

@eendebakpt eendebakpt left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code is a bit cleaner, a bit faster and it solves a (minor) bug.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Aug 4, 2026
eendebakpt added 2 commits August 4, 2026 20:10
@eendebakpt
Merge branch 'main' into fix-deepcopy-none-sentinel
a39f884
@eendebakpt
Merge branch 'main' into fix-deepcopy-none-sentinel
6857381
@eendebakpt
eendebakpt merged commit 5539c2a into python:main Sep 17, 2026
51 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting merge label Sep 17, 2026
@eendebakpt eendebakpt added 3.15 bugs and security fixes needs backport to 3.15 bugs and security fixes and removed 3.15 bugs and security fixes labels Oct 12, 2026
@miss-islington-app

miss-islington-app Bot commented Oct 12, 2026

Copy link
Copy Markdown

Thanks @tonghuaroot for the PR, and @eendebakpt for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app

bedevere-app Bot commented Oct 12, 2026

Copy link
Copy Markdown

GH-159255 is a backport of this pull request to the 3.15 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Oct 12, 2026
eendebakpt added a commit that referenced this pull request Oct 12, 2026
@miss-islington @tonghuaroot @eendebakpt
[3.15] gh-154594: Fix deepcopy memo lookup when __deepcopy__ returns …
9094a89
…None (GH-154595) (#159255)

gh-154594: Fix deepcopy memo lookup when __deepcopy__ returns None (GH-154595)

* gh-154594: Fix deepcopy memo lookup when __deepcopy__ returns None

* Use 'd in memo' lookup instead of a sentinel

Addresses review: clearer, no sentinel, and the test treats memo as opaque.

* Apply review suggestions: shorten NEWS, use class attribute in test

---------
(cherry picked from commit 5539c2a)

Co-authored-by: tonghuaroot (童话) <[email protected]>
Co-authored-by: Pieter Eendebak <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@eendebakpt eendebakpt eendebakpt approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@tonghuaroot @pochmann @eendebakpt
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.