GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-77188: Add support for pickling private methods and nested classes by ZackerySpytz · Pull Request #21480 · python/cpython

Conversation

@ZackerySpytz

@ZackerySpytz ZackerySpytz commented Jul 15, 2020 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

https://bugs.python.org/issue33007

#77188

  • Issue: Objects referencing private-mangled names do not roundtrip properly under pickling. #77188

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@ZackerySpytz
bpo-33007: Name-mangled private methods don't roundtrip when pickling
92f5a27
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Jul 15, 2020
@bedevere-bot bedevere-bot added the awaiting review label Jul 15, 2020
MojoVampire
MojoVampire approved these changes Oct 16, 2020
View reviewed changes
@bedevere-bot bedevere-bot added awaiting core review and removed awaiting review labels Oct 16, 2020
@anntzer anntzer mannequin mentioned this pull request Apr 10, 2022
Objects referencing private-mangled names do not roundtrip properly under pickling. #77188
Closed
@ezio-melotti ezio-melotti removed the CLA signed label Jul 13, 2022
nanjekyejoannah
nanjekyejoannah reviewed Oct 7, 2022
View reviewed changes

@nanjekyejoannah nanjekyejoannah left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, please fix the conflict for merge

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@furkanonder

furkanonder commented Nov 21, 2022

Copy link
Copy Markdown
Contributor

@ZackerySpytz Are you gonna continue this task? @nanjekyejoannah You can see the edited PR for this issue.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ZackerySpytz

ZackerySpytz commented Nov 22, 2022

Copy link
Copy Markdown
Contributor Author

Yes, I will update this PR.

nanjekyejoannah reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ZackerySpytz
Merge remote-tracking branch 'upstream/main' into bpo-33007-pickle-ma…
dfad2c0
…ngle-roundtrip
@kumaraditya303 kumaraditya303 added type-bug An unexpected behavior, bug, or error needs backport to 3.10 needs backport to 3.11 only security fixes labels Dec 17, 2022
@python python deleted a comment from netlify Bot Dec 17, 2022
@kumaraditya303
kumaraditya303 requested a review from serhiy-storchaka December 17, 2022 07:02
@serhiy-storchaka

serhiy-storchaka commented Dec 17, 2022

Copy link
Copy Markdown
Member

In general, LGTM.

But how much it affects the performance? Could you please run some microbenchmarks to measure the changes in the pickling time for non-private methods? If the difference is insignificant, it is good, but if it is large, it could be worth to optimize the code for non-private methods.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk hugovk removed the needs backport to 3.10 label Apr 7, 2023
serhiy-storchaka added 2 commits November 29, 2023 16:53
@serhiy-storchaka
Merge branch 'main' into bpo-33007-pickle-mangle-roundtrip
944d540
@serhiy-storchaka
Optimize.
90cf9b3
@erlend-aasland erlend-aasland changed the title bpo-33007: Name-mangled private methods don't roundtrip when pickling gh-77188: Name-mangled private methods don't roundtrip when pickling Jan 5, 2024
@erlend-aasland
Merge branch 'main' into bpo-33007-pickle-mangle-roundtrip
6444c67
@serhiy-storchaka serhiy-storchaka added needs backport to 3.12 only security fixes needs backport to 3.13 only security fixes and removed needs backport to 3.11 only security fixes labels May 9, 2024
vyrjana added a commit to vyrjana/lmfit-py that referenced this pull request Nov 23, 2024
@vyrjana
Renamed the Minimizer.__jacobian method to avoid a bug related to pic…
3ba95e8
…kling

- Attempting to make use of multiprocessing.Pool while providing, e.g., Minimizer.least_squares a function for calculating the Jacobian did not work prior to the renaming of the method due to an issue related to pickling and unpickling of private methods. See the (currently) open issue (python/cpython#77188) and pull request (python/cpython#21480) for more information.
newville pushed a commit to lmfit/lmfit-py that referenced this pull request Dec 5, 2024
@vyrjana
Fix issues related to providing a Jacobian function (#973)
6f8551a
* Updated the Minimizer.least_squares method to wrap the Jacobian function
- The Jacobian function provided by the user is now handled similarly to how it is handled in, e.g., Minimizer.leastsq.
- If the Jacobian function is called 'Dfun' in the keyword arguments, then it is also accepted (Minimizer.scalar_minimize already does this).

* Updated the Minimizer.__jacobian method signature to fix exception
- The signature now matches the signature of Minimizer.__residual. This was changed because the following exception was being raised whenever the user provided a function for calculating the Jacobian: "TypeError: Minimizer.__jacobian() got an unexpected keyword argument 'apply_bounds_transformation'".

* Updated the Minimizer.__jacobian method to use the new parameter
- The updating of the parameter values is now similar to how it is done in Minimizer.__residual.

* Updated test_least_squares_jacobian_types to handle the wrapped Jacobian function
- The params argument of the jac_array function is now an actual Parameters object like it is in the f function, which means that the parameters can be accessed via string keys.

* Updated the coerce_float64 function
- Added explicit support for sparse matrices (e.g., Block Sparse Row) and LinearOperator objects. This change is needed because the updated test_least_squares_jacobian_types would otherwise fail. Similar to how these types are handled when calculating the Hessian matrix in Minimizer.least_squares.

* Renamed the Minimizer.__jacobian method to avoid a bug related to pickling
- Attempting to make use of multiprocessing.Pool while providing, e.g., Minimizer.least_squares a function for calculating the Jacobian did not work prior to the renaming of the method due to an issue related to pickling and unpickling of private methods. See the (currently) open issue (python/cpython#77188) and pull request (python/cpython#21480) for more information.

* Added a test for when the "least_squares" method is provided a function for the Jacobian
- Verifies that a) the fitted parameters have the expected values, b) the jac function is actually called, and c) that there are indeed fewer function evaluations as a result.

* Updated the docstring of Minimizer._jacobian
- The docstring is updated to include the parameters and return values.

* Added tests for (un)pickling Minimizer when a Jacobian function is provided

* Updated timeout value in test_jacobian_with_forkingpickler
- Increased the timeout value so that slow runners can actually succeed instead of timing out. If no timeout was specified, then a failing test could end up hanging instead.

* Added an example that demonstrates the benefits of providing a Jacobian function

* Updated the list of contributors
@hugovk hugovk removed the needs backport to 3.12 only security fixes label Apr 25, 2025
@serhiy-storchaka serhiy-storchaka added the needs backport to 3.14 bugs and security fixes label May 8, 2025
gpshead
gpshead reviewed Jan 6, 2026
View reviewed changes
Comment thread Objects/classobject.c Outdated
(len = PyUnicode_GET_LENGTH(funcname)) > 2 &&
PyUnicode_READ_CHAR(funcname, 0) == '_' &&
PyUnicode_READ_CHAR(funcname, 1) == '_' &&
!(PyUnicode_READ_CHAR(funcname, len-1) == '_' &&

@gpshead gpshead Jan 6, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would also fail to trigger on methods named ___ and ____ (3 and 4 underscores). it seems rather pedantic to care, but perhaps add a len > 4 check before checking for trailing _s and cover def ____(self): in the regression test?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
serhiy-storchaka added 3 commits January 6, 2026 14:00
@serhiy-storchaka
Merge branch 'main' into bpo-33007-pickle-mangle-roundtrip
3c43a5b
@serhiy-storchaka
Refactor tests.
1dbe82a
@serhiy-storchaka
Add tests for other types of methods.
6317677
@serhiy-storchaka serhiy-storchaka removed needs backport to 3.13 only security fixes needs backport to 3.14 bugs and security fixes type-bug An unexpected behavior, bug, or error labels Jan 6, 2026
@serhiy-storchaka
Merge branch 'main' into bpo-33007-pickle-mangle-roundtrip
00fb920
@serhiy-storchaka serhiy-storchaka self-assigned this Jan 21, 2026
serhiy-storchaka added 2 commits January 21, 2026 19:27
@serhiy-storchaka
Merge branch 'main' into bpo-33007-pickle-mangle-roundtrip
62f6c64
@serhiy-storchaka
Add support for class methods, unbound methods, static methods and ne…
e11fc8a
…sted classes.
@serhiy-storchaka
serhiy-storchaka requested review from AA-Turner, JelleZijlstra and carljm as code owners January 22, 2026 10:25
@serhiy-storchaka serhiy-storchaka changed the title gh-77188: Name-mangled private methods don't roundtrip when pickling gh-77188: Add support for pickling private methods and nested classes Jan 22, 2026
@serhiy-storchaka

serhiy-storchaka commented Jan 22, 2026

Copy link
Copy Markdown
Member

Now it supports pickling class methods, static methods, unbound methods and nested classes. And most likely many private singletons.

It may not work for private methods defined in a metaclass, but this is very uncommon case.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@serhiy-storchaka
Merge branch 'main' into bpo-33007-pickle-mangle-roundtrip
1271c3f
@serhiy-storchaka
serhiy-storchaka enabled auto-merge (squash) February 5, 2026 19:23
@serhiy-storchaka
serhiy-storchaka merged commit 01a1dd2 into python:main Feb 5, 2026
47 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting core review label Feb 5, 2026
thunder-coding pushed a commit to thunder-coding/cpython that referenced this pull request Feb 15, 2026
@ZackerySpytz @serhiy-storchaka @thunder-coding
pythongh-77188: Add support for pickling private methods and nested c…
672ba32
…lasses (pythonGH-21480)

Co-authored-by: Serhiy Storchaka <[email protected]>
ljfp pushed a commit to ljfp/cpython that referenced this pull request Apr 25, 2026
@ZackerySpytz @serhiy-storchaka @ljfp
pythongh-77188: Add support for pickling private methods and nested c…
c321be9
…lasses (pythonGH-21480)

Co-authored-by: Serhiy Storchaka <[email protected]>
@serhiy-storchaka serhiy-storchaka removed their assignment Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gpshead gpshead gpshead left review comments
@nanjekyejoannah nanjekyejoannah nanjekyejoannah left review comments
@MojoVampire MojoVampire MojoVampire approved these changes
@serhiy-storchaka serhiy-storchaka Awaiting requested review from serhiy-storchaka
@JelleZijlstra JelleZijlstra Awaiting requested review from JelleZijlstra JelleZijlstra is a code owner
@carljm carljm Awaiting requested review from carljm carljm is a code owner
@AA-Turner AA-Turner Awaiting requested review from AA-Turner AA-Turner is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

12 participants

@ZackerySpytz @furkanonder @serhiy-storchaka @gpshead @MojoVampire @nanjekyejoannah @hugovk @the-knights-who-say-ni @ezio-melotti @bedevere-bot @kumaraditya303 @erlend-aasland
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.