Description

UltimateLobster

opened on Jul 24, 2022

Issue body actions

Bug report

Hello there, this is my first bug report so apologies if there's something I miss.

Consider the following code:

import pickle

classmethod_descriptor = dict.__dict__["fromkeys"]
unpickled_classmethod_descriptor = pickle.loads(pickle.dumps(classmethod_descriptor, protocol=5))

While it passes without raising any errors, classmethod_descriptor differs from unpickled_classmethod_descriptor. Instead of the original descriptor, the pickling process yields the function itself:

>>> classmethod_descriptor
<method 'fromkeys' of 'dict' objects>

>>> unpickled_classmethod_descriptor
<function dict.fromkeys(iterable, value=None, /)>

Manually calling the descriptor's __reduce_ex__ allows us to see the reason:

>>> classmethod_descriptor.__reduce_ex__(5)
(<function getattr>, (dict, 'fromkeys'))

This behavior differs from python implemented classmethods:

import pickle

class A:
    @classmethod
    def method(cls):
        pass

classmethod_descriptor = A.__dict__["method"]

# Any of these lines will raise "TypeError: cannot pickle 'classmethod' object"
pickle.dumps(classmethod_descriptor, protocol=5)
classmethod_descriptor.__reduce_ex__(5)

Since it fails with normal classmethods, I would expect it to fail with their c counterparts as well so their behavior would match.
Either that, or having both of them succeed during the pickling process, in which case, __reduce_ex__ should return a valid result in both cases.

The bug occurs with other c implemented class methods as well:

import pickle

from datetime import datetime

classmethod_descriptor = datetime.__dict__["now"]

# Returns False
pickle.loads(pickle.dumps(classmethod_descriptor, 5)) == classmethod_descriptor

# Returns True
pickle.loads(pickle.dumps(classmethod_descriptor, 5)) == datetime.now

Your environment

Python Versions:
I've tested it with both 3.10.5 and 3.11.04b
OS - Windows 10 Pro 64 bit

Activity

1. UltimateLobster

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Jul 24, 2022
2. UltimateLobster commented on Aug 23, 2022

   

   UltimateLobster

   on Aug 23, 2022

   Author

   More actions

   #96162

   Opened a PR to solve the issue
3. serhiy-storchaka

   linked a pull request that will close this issuegh-95196: Fixed instances of types.ClassMethodDescriptorType not being pickled correctly #96162on Aug 28, 2022
4. serhiy-storchaka

   added this to Pickle and copy issues 🥒on Aug 28, 2022
5. serhiy-storchaka commented on Aug 29, 2022

   

   serhiy-storchaka

   on Aug 29, 2022

   Member

   More actions

   Since this feature never worked correctly, and pickling of Python implemented classmethod descriptors and C and Python implemented staticmethod descriptors is not supported, and there are no common use cases for this (otherwise we would get such report years ago), and the correct implementation is not trivial if not use eval(), I thing that the best way to fix this is to disable it. There are no existing tests for this, and it looks that this code was added by mistake and was never tested.
6. serhiy-storchaka

   added a commit that references this issue on Aug 29, 2022

   pythongh-95196: Disable incorrect pickling of the C implemented class…

   abe2e5a
7. serhiy-storchaka

   added a commit that references this issue on Oct 5, 2022

   gh-95196: Disable incorrect pickling of the C implemented classmethod…

   77f0249
8. miss-islington

   added 2 commits that reference this issue on Oct 5, 2022

   pythongh-95196: Disable incorrect pickling of the C implemented class…

   b500b3c

   pythongh-95196: Disable incorrect pickling of the C implemented class…

   209ef4a
9. serhiy-storchaka

   closed this as completedon Oct 5, 2022
10. serhiy-storchaka

    moved this to Done in Pickle and copy issues 🥒on Oct 5, 2022
11. miss-islington

    added 2 commits that reference this issue on Oct 5, 2022

    gh-95196: Disable incorrect pickling of the C implemented classmethod…

    d108eeb

    gh-95196: Disable incorrect pickling of the C implemented classmethod…

    2b248b0
12. carljm

    added a commit that references this issue on Oct 6, 2022

    Merge branch 'main' into dictwatch

    4bb5d9d
13. mpage

    added a commit that references this issue on Oct 11, 2022

    pythongh-95196: Disable incorrect pickling of the C implemented class…

    a47df46
14. danigm

    added a commit that references this issue on Oct 28, 2022

    Move builtin classmethod_descriptor to a different test

    9b33280

Issue actions