GeezerVison is showing github.com — Support pickling slots in subclasses of common classes · Issue #70766 · python/cpython
Support pickling slots in subclasses of common classes #70766Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2022-04-06.17:02:02.310>
created_at = <Date 2016-03-17.08:43:06.421>
labels = ['extension-modules', 'type-feature', 'library', '3.11']
title = 'Support pickling slots in subclasses of common classes'
updated_at = <Date 2022-04-07.14:57:28.953>
user = 'https://github.com/serhiy-storchaka' bugs.python.org fields:
activity = <Date 2022-04-07.14:57:28.953>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2022-04-06.17:02:02.310>
closer = 'serhiy.storchaka'
components = ['Extension Modules', 'Library (Lib)']
creation = <Date 2016-03-17.08:43:06.421>
creator = 'serhiy.storchaka'
dependencies = []
files = ['42183', '43462', '43716']
hgrepos = []
issue_num = 26579
keywords = ['patch']
message_count = 5.0
messages = ['261903', '268830', '270403', '416888', '416931']
nosy_count = 7.0
nosy_names = ['rhettinger', 'pitrou', 'vstinner', 'alexandre.vassalotti', 'r.david.murray', 'berker.peksag', 'serhiy.storchaka']
pr_nums = ['2821']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue26579'
versions = ['Python 3.11'] Linked PRs
Activityserhiy-storchaka commented on Mar 17, 2016 Pickling and copying instances of subclasses of some basic classes pickles and copies instance attributes. Example:
>>> class BA(bytearray):
... pass
...
>>> b = BA(b'abc')
>>> b.x = 10
>>> c = copy.copy(b)
>>> c.x
10
>>> c = pickle.loads(pickle.dumps(b))
>>> c.x
10 But this doesn't work if attributes are saved not in instance dictionary, but in slots.
>>> class BA(bytearray):
... __slots__ = ('x',)
...
>>> b = BA(b'abc')
>>> b.x = 10
>>> c = copy.copy(b)
>>> c.x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: x
>>> c = pickle.loads(pickle.dumps(b))
>>> c.x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: xSince using __slots__ is implementation detail, this failure can be considered as a bug.
Proposed patch adds support of pickling and copying slots in subclasses of all classes that already support pickling and copying non-slot attributes. It is backward compatible, classes with slots can be unpickled on older Python versions without slots. Affected classes: bytearray, set, frozenset, weakref.WeakSet, collections.OrderedDict, collections.deque, datetime.tzinfo.
The patch adds the copyreg._getstate() function for Python classes and exposes the _PyObject_GetState() function for extension classes. An alternative (and simpler for end user) solution would be to add default implementation as object.__getstate__(). But this is not easy to reject non-pickleable classes (bpo-22995) in this case, since __getstate__ is looked up as instance attribute, not as other special methods. serhiy-storchaka commented on Jun 18, 2016 Synchronized with current sources. serhiy-storchaka commented on Jul 14, 2016 An alternative way is to expose a default state as object.__getstate__(). It is more efficient since it is implemented in C. Following patch implements this approach. serhiy-storchaka commented on Apr 6, 2022 vstinner commented on Apr 7, 2022 gpshead commented on Jul 13, 2023 serhiy-storchaka commented on Jul 14, 2023 I don't know what can be suggested as a general workaround. Each problem requires an individual solution, depending on why that code was broken by that change. Reacted by Gregory P. Smith lwinkler commented on Aug 23, 2023 This also creates a bug in pyyaml and ruamel.yaml. It seems that for an empty class, the new getstate method returns None. This causes the serialization to crash.
@serhiy-storchaka : Is there a good reason why getstate returns None for an empty class instead of an empty directory ?
Minimal code:
class Empty:
pass
empty = Empty()
print("getstate", empty.__getstate__())
prints: getstate None serhiy-storchaka commented on Aug 23, 2023
@serhiy-storchaka : Is there a good reason why getstate returns None for an empty class instead of an empty directory ?
Because pickle.dumps() creates more compact pickle if __getstate__() returns None. Also, it saves time in __getstate__() for creating an empty dict and saves time in pickle.loads() for calling __setstate__().
Add def __getstate__(self): return {} to an empty class and compare the result of pickle.dumps(). Reacted by Laurent Winkler and Gregory P. Smith nitzmahone commented on Aug 23, 2023 Accounting for this in PyYAML shouldn't be a big deal- IIUC None has always been a documented valid response from __getstate__() anyway, so we need to be able to handle and preserve that properly. Reacted by Laurent Winkler
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|