GeezerVison is showing github.com — Deprecate and schedule removal of collections.abc.ByteString and typing.ByteString · Issue #91896 · python/cpython
Deprecate and schedule removal of collections.abc.ByteString and typing.ByteString #91896DescriptionThe current docstring of collections.abc.ByteString is:
"""This unifies bytes and bytearray.
XXX Should add all their methods.
"""
Let's do that last thing. This will be useful for typing code that accepts both bytes and bytearray, especially with my proposal in PEP-688 to make bytes no longer acceptable as a shortcut for bytearray in the type system.
cc @rhettinger for collections.abc
Linked PRs
Activityrhettinger commented on Apr 25, 2022 The general rule is that we can never add methods to ABCs once they are published. The purpose of an ABC is promise that a minimal set of methods are available. If isinstance(x, SomeABC) returns true, the methods in the ABC are expected to be present. Adding methods presents a problem for existing code that has registered a class as being compliant with SomeABC. If it lacks the new methods, then the registered promise is invalid.
Perhaps because this is an empty ABC that is almost entirely unused, this might be okay. On the other hand, because it is an empty ABC that is almost entirely unused, there is almost nothing to be gained by adding it. AFAICT no one has ever asked for or needed this since the ABC was added 15 years ago. That suggests that there is no problem to be solved here. JelleZijlstra commented on Apr 25, 2022
Perhaps because this is an empty ABC that is almost entirely unused, this might be okay.
Right, the ABC is hardly useful with no methods. I searched on grep.app and found no uses of ByteString.register except for one of memoryview in what appears to be an old copy of typing.py.
On the other hand, because it is an empty ABC that is almost entirely unused, there is almost nothing to be gained by adding it. AFAICT no one has ever asked for or needed this since the ABC was added 15 years ago. That suggests that there is no problem to be solved here.
Well, I'm asking for it now. The use case is type annotating code that accepts both bytes and bytearray. rhettinger commented on Apr 25, 2022 If you were to do it correctly, the procedure would be to subclass ByteString and add the new methods in a subclass. Loosely, this is similar to why we had to add IterableUserDict in Python 2 rather than modifying the existing UserDict.
To annotate code that accepts both bytes or bytearray wouldn't the correct way be to write: b: bytes | bytearray just like you would with tuple | list?
One other thought: the name ByteString wasn't very good to begin as is suggests bytes | str. If you really must do this, it would be better to create a new, well-named ABC with the requisite methods and even leave the old ABC around (it isn't hurting anything) or deprecate it. rhettinger commented on Apr 25, 2022 I forgot to mention that the docs promise something narrower that the bytes/bytearray API. It says, "ABCs for read-only and mutable sequences". There is no promise of the extra methods found in bytes or bytearray. Existing code reasonably by subclassing from ByteString and not providing or expecting any of the stringlike methods.
All around, I think this proposal is a contract violation and that a new ABC should be created. A scan on grep.app is insufficient to show this won't be a breaking change (most of the world's Python code isn't publicly visible). JelleZijlstra commented on Apr 25, 2022
I forgot to mention that the docs promise something narrower that the bytes/bytearray API. It says, "ABCs for read-only and mutable sequences". There is no promise of the extra methods found in bytes or bytearray. Existing code reasonably by subclassing from ByteString and not providing or expecting any of the stringlike methods.
That documentation is for Sequence, MutableSequence, and ByteString together (https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence). "Read-only and mutable sequences" is a good description for the first two, but the documentation really doesn't tell me what ByteString is good for.
ByteString is also documented at https://docs.python.org/3/library/typing.html#typing.ByteString, but that documentation has a couple of problems:
memoryview is not in fact registered as a ByteStringByteString is not in fact generic (and it doesn't make sense for it to be generic)
All around, I think this proposal is a contract violation and that a new ABC should be created. A scan on grep.app is insufficient to show this won't be a breaking change (most of the world's Python code isn't publicly visible).
That's a reasonable point. If we can't use the existing ByteString ABC for bytes | bytearray, I don't think it's worth creating a new ABC—as you said above, the union annotation is good enough.
But if we keep ByteString as is, with no methods, I have no idea what it's useful for. We occasionally get people trying to use the ABC in type annotations, so the current state causes confusion.
Perhaps we could deprecate ByteString, or explicitly document its limited use. rhettinger commented on Apr 25, 2022
Perhaps we could deprecate ByteString, or explicitly document its limited use.
I vote for deprecation because the name is bad (implying bytes | str) and it would just be a continuing point of confusion. Reacted by Alex Waygood, Jelle Zijlstra, Martin DeMello and Sebastian Rittau changed the title [-]Give collections.abc.ByteString some methods[/-] [+]Deprecate collections.abc.ByteString[/+] on Apr 25, 2022serhiy-storchaka commented on Apr 26, 2022 The name is good to me. It implies the bytes-like object with str methods (find(), lower(), isspace(), etc).
The terms "buffer", "bytes-like", "bytestring" and "bytes string" are used loosely in the documentation, but there are several meanings of bytes-likeness:
- Supports the buffer protocol.
- Additionally supports
len() which returns the size in bytes.
- Additionally supports indexing.
- Has most methods of
str (except encode() of course).
Unfortunately there are no strongly defined terms and corresponding abstract classes, protocols or types in the code to express the requirements precisely. ankith26 commented on Aug 17, 2022 Hi! I'm one of the contributors to the pygame project.
I'm not sure whether this is the best best place to be asking, but it is relevant to the usage of ByteString.
So we have a function implemented with the python C API, and it uses y#, which according to the docs is a format string for a generic "bytes-like" sized object. The term "bytes-like" is defined here in the glossary which gives me the impression that any function using y# must accept a wide range of "byte-like" objects. Simply using bytes | bytearray would be narrow, and probably miss some kinds of objects. The same also mentions that "bytes-like object" is an object supporting the C level buffer protocol (which, is also not fully exposed on the python end in my understanding)
I was looking for a suitable ABC to typehint this, and the closest thing I could find that already exists is ByteString.
The next closest thing I found is typing.SupportsBytes which by the naming, gives me the impression that this is what I'm looking for, but weirdly enough, the concrete bytes object itself does not confirm to this protocol (due to missing the __bytes__ method)
I suppose a set of ABCs for the buffer protocol (if this were to be added) would be the closest replacement to ByteString, and would also work for my usecase (typing the C level y# arg format) TeamSpen210 commented on Aug 17, 2022 The problem with buffers is that it doesn't have a visible Python API, so the ABC would be really weird, being entirely empty. See previous discussion at python/typing#593 and then PEP 688. added 2 commits that reference this issue on May 9, 2023gh-91896: Improve visibility of `ByteString` deprecation warnings (#1… added a commit that references this issue on May 13, 2023gh-91896: Revert some very noisy DeprecationWarnings for `ByteString` (… changed the title [-]Deprecate collections.abc.ByteString[/-] [+]Deprecate collections.abc.ByteString and typing.ByteString[/+] on Jul 14, 2023changed the title [-]Deprecate collections.abc.ByteString and typing.ByteString[/-] [+]Deprecate and schedule removal of collections.abc.ByteString and typing.ByteString[/+] on Jul 14, 2023
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|