GeezerVison is showing github.com — memoryview & ctypes: incorrect itemsize for empty array · Issue #76963 · python/cpython
memoryview & ctypes: incorrect itemsize for empty array #76963Labels 3.10 (EOL)end of lifeend of life3.8 (EOL)end of lifeend of life3.9 (EOL)end of lifeend of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)topic-ctypestype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2018-02-06.18:53:21.025>
labels = ['interpreter-core', 'type-bug', '3.8', '3.9', '3.10', 'ctypes']
title = 'memoryview & ctypes: incorrect itemsize for empty array'
updated_at = <Date 2020-05-23.15:47:49.464>
user = 'https://github.com/eric-wieser' bugs.python.org fields:
activity = <Date 2020-05-23.15:47:49.464>
actor = 'cheryl.sabella'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core', 'ctypes']
creation = <Date 2018-02-06.18:53:21.025>
creator = 'Eric.Wieser'
dependencies = []
files = []
hgrepos = []
issue_num = 32782
keywords = ['patch']
message_count = 9.0
messages = ['311740', '321290', '324705', '340230', '340231', '340232', '340235', '340236', '348004']
nosy_count = 8.0
nosy_names = ['terry.reedy', 'teoliphant', 'amaury.forgeotdarc', 'belopolsky', 'skrah', 'meador.inge', 'Eric.Wieser', 'Eric Wieser']
pr_nums = ['5576']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue32782'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10'] Linked PRs
Activityeric-wieser commented on Feb 6, 2018 MannequinAuthor More actions Take the following simple structure:
class Foo(ctypes.Structure):
_fields_ = [('f', ctypes.uint32_t)]And construct some arrays with it:
def get_array_view(N):
return memoryview((Foo * N)())In most cases, this works as expected, returning the size of one item:
>>> get_array_view(10).itemsize
4
>>> get_array_view(1).itemsize
4But when N=0, it returns the wrong result
>>> get_array_view(0).itemsize
0Which contradicts its .format, which still describes a 4-byte struct
>>> get_array_view(0).format
'T{>I:one:}'This causes a downstream problem in numpy:
>>> np.array(get_array_view(0))
RuntimeWarning: Item size computed from the PEP 3118 buffer format string does not match the actual item size.eric-wieser commented on Jul 8, 2018 MannequinAuthor More actions eric-wieser commented on Sep 6, 2018 MannequinAuthor More actions Pinging again, for lack of a clearer path forward terryjreedy commented on Apr 14, 2019 This issue is about the itemsize attribute of instances of the built-in memoryview class. Ctypes in only involved in providing format information. Hence the nosy additions.
On Win10 with 3.8, ctypes has no uint attributes. Using 'c_int32' instead, I see the same behavior (itemsize 0 for empty structure).
About itemsize, https://www.python.org/dev/peps/pep-3118/ says
"This is a storage for the itemsize (in bytes) of each element of the shared memory. It is technically un-necessary as it can be obtained using PyBuffer_SizeFromFormat, however an exporter may know this information without parsing the format string and it is necessary to know the itemsize for proper interpretation of striding. Therefore, storing it is more convenient and faster."
The first line could be seen as implying that itemsize is undefined if there are no items (and as justifying numbytes/numitems otherwise). The 0 return could be seen as equivalent to a None return from a python-coded function. If so, it is not a bug, and there might be code that would break if it is changed.
On the other hand, the next lines imply that itemsize is *usually*, though not necessarily, a cache for PyBuffer_SizeFromFormat. This could be seen as implying that in the absence of other information, the itemsize should be calculated from the format, making 0 a bug. changed the title [-]ctypes: memoryview gives incorrect PEP3118 itemsize for array of length zero[/-] [+]memoryview gives incorrect PEP3118 itemsize for empty array[/+] on Apr 14, 2019terryjreedy commented on Apr 14, 2019 This is actually about memoryview.itemsize within ctypes. changed the title [-]memoryview gives incorrect PEP3118 itemsize for empty array[/-] [+]memoryview & ctypes: incorrect PEP3118 itemsize for empty array[/+] on Apr 14, 2019terryjreedy commented on Apr 14, 2019 https://docs.python.org/3/library/stdtypes.html#typememoryview
says "itemsize The size in bytes of each element of the memoryview"
Revising the code example to use an empty array:
>>> import array, struct
>>> m = memoryview(array.array('H', [0])
>>> m.itemsize
2
I agree that itemsize should also be non-zero for ctype formats.changed the title [-]memoryview & ctypes: incorrect PEP3118 itemsize for empty array[/-] [+]memoryview & ctypes: incorrect itemsize for empty array[/+] on Apr 14, 2019added a commit that references this issue on Dec 23, 2022gh-76963: PEP3118 itemsize of an empty ctypes array should not be 0 (G… mattip commented on Dec 23, 2022 Thanks for the fix that closed this. @eric-wieser do you remember where this impacted NumPy? I don't see the relevant issue or PR. eric-wieser commented on Dec 23, 2022 ContributorAuthor More actions I think the workaround in numpy needs to stay till after #5561 eric-wieser commented on Dec 23, 2022 ContributorAuthor More actions
MetadataMetadata3.10 (EOL)end of lifeend of life3.8 (EOL)end of lifeend of life3.9 (EOL)end of lifeend of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)topic-ctypestype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error StatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|