GeezerVison

Thanks for reviews!

This came up in NumPy: numpy/numpy#31204 and I am wondering whether you discussed if the buffer protocol should use D and F vs. Zd and Zf here?
On the one hand, sure, this follows the struct protocol, which is documented. But on the other hand the struct module didn't include it at the time, and the PEP says it should be Zd. And because of that this will be what NumPy (and everyone else) uses.

Now, understanding D as well is unproblematic, but I dunno if it is great for us to transition to D.

EDIT: sorry forgot the buffer protocol link: https://peps.python.org/pep-3118/#additions-to-the-struct-string-syntax (see the Z entry in the table)

EDIT2: And now I find #146241 which is the more interesting PR... but since everyone is the same person involved...

This came up in NumPy: numpy/numpy#31204 and I am wondering whether you discussed if the buffer protocol should use D and F vs. Zd and Zf here?

Yes, I think so. That was briefly discussed while adding support for complex types in the ctypes module, e.g.: #120894 (comment). Note that later type codes for ctypes/struct were changed to NumPy-style: #121249 (comment)

Currently, struct/array/ctypes modules use single-letter codes for fundamental types. So, adding support for complex types in PEP 3118-style requires rather complex changes in internal structures, interpreting of the format string and so on. Not sure if this feasible just for complex types alone.

BTW, note we are trying to be consistent across the stdlib modules (struct/array/ctypes), while NumPy uses different "type codes", e.g. 'D' with ndarray, instead of 'Zd'.

But on the other hand the struct module didn't include it at the time, and the PEP says it should be Zd.

I'm not sure that PEP actually says about this. It's not a part of the Specification. Note that nothing from this section was actually implemented (except '?' format type) during 20+ years.

I quickly read linked discussion thread and then did some search on PEP 3118 in mail lists. Sorry, I wasn't able to find where that part of the proposal was discussed, where a draft implementation was proposed. And it looks that some parts of that proposal are too vague (see this and this), and that includes support for complex types! IMO, it's not a part of the actual specification and the warning at top of the PEP clearly says that:

This PEP is a historical document. The up-to-date, canonical documentation can now be found at Buffer Protocol, PyBufferProcs, PyMemoryView_FromObject.

Not all features proposed here were implemented. Specifically:

PyObject_CopyToObject was not added.
Additions to the struct string-syntax were not added, except for ? (_Bool).
PyObject_GetMemoryView is named PyMemoryView_FromObject.

This PEP targets Python 3.0, which was released more than a decade ago. Any proposals to add missing functionality should be discussed as new features, not treated as finishing the implementation of this PEP.

See PEP 1 for how to propose changes.

This looks as a good advice.

Maybe you can open a discussion thread on https://discuss.python.org/c/core-dev/23 ? PR thread is not a best place for this.

functionality should be discussed as new features, not treated as finishing the implementation of this PEP.

Sure, that sounds like sound advice, with the caveat that on this one it isn't true: It was implemented in a vast ecosystem, just not Python itself.

It was implemented in a vast ecosystem, just not Python itself.

The NumPy uses also 'D' type code. Which convention we should adopt for the array module?

The struct module already uses D rather than Zd now. And sure, NumPy uses mainly D as well from a user perspective. So yeah, D is a clear reasonable choice.

But, just because I agree that D seems like the better choice…. And I would agree that the buffer protocol might have been better off just using D as well…. (Also because for future dtypes, I don't believe in single character type codes anyway.)

None of that changes the situation that NumPy cannot just change (export, import can support both easily).
And as with NumPy the whole ecosystem won't be able to export D for years.

Now, Python can be the odd one out and one could document that one should/must implement both D and Zd for import as most users use Zd for export for the time being, but Python uses D and hopes that in a some years we can slowly transition to D (i.e. in a few years, Python and NumPy could chose to give a deprecation warning when importing Zd and then a few years later we could only support D -- but even then it might not be a great experience).

But as much as I wouldn't mind that outcome... The inconvenience of just using Zd in Python honestly seems smaller to me right now?

Now d.p.o thread opened, lets continue here: https://discuss.python.org/t/106949

The inconvenience of just using Zd in Python honestly seems smaller to me right now?

We can't just switch to two-letters code for compatibility reasons, as 'D' type was introduced in v3.14. Though, we could add an alias. What next? Deprecate former type codes?

I prepared a reversion in #148674.