GeezerVison

The PR is big because it also replaces usage of the old private API with new public API. If the API is approved, I will split the PR into smaller pieces and measure the performance impact of these changes.

Some functions should be optimized after the removal of the private min_size member:

• unicode_encode_ucs1()
• utf8_encoder()
• PyBytes_FromFormatV()

These functions allocate a little bit too much memory, extend argument of PyBytesWriter_Extend() should be adjusted.

I just tried to make the code work, not to optimize it.

This change has no impact on performance, even if the new public API allocates memory on the heap, instead of allocating on the stack. It uses a freelist to optimize PyBytesWriter_Create().

Example of microbenchmark on 3 functions:

import pyperf
import binascii

runner = pyperf.Runner()
runner.bench_func('from list 100', bytes, list(b'x' * 100))
runner.bench_func('from list 1,000', bytes, list(b'x' * 1_000))

runner.bench_func('from hex 100', bytes.fromhex, bytes(range(100)).hex())
runner.bench_func('from hex 1,000', bytes.fromhex, (b'x' * 1_000).hex())

runner.bench_func('b2a_uu', binascii.b2a_uu, b'x' * 45)

Result:

Benchmark hidden because not significant (1): b2a_uu

pseudo-tangential idea: Could this instead just be a C wrapper for io.BytesIO? Working to try to get to one fast implementation to read/write a bytes object.

pseudo-tangential idea: Could this instead just be a C wrapper for io.BytesIO?

io.BytesIO API is basically the write(bytes) method, whereas proposed PyBytesWriter API gives directly a pointer into a buffer. It's a different API. I don't think that io.BytesIO API can or should be modified to give directly a pointer. io.BytesIO is more complex since it allows changing the position, the seek() method, and also reading, the read() method.

I created a discussion: https://discuss.python.org/t/add-pybyteswriter-public-c-api/81182

It seems like most developers are confused by the API which requires to pass writer and buf to most functions. I abandon this API.