GeezerVison

cc. @eendebakpt: you might be interested in this PR.

@rhpvorderman Thanks for making the PR. I will look at it the coming days. Which python operations do you expect (or hope) that will benefit from the PR? A (micro) benchmark for this would be nice

I did some benchmarking. As expected short decodings do not have much benefit. I tested on reading individual lines from a fairly large ASCII file. (FASTQ format, 1.5 GB)
The result before

Benchmark 1: ./python decode_test_lines_ascii.py ~/test/5millionreads_R1.fastq
  Time (mean ± σ):      1.567 s ±  0.019 s    [User: 1.339 s, System: 0.228 s]
  Range (min … max):    1.543 s …  1.597 s    10 runs
Benchmark 1: ./python decode_test_blocks_ascii.py ~/test/5millionreads_R1.fastq
  Time (mean ± σ):     789.3 ms ±   7.4 ms    [User: 535.3 ms, System: 253.9 ms]
  Range (min … max):   776.0 ms … 803.8 ms    10 runs

after

Benchmark 1: ./python decode_test_lines_ascii.py ~/test/5millionreads_R1.fastq
  Time (mean ± σ):      1.519 s ±  0.014 s    [User: 1.271 s, System: 0.247 s]
  Range (min … max):    1.504 s …  1.550 s    10 runs
Benchmark 1: ./python decode_test_blocks_ascii.py ~/test/5millionreads_R1.fastq
  Time (mean ± σ):     730.3 ms ±   5.8 ms    [User: 490.0 ms, System: 240.1 ms]
  Range (min … max):   723.4 ms … 739.9 ms    10 runs

The file had a pattern of line lengths of 30, 152, 2 152. As is visible, the PR barely has an impact on run time decoding individual lines.

When decoding large blocks of text (4KB per iteration) the performance benefit is clearly visible.

This would mostly benefit users therefore that

• Know how to make python perform well and operate on blocks of text
• Have no clue or don't care for their use case and read the entire contents of the file in one go.

I have only benchmarked ascii_decode here, but the benefits for find_max_char should be similar. Negligible for small strings, more substantial for bigger data.

I am not sure the performance gain is enough to make this worthwhile,

Multiplied by the times that python decodes utf-8 text I think the total power saving worldwide would amount to something significant.

When decoding large blocks of text (4KB per iteration) the performance benefit is clearly visible.

But it's still quite minor. In one instance (git main), you read your ASCII file at 1.9 GB/s. In the other instance (this PR), you read it at 2 GB/s. It's already unlikely that your processing pipeline is bottlenecked by the file read phase, so this difference will not really be noticeable in an actual use case.

Thoughts? @Fidget-Spinner @corona10 @markshannon

Sorry I'm not an expert on low-level OS or architecture internals, so I can't help here.

But it's still quite minor. In one instance (git main), you read your ASCII file at 1.9 GB/s. In the other instance (this PR), you read it at 2 GB/s.

I should note that performance should be even better at GCC 14, due to better autovectorization. That compiler is not widely shipped yet, and benchmarks are at GCC12.

For another perspective, on a pure micro-benchmark, Python 3.12 can already decode ASCII bytes at 17 GB/s.

$ python3.12 -m timeit -s "s = b'x' * (1000**2)" "s.decode('ascii')"
5000 loops, best of 5: 56.7 usec per loop

I find it unlikely that going faster than 17 GB/s will make a meaningful difference on real-world workloads.

On my PC before:

$ ./python -m timeit -s "s = b'x' * (1000**2)" "s.decode('ascii')"
5000 loops, best of 5: 67.8 usec per loop

after:

./python -m timeit -s "s = b'x' * (1000**2)" "s.decode('ascii')"
5000 loops, best of 5: 49.3 usec per loop

On your not real-world microbenchmark it is 28% less runtime.
I work with 300GB+ ASCII files regularly, in bioinformatics. That is quite niche, sure, but it is a real-world work load.

EDIT: I managed to compile with GCC14:

./python -m timeit -s "s = b'x' * (1000**2)" "s.decode('ascii')"
5000 loops, best of 5: 45.2 usec per loop

@pitrou, thanks to your comment I simplified the code by only using unaligned loads with memcpy. In general on x86-64 and ARM64 unaligned loads only occur a penalty when it is across a cacheline boundary, but that is 1 cycle of latency. On other platforms I do not worry so much. If unaligned loads are illegal, memcpy won't translate into an unaligned load.

I found that bytes.isascii was not using find_max_char, which is a pity, so I remedied that too.

ascii_decode becomes much simpler when simply using memcpy and not worrying about unaligned loads and stores.

After these changes data volume is not an issue any more, also line by line reading and decoding small amounts of data is faster.

before:

./python -m timeit -s 'data=b"1234567890123456789012345678901"' 'data.decode("latin-1")'
5000000 loops, best of 5: 70.5 nsec per loop

after:

./python -m timeit -s 'data=b"1234567890123456789012345678901"' 'data.decode("latin-1")'
5000000 loops, best of 5: 62.4 nsec per loop

That really is the effect of the unaligned load at the end. Unlike my initial attempt, this also makes line by line reading about 3% faster.

I also notice that for larger volumes latin-1 now tends to outstrip ascii decoding on my PC (very much n=1, I know).

Since latin-1 decoding and ascii decoding have about the same speed, and latin-1 decoding actually being faster for larger volumes , I do wonder if the default latin-1 strategy, which is copying+finding max char, is better than the ascii decode strategy , copying while looking for max char.
In theory there could be a single function: find_non_ascii_char which returns the exact offset of a non_ascii_char. This can then be used by both ascii_decode and find_max_char. That would further reduce the amount of code in the codebase.

I have been reflecting a bit on the code and I understand this code change is controversial, it is adding a lot of lines for a limited speed gain. Especially the find_max_char code, while faster, doesn't look much better looks worse.
However I feel that the ascii_decode function that uses unaligned loads is an improvement over the current version. It does not require alignment of both src and dest pointers to use the optimal code path, it is less nested and shorter. I could break that out in a separate PR.

@pitrou:

I find it unlikely that going faster than 17 GB/s will make a meaningful difference on real-world workloads.

I concur with Antoine: I'm not convinced that this PR is needed. It makes the code more complex, so harder to maintain, and I'm not impressed by the benchmark results. It's up to 1.37x faster on a micro-benchmark, ok, but when the workload is only "decode very long ASCII strings".

I concur with Antoine: I'm not convinced that this PR is needed. It makes the code more complex, so harder to maintain, and I'm not impressed by the benchmark results. It's up to 1.37x faster on #120212 (comment), ok, but when the workload is only "decode very long ASCII strings".

Thanks for elaborating on this. Indeed 3% faster in typical use cases (reading line by line) only becomes notable when processing files that span multiple gigabytes. (I regularly work with those, but that contorts my view on normal workloads).

I will close this then.

I will close this then.

I would suggest you to write an optimized ASCII decoder on PyPI which uses SIMD or something like that to control exactly the instructions and really get the best from the CPU. But using explicitly SIMD is too low-level for CPython :-(

Have already done that 😅 ... I am just backporting some ideas to cpython 😉 . But I will reserve that in the future for things that make a really noticable difference. Like #120398. Rather than using your precious review time for small incremental gains. Sorry for that.

Have already done that 😅

What is the project name?

The project dnaio can be used to parse a format called FASTQ, which is by definition all ASCII.

This project reads 128 KiB buffers and copies ASCII strings from then. (Typical length is 150bp for DNA sequences). Rather than invoking PyUnicode_DecodeASCII which does an individual check per string, the 128KiB buffer is checked in its entirety with this code: https://github.com/marcelm/dnaio/blob/main/src/dnaio/ascii_check.h.
As a result there is a guarantee that any string copied from the buffer is ascii and PyUnicode_New can be used with a memcpy.

The ASCII checking code used SIMD at first, but compilers are clever enough and it is much more portable to write code that can be optimized by the compiler. In this case GCC can optimize the 4 size_t OR calls into 2 __m128i OR calls. Since most CPUs can execute multiple OR operations in one clock tick, the unrolled code is also a lot faster on compilers that do not autovectorize as well.

EDIT: Because everything should be ASCII, there is also no need to provide an early exit within the for loop. When a bigger than ASCII character is found an error should be raised, but that error does not have to be raised within nanoseconds.