Description

ptweezy

opened on Nov 2, 2026

Issue body actions

Feature or enhancement

str.lower() and str.upper() on ASCII strings, and lower() and upper() on bytes and bytearray, all go through _Py_bytes_lower() and _Py_bytes_upper() in Objects/bytes_methods.c. These convert one byte at a time through the _Py_ctype_tolower table, which compilers can't vectorize.

Converting a machine word at a time (8 bytes on 64-bit platforms, 4 on 32-bit) with plain integer arithmetic, and keeping the table loop for the last few bytes, gives these pyperf results with gcc 13 -O3 on an x86-64 Linux VM:

	before	after
s.lower(), 1,024 ASCII chars	430 ns	131 ns (3.3x)
s.upper(), 1,024 ASCII chars	420 ns	135 ns (3.1x)
b.lower(), 1,024 bytes	402 ns	134 ns (3.0x)
s.lower(), 64K ASCII chars	22.0 µs	4.7 µs (4.7x)
s.lower(), 32 ASCII chars	40.3 ns	35.7 ns (1.13x)
s.lower(), 8 ASCII chars	33.4 ns	no measurable change

clang 18 gives 2.8 to 3.2x at 1,024 characters. MSVC on Windows x64 doesn't vectorize the new loop and gets about 2.2x, with or without PGO, and a gcc PGO+LTO build gets about 1.9x. 32-bit builds gain 1.2 to 1.3x. Short strings gain little because the method call and the allocation dominate: at 8 characters the results range from about 9% faster to 7% slower, depending on the build. The PR will have the full tables.

The change is about 35 lines in that one file: a small helper with a comment explaining the arithmetic, and two loops, with no intrinsics or #ifdefs. Output is identical for every byte value. I checked it exhaustively at word level on x86-64, 32-bit x86, arm64 and big-endian s390x, with gcc, clang and MSVC.

These loops were last optimized in gh-57947 (2012). I have a PR ready if this sounds acceptable.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

• gh-161155: Speed up ASCII lower() and upper() for str, bytes and bytearray #161201

Reacted by Guido van Rossum

Activity

1. NekoAsakura

   added

   type-featureA feature request or enhancementA feature request or enhancement

   performancePerformance or resource usagePerformance or resource usage

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   on Nov 2, 2026
2. sergey-miryanov commented on Nov 2, 2026

   

   sergey-miryanov

   on Nov 2, 2026

   Contributor

   More actions

   cc @eendebakpt
3. eendebakpt commented on Nov 2, 2026

   

   eendebakpt

   on Nov 2, 2026

   Contributor

   More actions

   A PR seems reasonable (but I am not convinced yet). As always we have to balance performance gain to added complexity and possible regressions for important cases (like small strings here). 35 lines extra/diff might be ok. If possible aim for a small diff and easy to review patch.
4. StanFromIreland commented on Nov 2, 2026

   

   StanFromIreland

   on Nov 2, 2026

   Member

   More actions

   Please also benchmark without vectorisation.
5. ptweezy commented on Nov 2, 2026

   

   ptweezy

   on Nov 2, 2026

   Author

   More actions

   Updated for the PR's current revision (9433a66).

   Release builds with auto-vectorization off in both trees (gcc -fno-tree-vectorize, clang -fno-vectorize -fno-slp-vectorize), pyperf on GitHub-hosted runners, time on main / time with the PR:

   Runner	1-7 chars	8-16	32	1,024	64K
   Linux x86-64 (AMD EPYC), gcc 13	1.03x	1.09x	1.19x	2.17x	2.39x
   Linux arm64 (Neoverse-N2), gcc 13	0.99x	1.07x	1.20x	2.54x	3.41x
   macOS arm64 (M1), Apple clang 17	1.03x	1.17x	1.46x	3.28x	3.79x
   macOS x86-64 (i7-8700B), Apple clang 17	0.98x	0.97x	1.11x	1.98x	3.03x

   Of the default builds tested, gcc -O3 and MSVC x86-64 release vectorize the loop; Apple clang 17, gcc PGO+LTO and MSVC PGO do not. Windows release at 1,024 characters: 2.85x (x86-64), 2.20x (arm64).

   Changes from the original proposal: strings shorter than a word skip the word loop after one length check; the last word overlaps the previous one instead of falling back to the byte loop; under MSVC the word loop counts words instead of bytes, because MSVC vectorizes only loops that count by one (the patch's only #if). The patch adds 59 lines to Objects/bytes_methods.c, 45 excluding comments and blank lines.

   All results: https://github.com/ptweezy/cpython-ascii-case-bench/blob/main/RESULTS.md
6. ptweezy

   added 2 commits that reference this issue on Nov 3, 2026

   pythongh-161155: Let MSVC vectorize the word loop

   91c788f

   pythongh-161155: Use the count-by-one loop only under MSVC

   9433a66

Issue actions