GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-161145: Test more bytes simultaneously in find_first_nonascii by cmaloney · Pull Request #161147 · python/cpython

Conversation

@cmaloney

@cmaloney cmaloney commented Nov 1, 2026 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Long ASCII runs now OR eight aligned words and test the high bits once, one branch per 64 bytes (32 on 32-bit) instead of one per word. When a block contains a non-ASCII byte, the existing one-word loop finds it. It is plain C: GCC and Clang compile it to SSE2/NEON loads, other compilers to ORed word loads with the same single branch.

Multiple compiler and architecture codegen sample: https://godbolt.org/z/8bM83vx3a

pyperf, PGO+LTO, 2 interleaved rounds:

Benchmark main this
(b'a' * 4096).decode() 165 ns 108 ns: 1.53x faster
(b'a' * 200).decode() 28.9 ns 26.5 ns: 1.09x faster
b'a' * 200 + 'é' + ..., .decode() 60.9 ns 56.0 ns: 1.09x faster
b'a' * 136 + 'é' + ..., .decode() 53.8 ns 51.3 ns: 1.05x faster
8-100 byte decodes, 'é' within 100 within 1.2 ns
import pyperf

runner = pyperf.Runner()
tail = 'é'.encode() + b'a' * 50
for n in (8, 100, 200, 4096):
    runner.timeit(f"(b'a' * {n}).decode()", "s.decode()",
                  globals={"s": b'a' * n})
for n in (16, 100, 136, 200):
    runner.timeit(f"(b'a' * {n} + 'é'.encode() + b'a' * 50).decode()",
                  "s.decode()", globals={"s": b'a' * n + tail})
  • Issue: Speed up UTF-8 validation in tokenizer and ASCII scanning in UTF-8 decoding #161145

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@cmaloney @claude
pythongh-161145: Test more bytes simultaneously find_first_nonascii
54f4722
Long ASCII runs now OR eight aligned words and test the high bits once,
one branch per 64 bytes (32 on 32-bit) instead of one per word.  When
a block contains a non-ASCII byte, the existing one-word loop finds it.
It is plain C: GCC and Clang compile it to SSE2/NEON loads, other
compilers to ORed word loads with the same single branch.

Multiple compiler and architecture codegen sample: https://godbolt.org/z/8bM83vx3a

pyperf, PGO+LTO, 2 interleaved rounds:

| Benchmark | main | this |
|---|---:|---:|
| (b'a' * 4096).decode() | 165 ns | 108 ns: 1.53x faster |
| (b'a' * 200).decode() | 28.9 ns | 26.5 ns: 1.09x faster |
| b'a' * 200 + 'é' + ..., .decode() | 60.9 ns | 56.0 ns: 1.09x faster |
| b'a' * 136 + 'é' + ..., .decode() | 53.8 ns | 51.3 ns: 1.05x faster |
| 8-100 byte decodes, 'é' within 100 | | within 1.2 ns |

```python
import pyperf

runner = pyperf.Runner()
tail = 'é'.encode() + b'a' * 50
for n in (8, 100, 200, 4096):
    runner.timeit(f"(b'a' * {n}).decode()", "s.decode()",
                  globals={"s": b'a' * n})
for n in (16, 100, 136, 200):
    runner.timeit(f"(b'a' * {n} + 'é'.encode() + b'a' * 50).decode()",
                  "s.decode()", globals={"s": b'a' * n + tail})
```

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@bedevere-app bedevere-app Bot added awaiting review type-feature A feature request or enhancement labels Nov 1, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Nov 1, 2026
Speed up UTF-8 validation in tokenizer and ASCII scanning in UTF-8 decoding #161145
Open
@cmaloney cmaloney added the skip news label Nov 1, 2026
@cmaloney @claude
pythongh-161145: Add NEWS entry
6636b05
Co-Authored-By: Claude Opus 5.5 <[email protected]>
@cmaloney cmaloney removed the skip news label Nov 2, 2026
@cmaloney cmaloney changed the title gh-161145: Test more bytes simultaneously find_first_nonascii gh-161145: Test more bytes simultaneously in find_first_nonascii Nov 2, 2026
@cmaloney cmaloney mentioned this pull request Nov 7, 2026
gh-161242: Cache strerror() messages per errno #161455
Draft
@vstinner

vstinner commented Nov 7, 2026

Copy link
Copy Markdown
Member

See #122902 which is similar.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@cmaloney
cmaloney marked this pull request as draft November 8, 2026 19:55
@bedevere-app bedevere-app Bot removed the awaiting review label Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

type-feature A feature request or enhancement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@cmaloney @vstinner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.