GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-153668: Replace Unicode numeric switch with binary-search tables by scorpi023 · Pull Request #156081 · python/cpython

Conversation

@scorpi023

@scorpi023 scorpi023 commented Aug 20, 2026 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Fixes #153668.

Summary

makeunicodedata.py currently emits _PyUnicode_ToNumeric() as a switch
with 2,023 case labels. On Windows ARM64, MSVC's /O2 optimizer consumes
excessive memory and the x86-hosted ARM64 compiler fails with C1002 while
compiling Objects/unicodectype.c.

Generate a sorted code-point table, a parallel value-index table, and a
table of the 151 UCD numeric-value entries instead. _PyUnicode_ToNumeric()
now uses binary search and still returns -1.0 for code points without a
numeric value. NUMERIC_MASK and _PyUnicode_IsNumeric() are unchanged.

This is a source-level alternative to the temporary build workaround in
#153669. That PR disables optimization for affected MSVC toolsets; this
change keeps optimization enabled by removing the compiler-stressing
generated switch.

Windows ARM64 result

VCTools 14.51.36231, Release /O2, on a Windows 11 ARM64 lab host:

Compiler host main This PR
HostX86 -> ARM64 C1002 after 11.487 s / 3.544 GB private success in 0.971 s / 0.037 GB
HostARM64 -> ARM64 success in 29.482 s / 25.196 GB private success in 0.598 s / 0.050 GB

Validation

  • Full native ARM64 Release build: success.
  • python -m test -j1 test_unicodedata test_str: 212 tests passed.
  • python -m test -j1 -u cpu test_unicodedata: 73 tests passed.
  • python -m test -j0 -x test_profiling: 49,207 tests passed across
    442 test files. test_profiling was excluded because its sampling
    coordinator hangs independently on this lab image.
  • Exhaustively compared all 1,114,112 Unicode code points with the previous
    generated switch; all 2,023 numeric mappings and non-numeric results match.
  • Tools\patchcheck\patchcheck.py: passed.

AI assistance: OpenAI Codex assisted with the investigation, implementation,
and validation. This PR is opened as a draft so the submitter can complete
the required human review before marking it ready.

  • Issue: Windows Arm64 release build fails #153668

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@scorpi023
pythongh-153668: Replace Unicode numeric switch with binary-search ta…
7eaf834
…bles
@python-cla-bot

python-cla-bot Bot commented Aug 20, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot mentioned this pull request Aug 20, 2026
Windows Arm64 release build fails #153668
Open
@scorpi023
scorpi023 marked this pull request as ready for review August 20, 2026 03:32
@bedevere-app bedevere-app Bot added the awaiting review label Aug 20, 2026
scorpi023 added 2 commits August 20, 2026 15:22
@scorpi023
Merge branch 'main' into pythongh-153668-msvc-arm64-numeric-lookup
aee3ab1
@scorpi023
Merge branch 'main' into pythongh-153668-msvc-arm64-numeric-lookup
cdc79d2
@chris-eibl

chris-eibl commented Aug 29, 2026

Copy link
Copy Markdown
Member

Although it is definitely a MSVC compiler bug running out of memory during compiling, this seems to be an interesting approach as well which might have pros/cons above my head.

cc @serhiy-storchaka who has commited in this domain.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@scorpi023
Merge branch 'main' into pythongh-153668-msvc-arm64-numeric-lookup
ee8001e
@hugovk

hugovk commented Sep 19, 2026

Copy link
Copy Markdown
Member

!buildbot ARM64 Windows Non-Debug

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Sep 19, 2026

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @hugovk for commit ee8001e 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F156081%2Fmerge

The command will test the builders whose names match following regular expression: ARM64 Windows Non-Debug

The builders matched are:

  • ARM64 Windows Non-Debug PR
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@StanFromIreland

StanFromIreland commented Oct 29, 2026

Copy link
Copy Markdown
Member

I don't think we should do this to hide an MSVC bug, unless there's additional performance benefits?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@StanFromIreland StanFromIreland added the pending The issue will be closed if no feedback is provided label Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review pending The issue will be closed if no feedback is provided

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Windows Arm64 release build fails

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@scorpi023 @chris-eibl @hugovk @bedevere-bot @StanFromIreland
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.