GeezerVison is showing github.com — gh-95555: Add the CATEGORY_UCD opcode and the simple enumerated properties by serhiy-storchaka · Pull Request #153023 · python/cpython
Conversation
serhiy-storchaka
commented
Jul 4, 2026
•
edited by bedevere-app
Bot
Loading
Uh oh!
There was an error while loading. Please reload this page.
First of four stacked PRs that complete \p{...} support with the properties backed by the Unicode Character Database, matched in C through a unicodedata capsule. This PR adds the machinery and the simplest tier: the enumerated properties stored as a single byte in the per-character record — Bidi_Class (bc), East_Asian_Width (ea), Grapheme_Cluster_Break (gcb) and Indic_Conjunct_Break (incb).
unicodedata exports the _ucd_re_CAPI capsule (the \N{...} precedent) and _ucd_re_info(), which lists the property selectors and value names. The parser resolves a property name and value to a selector and value index; the new CATEGORY_UCD opcode packs (negate, property, value) into one operand and is matched in C by sre_category_ucd(), which compares the index returned by the capsule. A negated single value is one charset item, so \P{bc=AL} composes inside a set.
The follow-up PRs add, in order: the numeric and binary properties (ccc, Bidi_Mirrored, Extended_Pictographic), the computed properties (Block, Decomposition_Type, Numeric_Type) and the remaining General_Category values and groups (Ll, Lo, M/P/S) with POSIX punct.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
… properties
Introduce the unicodedata capsule used to match \p{...} properties that need
the Unicode Character Database, starting with the simplest tier: the
enumerated properties stored as a single byte in the per-character record --
Bidi_Class (bc), East_Asian_Width (ea), Grapheme_Cluster_Break (gcb) and
Indic_Conjunct_Break (incb).
unicodedata exports the _ucd_re_CAPI capsule (the \N{...} precedent) and
_ucd_re_info(), which lists the property selectors and value names. The new
CATEGORY_UCD opcode packs (negate, property, value) and is matched in C by
sre_category_ucd(). A negated single value is one charset item, so \P{bc=AL}
composes inside a set.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
The CATEGORY_UCD support adds three process-global tables that are set
once and effectively constant: the cached capsule pointer in
sre_category_ucd(), the capsule struct built by unicodedata_create_re_capi(),
and the enumerated-property table in unicodedata_ucd_re_info(). List them
in the c-analyzer ignore file, as done for the name-lookup capsule.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
|
This PR is stale because it has been open for 90 days with no activity.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
This PR is superseded by #156490, a newer open pull request for the same issue (gh-95555), and it no longer merges cleanly. Closing to cull the number of open pull requests. Do not reopen this pull request without getting a go-ahead from a triager or core developer.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Reopening, incorrectly closed as a duplicate.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Reviewers
ericsnowcurrently
Awaiting requested review from ericsnowcurrently
ericsnowcurrently is a code owner
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|