GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — Implement unicodedata XID and block properties by youknowdot · Pull Request #9458 · RustPython/RustPython

Conversation

youknowdot commented Oct 29, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor

Summary

Implement Python 3.15's module-level unicodedata.isxidstart(), isxidcontinue(), and block().

Reuse the existing XID property predicates (not Python's underscore-inclusive identifier-start policy).
Generate a compact block table using the existing Unicode build machinery and the unmodified official Unicode 17.0.0 Blocks.txt. No new dependency. All 346 ranges/names match CPython's pinned generated table.
Follow CPython's single-codepoint conversion, string-subclass acceptance, type/length diagnostics, surrogate behavior, and No_Block fallback. Keep these APIs absent on ucd_3_2_0.
Run scripts/update_lib migrate against the complete pinned test_unicodedata.py; retire exactly four proven expectedFailure markers, retaining every unrelated local adaptation. No hand-edited tests or duplicate snippets.

CPython basis

Reference: d220409a13b77de7a6c30abe16ea90db01528edc.

Validation

On native-source base 698c196b2e188c46772f8a3cf76164227282a645 with Rust 1.99:

Locked/offline scoped cargo check and cargo clippy -- -D warnings for rustpython-stdlib, features compiler,threading,host_env,doc,ucd_3_2_0: passed.
CLI build with threading,stdlib,stdio,importlib,host_env,doc,ucd_3_2_0: passed. Initial bounded build timed out; one fresh cached resume completed successfully.
The four unmodified pinned CPython methods (test_isxidstart, test_isxidcontinue, test_block, test_block_invalid_input) pass on both the pinned CPython build and this RustPython build, with no skip/expected failure.
A bounded 63-case argument/value oracle per interpreter produces identical JSON, including underscores, unassigned code points, surrogates, block boundaries, No_Block, bad lengths/types, string subclasses, and legacy API absence.
All applicable normal pre-commit hooks passed, including cspell (two canonical API names added to its existing dictionary).
Final migrated methods are AST-identical to the tested canonical methods; native source is unchanged after the build.

Limits: this does not claim whole-module or whole-workspace test completion. Existing shared argument-binding arity/keyword error wording is not changed: rejection and TypeError are checked there, not exact METH_O wording. Unrelated legacy UCD/normalization/decomposition issues remain. Latest-main integration is left to PR merge CI; the exact local validation base is stated above.

AI assistance: implementation and validation prepared with OpenAI Codex.

Summary by CodeRabbit

New Features
- Added unicodedata.isxidstart() and unicodedata.isxidcontinue() to check whether a character can start or continue a valid Unicode identifier.
- Added unicodedata.block() to return the Unicode block name for a character, or No_Block when it belongs to no named block.
- These functions require a string containing exactly one Unicode code point and report errors for invalid input.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions


          Implement unicodedata XID and block properties

3d40a27

Follow CPython 3.15 character conversion and Unicode 17 block data; retire four verified test markers via update_lib.

Assisted-by: OpenAI Codex:GPT-6.1

coderabbitai Bot commented Oct 29, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor

No actionable comments were generated in the recent review.

Recent review info

Run configuration

Source mode: Auto
Repository: RustPython/RustPython
Branch: main
File: .coderabbit.yml
Commit: bcaa07a
Layers: Repository YAML
Review profile: CHILL
Plan: Advanced
Run ID: 4ebbc06b-4cab-484b-bb64-c99d26c4adcf

Commits

Reviewing files that changed from the base of the PR and between bcaa07a and 3d40a27.

Files ignored due to path filters: 1

Lib/test/test_unicodedata.py is excluded by !Lib/**

Files selected for processing: 6

.cspell.json
crates/stdlib/src/unicodedata.rs
crates/unicode/build.rs
crates/unicode/src/data.rs
crates/unicode/unicode/README.md
crates/unicode/unicode/latest/Blocks.txt

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

Walkthrough

The change adds Unicode block data generation and lookup, and exposes unicodedata.isxidstart, unicodedata.isxidcontinue, and unicodedata.block. The new functions validate that their argument is a string containing exactly one code point.

Changes: Unicode property and block APIs

Layer / File(s)	Summary
Unicode block data and lookup `crates/unicode/unicode/latest/Blocks.txt`, `crates/unicode/build.rs`, `crates/unicode/src/data.rs`, `crates/unicode/unicode/README.md`	Adds Unicode 17.0.0 block ranges, generates a block table from `Blocks.txt`, and adds a lookup that returns the matching block name or `No_Block`. The README identifies the data source and version.
unicodedata property functions `crates/stdlib/src/unicodedata.rs`, `.cspell.json`	Adds argument validation and the `isxidstart`, `isxidcontinue`, and `block` functions. Adds the two XID function names to the spelling word list.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Feature

Possibly related issues

RustPython/RustPython#6839 — The new unicodedata APIs support the issue’s objective of syncing Python 3.15 library behavior; the provided summary does not show the related bundled-library or test updates.

Suggested reviewers: youknowone

Merge Risk: Minimal · up to 3d40a

The new Unicode APIs reject keyword arguments as required. No actionable merge-blocking issue remains; merge after normal checks.

Pre-merge checks: 5

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main changes: adding Unicode XID and block properties to unicodedata.
Docstring Coverage		Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentati…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 5 functions across 3 files. (3 skipped: 3 unsupported.)

Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

Biome (2.5.15)

.cspell.json

File contains syntax errors that prevent linting: Line 1: Expected an array, an object, or a literal but instead found '// See: https://github.com/streetsidesoftware/cspell/tree/; Line 6: Expected an array, an object, or a literal but instead found '// "@cspell/dict-cpp/cspell-ext.json",'.; Line 2: End of file expected; Line 6: End of file expected; Line 7: End of file expected; Line 7: End of file expected; Line 8: End of file expected; Line 8: End of file expected; Line 9: End of file expected; Line 9: End of file expected; Line 10: End of file expected; Line 10: End of file expected; Line 12: End of file expected; Line 12: End of file expected; Line 12: End of file expected; Line 12: End of file expected; Line 14: End of file expected; Line 14: End of file expected; Line 14: End of file expected; Line 14: End of file expected; Line 16: End of file expected; Line 16: End of file expected; Line 17: Expected an array, an object, or a literal but instead found '// Sometimes keeping same

... [truncated 1397 characters] ...

; Line 130: End of file expected; Line 131: End of file expected; Line 131: End of file expected; Line 132: End of file expected; Line 132: End of file expected; Line 133: End of file expected; Line 133: End of file expected; Line 134: End of file expected; Line 134: End of file expected; Line 135: End of file expected; Line 135: End of file expected; Line 137: End of file expected; Line 138: End of file expected; Line 140: End of file expected; Line 140: End of file expected; Line 140: End of file expected; Line 141: End of file expected; Line 142: End of file expected; Line 142: End of file expected; Line 142: End of file expected; Line 143: End of file expected; Line 145: End of file expected; Line 145: End of file expected; Line 145: End of file expected; Line 151: End of file expected

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

github-actions Bot commented Oct 29, 2026

Copy link

Copy Markdown

Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[ ] test: cpython/Lib/test/test_free_threading

dependencies:

dependent tests: (no tests depend on free_threading)

[ ] test: cpython/Lib/test/test_unicodedata.py (TODO: 13)
[x] test: cpython/Lib/test/test_unicode_file.py
[x] test: cpython/Lib/test/test_unicode_file_functions.py
[x] test: cpython/Lib/test/test_unicode_identifiers.py
[x] test: cpython/Lib/test/test_ucn.py (TODO: 1)

dependencies:

dependent tests: (no tests depend on unicode)

Legend:

[+] path exists in CPython
[x] up-to-date, [ ] outdated

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

youknowone approved these changes

Oct 29, 2026

View reviewed changes

youknowone marked this pull request as ready for review

October 29, 2026 12:10

chatgpt-codex-connector Bot commented Oct 29, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	⚠️ Failed 2026-10-29T12:11:37.273223Z We couldn’t complete this review. Comment `@codex review` to try again.	`3d40a27`	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

youknowone merged commit f98543d into RustPython:main

Oct 29, 2026

21 checks passed

Uh oh!

There was an error while loading. Please reload this page.

coderabbitai Bot mentioned this pull request

Nov 12, 2026

No alloc Unicode version #9820

Open

1 of 3 tasks

Reviewers

youknowone

youknowone approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

youknowdot commented Oct 29, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

CPython basis

Validation

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 29, 2026

📦 Library Dependencies

Uh oh!

chatgpt-codex-connector Bot commented Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

youknowdot commented Oct 29, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

There was an error while loading. Please reload this page.

coderabbitai Bot commented Oct 29, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

chatgpt-codex-connector Bot commented Oct 29, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.