GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — Align Unicode lookup and decomposition with CPython 3.15 by youknowdot · Pull Request #9476 · RustPython/RustPython

Conversation

youknowdot commented Oct 30, 2026

Copy link

Copy Markdown

Contributor

Summary

Validate names returned by unicode_names2 against canonical names, while preserving exact case-insensitive alias and named-sequence lookup. Reject loose whitespace/hyphen matching and noncanonical CJK hexadecimal names.
Match CPython's distinction for the old UCD view: lookup() rejects aliases/sequences but accepts ordinary names from the shared database; name() and decomposition() retain historical membership checks.
Return full Hangul L/V/T decomposition and use current decomposition mappings in both views, removing the incorrect historical normalization-correction override. Normalization code is unchanged.
Use scripts/update_lib migrate to retire four related test_unicodedata markers while preserving the other nine adaptations.

Sources: pinned CPython lookup, decomposition, and canonical tests.

Validation and limits

Scoped locked/offline CLI check, strict Clippy (-D warnings) and build succeeded.
120 bounded scratch observations matched pinned CPython exactly, including exception text, aliases/sequences, significant hyphens, both UCD views, Hangul and corrected decomposition mappings.
Four restored test_unicodedata methods passed in both interpreters. Of twelve selected canonical tests, CPython passed twelve; RustPython passed eleven and retained one pre-existing expected failure in test_ucn.test_tangut_ideographs. The strict wrapper therefore exited 1, not a full-suite pass.
A separately bounded single-test diagnostic confirmed that expected failure is the existing ast.literal_eval/compile Unicode-name literal SyntaxError, not unicodedata.lookup. Its decorator and test body remain unchanged; direct Tangut lookup is covered by the passing differential observations.

No full Unicode sweep, full workspace/CAPI suite, normalization corpus or cross-platform runtime validation was performed. This does not claim full Unicode or parser compatibility.

AI assistance: OpenAI Codex (model version unavailable), disclosed in the commit trailer. Automated validation is not human review.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions


          Align Unicode lookup and decomposition with CPython 3.15

cccd90d

Validate Unicode names instead of accepting loose dependency matching, preserve
old-UCD alias and sequence restrictions without filtering ordinary names, and
match CPython's Hangul and historical-membership decomposition behavior.
Remove the incorrect historical correction override and refresh four related
unicodedata test markers using update_lib. Leave normalization unchanged.

Assisted-by: OpenAI Codex: model version unavailable

coderabbitai Bot commented Oct 30, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor

Caution

Review failed

The pull request is closed.

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

github-actions Bot commented Oct 30, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[x] lib: cpython/Lib/weakref.py
[x] lib: cpython/Lib/_weakrefset.py
[x] test: cpython/Lib/test/test_weakref.py (TODO: 7)
[x] test: cpython/Lib/test/test_weakset.py

dependencies:

weakref

dependent tests: (219 tests)

weakref: test_array test_ast test_asyncio test_builtin test_capi test_code test_concurrent_futures test_context test_contextlib test_copy test_ctypes test_deque test_descr test_dict test_enum test_exceptions test_finalization test_frame test_free_threading test_functools test_gc test_generators test_genericalias test_importlib test_inspect test_io test_ipaddress test_itertools test_logging test_memoryview test_mmap test_operator test_ordered_dict test_pickle test_picklebuffer test_queue test_re test_scope test_set test_slice test_socket test_sqlite3 test_ssl test_struct test_sys test_tempfile test_thread test_threading test_threading_local test_tkinter test_type_params test_types test_typing test_unicodedata test_unittest test_uuid test_weakref test_weakset test_xml_etree
- asyncio: test_asyncio test_concurrent_futures test_external_inspection test_os test_pdb test_unittest
  - pdb: test_remote_pdb
- bdb: test_bdb
- concurrent: test_compileall test_concurrent_futures test_free_threading test_interpreters test_wmi
- copy: test_argparse test_bytes test_codecs test_collections test_copyreg test_coroutines test_csv test_decimal test_defaultdict test_dictviews test_email test_fractions test_free_threading test_genericpath test_http_cookies test_minidom test_opcache test_optparse test_os test_platform test_plistlib test_site test_statistics test_structseq test_super test_sysconfig test_tomllib test_urllib2 test_urlparse test_xml_dom_minicompat test_zlib
  - argparse: test_profiling
  - collections: test_annotationlib test_bisect test_c_locale_coercion test_call test_capi test_configparser test_contains test_ctypes test_embed test_exception_group test_fileinput test_free_threading test_funcattrs test_hash test_iter test_iterlen test_json test_math test_monitoring test_pathlib test_patma test_pprint test_profiling test_pydoc test_random test_reprlib test_richcmp test_shelve test_sqlite3 test_string test_tkinter test_traceback test_tuple test_type_cache test_urllib test_userdict test_userlist test_userstring test_with
  - dataclasses: test__colorize test_ctypes test_regrtest test_zoneinfo
  - email.generator: test_email
  - gettext: test_gettext test_tools
  - http.cookiejar: test_http_cookiejar
  - logging.handlers: test_pkgutil
  - mailbox: test_mailbox
  - smtplib: test_smtplib test_smtpnet
  - tarfile: test_shutil test_tarfile
  - webbrowser: test_webbrowser
- gzip: test_xmlrpc
- logging: test_hashlib test_support test_urllib2net
  - venv: test_venv
- multiprocessing: test_fcntl test_importlib test_multiprocessing_forkserver test_multiprocessing_main_handling test_profiling
- symtable: test_symtable
- tempfile: test_bz2 test_cmd_line test_compile test_ctypes test_curses test_dis test_doctest test_dtrace test_ensurepip test_faulthandler test_filecmp test_generated_cases test_httpservers test_importlib test_interpreters test_launcher test_linecache test_modulefinder test_ntpath test_peg_generator test_pickletools test_pkg test_profiling test_pstats test_py_compile test_pyrepl test_readline test_runpy test_selectors test_shlex test_source_encoding test_string_literals test_subprocess test_sys_settrace test_tabnanny test_termios test_threadedtempfile test_tokenize test_turtle test_urllib_response test_wave test_winconsoleio test_zipapp test_zipfile test_zipfile64 test_zstd
  - ctypes.util: test_ctypes
  - urllib.request: test_sax test_urllib2_localnet test_urllibnet

[x] test: cpython/Lib/test/test_dict.py (TODO: 1)
[x] test: cpython/Lib/test/test_dictcomps.py
[x] test: cpython/Lib/test/test_dictviews.py (TODO: 1)
[x] test: cpython/Lib/test/test_userdict.py
[x] test: cpython/Lib/test/mapping_tests.py

dependencies:

dependent tests: (no tests depend on dict)

[ ] test: cpython/Lib/test/test_free_threading (TODO: 1)

dependencies:

dependent tests: (no tests depend on free_threading)

[x] test: cpython/Lib/test/test_grp.py (TODO: 1)

dependencies:

dependent tests: (195 tests)

grp: test_pathlib test_shutil test_subprocess test_tarfile
- pathlib: test_ast test_dbm_sqlite3 test_ensurepip test_httpservers test_importlib test_json test_launcher test_logging test_netrc test_os test_pathlib test_peg_generator test_pkgutil test_profiling test_pyrepl test_runpy test_tempfile test_tomllib test_tools test_traceback test_unparse test_venv test_winapi test_zipapp test_zipfile test_zoneinfo test_zstd
  - compileall: test_compileall
  - importlib: test_asdl_parser test_bdb test_capi test_cmd_line_script test_codecs test_ctypes test_doctest test_external_inspection test_frozen test_hashlib test_importlib test_inspect test_interpreters test_linecache test_modulefinder test_multiprocessing_main_handling test_py_compile test_pyclbr test_pydoc test_pyrepl test_reprlib test_source_encoding test_sundry test_support test_unittest test_zipfile test_zipimport test_zoneinfo
  - profiling.sampling.heatmap_collector: test_profiling
  - traceback: test_asyncio test_builtin test_code_module test_concurrent_futures test_contextlib test_contextlib_async test_coroutines test_crossinterp test_dictcomps test_exceptions test_http_cookiejar test_iter test_listcomps test_pyexpat test_setcomps test_socket test_ssl test_sys test_threadedtempfile test_threading test_unittest test_with
  - zipapp: test_pdb
- shutil: test_argparse test_bz2 test_ctypes test_embed test_filecmp test_glob test_io test_multiprocessing_forkserver test_os test_peepholer test_profiling test_sax test_site test_string_literals test_sysconfig test_unicode_file
  - argparse: test_profiling
  - ctypes.util: test_ctypes
  - http.server: test_robotparser test_urllib2_localnet test_xmlrpc
  - multiprocessing.util: test_asyncio test_concurrent_futures
  - tempfile: test_asyncio test_bytes test_cmd_line test_compile test_concurrent_futures test_csv test_curses test_dis test_dtrace test_faulthandler test_fileinput test_free_threading test_generated_cases test_genericalias test_importlib test_interpreters test_mailbox test_ntpath test_os test_pickle test_pickletools test_pkg test_profiling test_pstats test_pyrepl test_readline test_regrtest test_selectors test_shlex test_sys_settrace test_tabnanny test_termios test_tokenize test_turtle test_urllib test_urllib2 test_urllib_response test_wave test_winconsoleio test_zipfile64
  - uuid: test_free_threading
  - webbrowser: test_webbrowser
- subprocess: test_android test_asyncio test_atexit test_audit test_c_locale_coercion test_concurrent_futures test_file_eintr test_gc test_gzip test_msvcrt test_osx_env test_platform test_plistlib test_poll test_profiling test_pyrepl test_quopri test_remote_pdb test_repl test_script_helper test_select test_signal test_sqlite3 test_tcl test_tkinter test_tstring test_unittest test_utf8_mode test_wait3 test_xpickle
  - asyncio: test_asyncio
  - imaplib: test_imaplib
  - platform: test__locale test__osx_support test_baseexception test_calendar test_cmath test_fcntl test_math test_mimetypes test_startfile test_strptime test_time test_tkinter test_winreg test_wsgiref

[x] lib: cpython/Lib/imaplib.py
[x] test: cpython/Lib/test/test_imaplib.py (TODO: 1)

dependencies:

imaplib

dependent tests: (1 tests)

imaplib: test_imaplib

[ ] test: cpython/Lib/test/test_capi

dependencies:

dependent tests: (no tests depend on capi)

[x] lib: cpython/Lib/functools.py
[ ] test: cpython/Lib/test/test_functools.py (TODO: 1)

dependencies:

functools

dependent tests: (53 tests)

functools: test_annotationlib test_argparse test_asyncio test_builtin test_bytes test_call test_clinic test_codeop test_context test_contextlib_async test_ctypes test_curses test_datetime test_dbm_dumb test_dbm_sqlite3 test_dis test_doctest test_enum test_fractions test_free_threading test_functools test_genericalias test_gettext test_gzip test_hmac test_inspect test_io test_iter test_itertools test_math_property test_monitoring test_posixpath test_profiling test_pyexpat test_pyrepl test_random test_repl test_reprlib test_shutil test_signal test_sort test_sqlite3 test_ssl test_sys_settrace test_timeout test_tkinter test_traceback test_typing test_unicodedata test_unittest test_winsound test_xml_etree test_zoneinfo

[x] lib: cpython/Lib/enum.py
[x] test: cpython/Lib/test/test_enum.py (TODO: 1)

dependencies:

enum

dependent tests: (18 tests)

enum: test_argparse test_ast test_enum test_httplib test_json test_opcache test_patma test_pstats test_pydoc test_signal test_socket test_ssl test_str test_time test_tkinter test_types test_typing test_uuid

[x] lib: cpython/Lib/sqlite3
[x] test: cpython/Lib/test/test_sqlite3 (TODO: 19)

dependencies:

sqlite3

dependent tests: (2 tests)

sqlite3: test_dbm_sqlite3 test_sqlite3

[ ] lib: cpython/Lib/io.py
[x] lib: cpython/Lib/_pyio.py
[ ] test: cpython/Lib/test/test_io (TODO: 10)

dependencies:

io (native: _io, _thread, errno, msvcrt, sys)
- _pyio
- _collections_abc, abc, codecs, locale, os, stat, warnings

dependent tests: (105 tests)

io: test__colorize test_android test_argparse test_ast test_asyncio test_base64 test_buffer test_builtin test_bz2 test_calendar test_cmd test_cmd_line_script test_codecs test_compile test_compileall test_compiler_assemble test_concurrent_futures test_configparser test_contextlib test_csv test_dbm_dumb test_descr test_dis test_email test_enum test_fileinput test_free_threading test_ftplib test_generated_cases test_getpass test_gzip test_hashlib test_http_cookiejar test_httplib test_httpservers test_importlib test_inspect test_io test_json test_logging test_lzma test_mailbox test_marshal test_memoryview test_mimetypes test_minidom test_multibytecodec test_optparse test_pathlib test_pdb test_peg_generator test_pickle test_pickletools test_platform test_plistlib test_pprint test_print test_profile test_profiling test_pstats test_pty test_pulldom test_pydoc test_pyexpat test_pyrepl test_quopri test_regrtest test_remote_pdb test_robotparser test_sax test_shlex test_shutil test_site test_smtplib test_socket test_socketserver test_subprocess test_support test_sys test_tarfile test_tempfile test_threadedtempfile test_timeit test_tokenize test_traceback test_types test_typing test_unittest test_urllib test_urllib2 test_uuid test_wave test_webbrowser test_winconsoleio test_wsgiref test_xml_dom_xmlbuilder test_xml_etree test_xml_etree_c test_xmlrpc test_xpickle test_zipapp test_zipfile test_zipimport test_zoneinfo test_zstd

[ ] test: cpython/Lib/test/test_pyrepl (TODO: 5)
[x] test: cpython/Lib/test/test_repl.py (TODO: 3)

dependencies:

dependent tests: (no tests depend on pyrepl)

[x] lib: cpython/Lib/xml
[x] test: cpython/Lib/test/test_xml_etree.py (TODO: 18)
[ ] test: cpython/Lib/test/test_xml_etree_c.py
[x] test: cpython/Lib/test/test_minidom.py (TODO: 2)
[x] test: cpython/Lib/test/test_pulldom.py
[ ] test: cpython/Lib/test/test_pyexpat.py (TODO: 41)
[x] test: cpython/Lib/test/test_sax.py (TODO: 25)
[x] test: cpython/Lib/test/test_xml_dom_minicompat.py
[x] test: cpython/Lib/test/test_xml_dom_xmlbuilder.py

dependencies:

dependent tests: (39 tests)

xml: test_doctest test_minidom test_pulldom test_pydoc test_pyexpat test_regrtest test_sax test_typing test_xml test_xml_dom_minicompat test_xml_dom_xmlbuilder test_xml_etree
- plistlib: test_plistlib
  - platform: test__locale test__osx_support test_android test_asyncio test_baseexception test_builtin test_calendar test_cmath test_ctypes test_fcntl test_math test_mimetypes test_os test_platform test_shutil test_socket test_ssl test_startfile test_strptime test_sysconfig test_time test_tkinter test_winreg test_wsgiref
  - webbrowser: test_webbrowser
- xmlrpc.client: test_xmlrpc

[x] lib: cpython/Lib/test/libregrtest

dependencies:

regrtest

dependent tests: (no tests depend on regrtest)

[x] lib: cpython/Lib/re
[ ] test: cpython/Lib/test/test_re.py (TODO: 3)
[x] test: cpython/Lib/test/re_tests.py

dependencies:

dependent tests: (86 tests)

re: test_android test_ast test_asyncio test_binascii test_builtin test_bytes test_clinic test_cmd_line test_ctypes test_dict test_dis test_docxmlrpc test_dtrace test_email test_embed test_faulthandler test_filecmp test_fileinput test_fnmatch test_format test_frame test_free_threading test_fstring test_functools test_future_stmt test_genericalias test_glob test_hashlib test_http_cookiejar test_httplib test_httpservers test_imaplib test_importlib test_inspect test_ipaddress test_launcher test_logging test_mailbox test_mmap test_optparse test_ordered_dict test_pdb test_pprint test_profiling test_pydoc test_pyexpat test_pyrepl test_re test_regrtest test_remote_pdb test_runpy test_set test_site test_smtplib test_socket test_ssl test_strftime test_string test_strptime test_strtod test_structseq test_symtable test_syntax test_sys test_sysconfig test_tarfile test_tempfile test_tkinter test_tokenize test_tools test_traceback test_turtle test_types test_typing test_unittest test_unparse test_venv test_webbrowser test_winapi test_with test_wsgiref test_xmlrpc test_zipfile test_zipimport test_zoneinfo test_zstd

[x] lib: cpython/Lib/profiling
[x] test: cpython/Lib/test/test_profiling

dependencies:

profiling

dependent tests: (3 tests)

profiling: test_profiling
- cProfile: test_free_threading test_pstats

[x] lib: cpython/Lib/codecs.py
[x] test: cpython/Lib/test/test_charmapcodec.py
[x] test: cpython/Lib/test/test_codeccallbacks.py (TODO: 3)
[x] test: cpython/Lib/test/test_codecencodings_cn.py
[x] test: cpython/Lib/test/test_codecencodings_hk.py
[x] test: cpython/Lib/test/test_codecencodings_iso2022.py
[x] test: cpython/Lib/test/test_codecencodings_jp.py
[x] test: cpython/Lib/test/test_codecencodings_kr.py
[x] test: cpython/Lib/test/test_codecencodings_tw.py
[x] test: cpython/Lib/test/test_codecmaps_cn.py
[x] test: cpython/Lib/test/test_codecmaps_hk.py
[x] test: cpython/Lib/test/test_codecmaps_jp.py
[x] test: cpython/Lib/test/test_codecmaps_kr.py
[x] test: cpython/Lib/test/test_codecmaps_tw.py
[x] test: cpython/Lib/test/test_codecs.py (TODO: 2)
[x] test: cpython/Lib/test/test_multibytecodec.py
[x] test: cpython/Lib/test/testcodec.py

dependencies:

codecs

dependent tests: (200 tests)

codecs: test_charmapcodec test_codeccallbacks test_codecs test_curses test_eof test_exceptions test_free_threading test_importlib test_inspect test_io test_json test_locale test_logging test_multibytecodec test_os test_pdb test_plistlib test_readline test_sax test_str test_sys
- email.charset: test_email
- email.utils: test_httpservers test_smtplib test_urllib2
  - http.server: test_robotparser test_urllib2_localnet test_xmlrpc
  - logging.handlers: test_concurrent_futures test_pkgutil
  - smtplib: test_smtpnet
  - urllib.request: test_http_cookiejar test_pathlib test_pydoc test_site test_ssl test_urllib test_urllib2net test_urllibnet
- encodings: test_email
  - locale: test__locale test_builtin test_c_locale_coercion test_calendar test_cmd_line test_decimal test_float test_format test_re test_regrtest test_strftime test_strptime test_tkinter test_types test_utf8_mode
- json: test_embed test_interpreters test_json test_profiling test_pyrepl test_remote_pdb test_subprocess test_sysconfig test_tomllib test_tools test_traceback test_zoneinfo
  - importlib.metadata: test_importlib
  - profiling: test_profiling
- pickle: test_annotationlib test_argparse test_array test_ast test_asyncio test_bool test_bytes test_bz2 test_collections test_concurrent_futures test_configparser test_coroutines test_csv test_ctypes test_defaultdict test_deque test_descr test_dict test_dictviews test_email test_enum test_enumerate test_external_inspection test_fractions test_free_threading test_functools test_generators test_genericalias test_genericpath test_http_cookies test_interpreters test_io test_ipaddress test_iter test_itertools test_list test_lzma test_memoryview test_minidom test_opcache test_operator test_ordered_dict test_os test_pickle test_picklebuffer test_pickletools test_platform test_positional_only_arg test_random test_range test_set test_shelve test_slice test_socket test_statistics test_string test_structseq test_super test_time test_trace test_tuple test_turtle test_type_aliases test_type_params test_typing test_unittest test_urlparse test_uuid test_xml_dom_minicompat test_xml_etree test_xpickle test_zipfile test_zlib test_zoneinfo
  - tracemalloc: test_tracemalloc
- plistlib:
  - platform: test__osx_support test_android test_asyncio test_baseexception test_cmath test_ctypes test_fcntl test_math test_mimetypes test_shutil test_startfile test_tkinter test_winreg test_wsgiref
  - webbrowser: test_webbrowser
- tokenize: test_free_threading test_linecache test_peg_generator test_tabnanny test_tokenize test_unparse
  - inspect: test_abc test_asyncgen test_buffer test_clinic test_code test_free_threading test_grammar test_monitoring test_ntpath test_patma test_posixpath test_pyrepl test_signal test_sqlite3 test_support test_type_annotations test_yield_from test_zipimport test_zipimport_support
  - linecache: test_bdb
  - traceback: test_asyncio test_code_module test_concurrent_futures test_contextlib test_contextlib_async test_crossinterp test_dictcomps test_importlib test_listcomps test_pyexpat test_setcomps test_threadedtempfile test_threading test_unittest test_with

[x] lib: cpython/Lib/typing.py
[ ] test: cpython/Lib/test/test_typing.py (TODO: 1)
[x] test: cpython/Lib/test/test_type_aliases.py
[x] test: cpython/Lib/test/test_type_annotations.py
[x] test: cpython/Lib/test/test_type_params.py
[x] test: cpython/Lib/test/test_genericalias.py

dependencies:

typing

dependent tests: (22 tests)

typing: test_annotationlib test_builtin test_copy test_descr test_enum test_fractions test_free_threading test_funcattrs test_functools test_genericalias test_grammar test_inspect test_isinstance test_patma test_peg_generator test_pydoc test_pyrepl test_remote_pdb test_type_aliases test_type_params test_types test_typing

[ ] test: cpython/Lib/test/test_metaclass.py (TODO: 4)

dependencies:

dependent tests: (no tests depend on metaclass)

[ ] test: cpython/Lib/test/test_with.py

dependencies:

dependent tests: (no tests depend on with)

[ ] test: cpython/Lib/test/test_cmd_line.py (TODO: 18)

dependencies:

dependent tests: (no tests depend on cmd_line)

[ ] test: cpython/Lib/test/test_faulthandler.py (TODO: 4)

dependencies:

dependent tests: (5 tests)

faulthandler: test_cmd_line test_concurrent_futures test_faulthandler test_inspect test_xxtestfuzz

[ ] test: cpython/Lib/test/test_unicodedata.py (TODO: 7)
[x] test: cpython/Lib/test/test_unicode_file.py
[x] test: cpython/Lib/test/test_unicode_file_functions.py
[x] test: cpython/Lib/test/test_unicode_identifiers.py
[x] test: cpython/Lib/test/test_ucn.py (TODO: 1)

dependencies:

dependent tests: (no tests depend on unicode)

[x] lib: cpython/Lib/_pylong.py
[ ] test: cpython/Lib/test/test_int.py (TODO: 2)
[x] test: cpython/Lib/test/test_long.py (TODO: 4)
[x] test: cpython/Lib/test/test_int_literal.py

dependencies:

dependent tests: (no tests depend on int)

[ ] test: cpython/Lib/test/test_set.py (TODO: 1)

dependencies:

dependent tests: (no tests depend on set)

[ ] test: cpython/Lib/test/test_bytes.py (TODO: 11)

dependencies:

dependent tests: (no tests depend on bytes)

[x] lib: cpython/Lib/compileall.py
[x] test: cpython/Lib/test/test_compileall.py (TODO: 2)

dependencies:

compileall

dependent tests: (1 tests)

compileall: test_compileall

[x] lib: cpython/Lib/collections
[x] lib: cpython/Lib/_collections_abc.py
[x] test: cpython/Lib/test/test_collections.py
[x] test: cpython/Lib/test/test_deque.py (TODO: 1)
[x] test: cpython/Lib/test/test_defaultdict.py
[x] test: cpython/Lib/test/test_ordered_dict.py

dependencies:

collections

dependent tests: (378 tests)

collections: test_annotationlib test_array test_asyncio test_bisect test_builtin test_c_locale_coercion test_call test_capi test_collections test_configparser test_contains test_context test_copy test_csv test_ctypes test_defaultdict test_deque test_descr test_dict test_dictviews test_embed test_enum test_exception_group test_fileinput test_frame test_free_threading test_funcattrs test_functools test_genericalias test_hash test_inspect test_io test_ipaddress test_iter test_iterlen test_json test_logging test_math test_monitoring test_opcache test_ordered_dict test_pathlib test_patma test_pickle test_plistlib test_pprint test_profiling test_pydoc test_random test_reprlib test_richcmp test_set test_shelve test_sqlite3 test_statistics test_string test_struct test_sys test_tkinter test_traceback test_tuple test_type_cache test_types test_typing test_unittest test_urllib test_userdict test_userlist test_userstring test_weakref test_weakset test_with
- ast: test_ast test_codeop test_compile test_compiler_codegen test_coroutines test_dis test_free_threading test_fstring test_future_stmt test_peepholer test_peg_generator test_site test_ssl test_type_comments test_ucn test_unparse
  - annotationlib: test_grammar test_type_annotations test_type_params
  - dbm.dumb: test_dbm_dumb
  - inspect: test_abc test_argparse test_asyncgen test_buffer test_clinic test_code test_curses test_decimal test_free_threading test_generators test_ntpath test_operator test_posixpath test_profiling test_pyrepl test_signal test_support test_turtle test_yield_from test_zipimport test_zipimport_support test_zoneinfo
  - pyclbr: test_pyclbr
  - traceback: test_asyncio test_code_module test_concurrent_futures test_contextlib test_contextlib_async test_crossinterp test_dictcomps test_exceptions test_http_cookiejar test_importlib test_listcomps test_pyexpat test_setcomps test_socket test_subprocess test_threadedtempfile test_threading test_unittest
- asyncio: test_asyncio test_concurrent_futures test_external_inspection test_os test_pdb
  - pdb: test_remote_pdb
- concurrent.futures._base: test_concurrent_futures
- csv: test_free_threading
  - importlib.metadata: test_importlib
- dbm.sqlite3: test_dbm_sqlite3
- difflib: test_difflib test_profile test_sys_settrace
  - argparse: test_profiling
- dis: test__opcode test_compiler_assemble test_dtrace test_positional_only_arg
  - bdb: test_bdb
  - modulefinder: test_importlib test_modulefinder
  - trace: test_trace
- email.feedparser: test_email
- http.client: test_docxmlrpc test_hashlib test_httpservers test_unicodedata test_urllib2 test_wsgiref test_xmlrpc
  - urllib.request: test_sax test_urllib2_localnet test_urllib2net test_urllibnet
- idlelib: test_idle
- importlib: test_asdl_parser test_capi test_cmd_line_script test_codecs test_compileall test_ctypes test_doctest test_frozen test_importlib test_interpreters test_linecache test_multiprocessing_main_handling test_pkgutil test_py_compile test_pyrepl test_runpy test_source_encoding test_sundry test_tomllib test_tools test_unittest test_zipfile test_zoneinfo
  - ctypes.util: test_ctypes
  - ensurepip: test_ensurepip test_venv
  - profiling: test_profiling
  - py_compile: test_importlib
  - sysconfig: test_cmd_line test_gc test_importlib test_launcher test_os test_osx_env test_peg_generator test_perfmaps test_regrtest test_sysconfig test_tempfile test_time test_tools
  - zipfile: test_shutil test_zipapp test_zipfile test_zipfile64
  - zipimport: test_importlib
- inspect:
  - cmd: test_cmd
  - dataclasses: test__colorize test_ctypes
  - rlcompleter: test_pyrepl test_rlcompleter
- logging:
  - multiprocessing.util: test_concurrent_futures
- multiprocessing: test_fcntl test_memoryview test_multiprocessing_forkserver test_re
- platform: test__locale test__osx_support test_android test_baseexception test_calendar test_cmath test_mimetypes test_platform test_startfile test_strptime test_winreg
  - uuid: test_free_threading test_os
- pprint: test_htmlparser test_sys_setprofile
  - pickle: test_bool test_bytes test_bz2 test_concurrent_futures test_ctypes test_email test_enumerate test_fractions test_free_threading test_genericpath test_http_cookies test_interpreters test_io test_itertools test_json test_list test_lzma test_minidom test_picklebuffer test_pickletools test_range test_slice test_str test_structseq test_super test_type_aliases test_unittest test_urlparse test_uuid test_xml_dom_minicompat test_xml_etree test_xpickle test_zlib
- profiling:
  - cProfile: test_free_threading test_pstats
- queue: test_sched
- selectors: test_selectors
  - socket: test_epoll test_exception_hierarchy test_ftplib test_httplib test_imaplib test_io test_kqueue test_mailbox test_mmap test_poplib test_pty test_smtplib test_smtpnet test_socketserver test_stat test_timeout test_urllib_response
  - subprocess: test_atexit test_audit test_faulthandler test_file_eintr test_gzip test_interpreters test_json test_msvcrt test_poll test_pyrepl test_quopri test_repl test_script_helper test_select test_tcl test_tkinter test_tstring test_unittest test_utf8_mode test_wait3 test_webbrowser
- shlex: test_shlex
- shutil: test_filecmp test_glob test_string_literals test_tarfile test_unicode_file
  - pathlib: test_importlib test_netrc test_pathlib test_tomllib test_tools test_winapi test_zstd
  - tempfile: test_generated_cases test_pkg test_pyrepl test_readline test_tabnanny test_termios test_tokenize test_wave test_winconsoleio
- statistics:
  - random: test_capi test_complex test_devpoll test_email test_float test_free_threading test_grp test_heapq test_hmac test_int test_long test_numeric_tower test_pow test_pwd test_queue test_sort test_strtod test_thread
- string: test_base64 test_email test_fnmatch test_free_threading test_pyrepl test_secrets test_string
- threading: test_concurrent_futures test_ctypes test_fork1 test_free_threading test_importlib test_interpreters test_io test_ioctl test_pyrepl test_robotparser test_syslog test_threading_local
- tkinter: test_tkinter test_ttk test_ttk_textonly
- traceback:
  - timeit: test_timeit
- tracemalloc: test_tracemalloc

[x] test: cpython/Lib/test/test_sys.py (TODO: 4)
[x] test: cpython/Lib/test/test_syslog.py
[x] test: cpython/Lib/test/test_sys_setprofile.py (TODO: 1)
[x] test: cpython/Lib/test/test_sys_settrace.py
[x] test: cpython/Lib/test/test_audit.py (TODO: 16)
[x] test: cpython/Lib/test/audit-tests.py

dependencies:

dependent tests: (250 tests)

sys: regrtestdata test___all__ test__colorize test__locale test__osx_support test_android test_annotationlib test_argparse test_array test_asdl_parser test_ast test_asyncio test_audit test_base64 test_bdb test_bigaddrspace test_bigmem test_binascii test_bisect test_buffer test_builtin test_bytes test_bz2 test_c_locale_coercion test_calendar test_capi test_class test_clinic test_cmath test_cmd test_cmd_line test_cmd_line_script test_code test_code_module test_codeccallbacks test_codecs test_collections test_compile test_compileall test_complex test_concurrent_futures test_context test_contextlib test_coroutines test_crossinterp test_csv test_ctypes test_curses test_datetime test_dbm test_dbm_sqlite3 test_decimal test_descr test_dict test_difflib test_dis test_doctest test_docxmlrpc test_dtrace test_dynamic test_dynamicclassattribute test_email test_embed test_ensurepip test_enum test_enumerate test_eof test_epoll test_except_star test_exceptions test_external_inspection test_faulthandler test_fcntl test_file_eintr test_fileinput test_float test_fork1 test_format test_fractions test_frame test_free_threading test_frozen test_functools test_future_stmt test_gc test_generated_cases test_generators test_genericpath test_genexps test_getopt test_glob test_grammar test_gzip test_hash test_hashlib test_http_cookiejar test_httpservers test_importlib test_inspect test_int test_interpreters test_io test_ioctl test_iter test_itertools test_json test_launcher test_list test_listcomps test_locale test_logging test_long test_lzma test_mailbox test_marshal test_math test_memoryview test_metaclass test_mimetypes test_mmap test_monitoring test_msvcrt test_multibytecodec test_multiprocessing_forkserver test_multiprocessing_main_handling test_netrc test_ntpath test_numeric_tower test_opcache test_operator test_optparse test_ordered_dict test_os test_osx_env test_pathlib test_patma test_pdb test_peepholer test_perfmaps test_pickle test_pkg test_pkgutil test_platform test_plistlib test_popen test_posixpath test_print test_profile test_profiling test_property test_pty test_pwd test_py_compile test_pyclbr test_pydoc test_pyexpat test_pyrepl test_quopri test_raise test_range test_re test_readline test_regrtest test_remote_pdb test_repl test_reprlib test_resource test_runpy test_sax test_scope test_script_helper test_select test_selectors test_shutil test_signal test_site test_slice test_smtplib test_socket test_source_encoding test_sqlite3 test_ssl test_stable_abi_ctypes test_startfile test_stat test_statistics test_str test_strftime test_string_literals test_strptime test_strtod test_struct test_subprocess test_support test_sys test_sys_setprofile test_sys_settrace test_sysconfig test_syslog test_tabnanny test_tarfile test_tcl test_tempfile test_termios test_threading test_threading_local test_threadsignals test_time test_timeit test_tkinter test_tomllib test_tools test_trace test_traceback test_tracemalloc test_ttk test_tuple test_type_cache test_type_comments test_types test_typing test_unicode_file test_unicode_file_functions test_unicodedata test_unittest test_urllib test_urllib2 test_urllib2net test_urlparse test_utf8_mode test_uuid test_venv test_wait3 test_wait4 test_wave test_weakref test_webbrowser test_winconsoleio test_winreg test_with test_wsgiref test_xml_etree test_xmlrpc test_xpickle test_zipapp test_zipfile test_zipfile64 test_zipimport test_zipimport_support test_zlib

Legend:

[+] path exists in CPython
[x] up-to-date, [ ] outdated

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

codspeed Bot commented Oct 30, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor

Merging this PR will not alter performance

✅ 62 untouched benchmarks
⏩ 4 skipped benchmarks¹

_{Comparing youknowdot:python315-unicode-lookup-decomposition-20261030 (d194a68) with main (cc18267)}

Footnotes

4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.


          Preserve Unicode 3.2 widths for still-unassigned characters

d194a68

Match CPython's old-UCD width policy: retain modern width defaults for
characters still unassigned, and use the historical table for assigned
characters. Restore the bounded east Asian width tests and retire the
obsolete 9.0 override through update_lib.

Permit explicit exact-source retirement of unconditional skip markers in
the existing updater path, preserving its exact-match and consumed-once
guards. Add focused tool tests for successful and rejected retirements.

Assisted-by: OpenAI Codex: model version unavailable

youknowone approved these changes

Oct 30, 2026

View reviewed changes

youknowone requested changes

Oct 30, 2026

View reviewed changes

youknowone left a comment

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review this again if it is aligned to 3.2.0 feature well

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

youknowdot commented Oct 30, 2026

Copy link

Copy Markdown

Contributor Author

In response to the 3.2 feature review:

I rechecked the 3.2 feature against pinned CPython source and our actual old data. #9476 targets CPython's ucd_3_2_0 API, not a literal historical database for every operation:

Old lookup accepts current ordinary names but rejects aliases/sequences; unlike name, it does not check old membership. The lookup correction preserves this asymmetry.
Old decomposition rejects newly assigned characters, then uses current mappings/full Hangul decomposition. Thus U+2F868 gives 36FC; historical 2136A belongs to the separate old normalization corrections, not this API.
EAW distinguishes newly assigned from still-unassigned characters: old-view U+9FA6 is N, but still-unassigned U+3FFFD remains W. The generator leaves current width defaults for the latter. Our historical derived-width file explicitly omits unassigned points.

The earlier scoped comparisons covered 120 lookup/decomposition rows and 100 EAW/property rows across both views, on their respective implementation heads. This review additionally checked 27 finite source-table boundaries, not an exhaustive sweep or new runtime run.

Importantly, RustPython's pre-existing normalize/is_normalized still ignore the old view. Those functions and feature wiring are unchanged; this PR does not complete all Unicode 3.2 semantics.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

youknowdot force-pushed the python315-unicode-lookup-decomposition-20261030 branch from 8cb31d7 to d194a68 Compare

November 3, 2026 07:13

youknowdot mentioned this pull request

Nov 3, 2026

Accept bytes in Unicode name lookup #9585

Merged


          Keep Unicode changes independent of updater tooling

f6b8464

Consolidate the generic marker-retirement helper and regression coverage in RustPython#9578. Preserve the Unicode implementation and whole-file test changes unchanged.

Assisted-by: OpenAI Codex:model version not exposed

youknowdot mentioned this pull request

Nov 5, 2026

Consolidate whole-file update_lib migration guards #9578

Merged

youknowone approved these changes

Nov 6, 2026

View reviewed changes

youknowone marked this pull request as ready for review

November 6, 2026 12:08

chatgpt-codex-connector Bot commented Nov 6, 2026

Copy link

Copy Markdown

Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	❌ Failed 2026-11-06T12:08:35.811858Z We couldn’t complete this review. Comment `@codex review` to try again.	`f6b8464`	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

chatgpt-codex-connector Bot commented Nov 6, 2026

Copy link

Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

youknowone merged commit 83ba227 into RustPython:main

Nov 6, 2026

20 of 21 checks passed

Uh oh!

There was an error while loading. Please reload this page.

Reviewers

youknowone

youknowone approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

youknowdot commented Oct 30, 2026

Summary

Validation and limits

Uh oh!

coderabbitai Bot commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review failed

Uh oh!

github-actions Bot commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📦 Library Dependencies

Uh oh!

codspeed Bot commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

youknowone left a comment

Choose a reason for hiding this comment

Uh oh!

youknowdot commented Oct 30, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented Nov 6, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

coderabbitai Bot commented Oct 30, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

github-actions Bot commented Oct 30, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

codspeed Bot commented Oct 30, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.