GeezerVison is showing github.com — Reorganize the re module sources · Issue #91308 · python/cpython
Reorganize the re module sources #91308Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2022-03-29.15:54:32.145>
labels = ['expert-regex', 'library', '3.11']
title = 'Reorganize the re module sources'
updated_at = <Date 2022-04-05.08:10:39.512>
user = 'https://github.com/serhiy-storchaka' bugs.python.org fields:
activity = <Date 2022-04-05.08:10:39.512>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'Regular Expressions']
creation = <Date 2022-03-29.15:54:32.145>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 47152
keywords = ['patch']
message_count = 26.0
messages = ['416268', '416294', '416320', '416328', '416471', '416497', '416502', '416523', '416543', '416545', '416547', '416548', '416551', '416557', '416563', '416591', '416593', '416595', '416615', '416657', '416659', '416667', '416676', '416693', '416747', '416761']
nosy_count = 8.0
nosy_names = ['gvanrossum', 'vstinner', 'ezio.melotti', 'mrabarnett', 'serhiy.storchaka', 'malin', 'Anthony Sottile', 'dom1310df']
pr_nums = ['32177', '32188', '32290', '32298']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue47152'
versions = ['Python 3.11'] Activityserhiy-storchaka commented on Mar 29, 2022 I proposed it several years ago on the Python-Dev mailing list and that change was approved in general. The reorganization was deferred because there were several known bugs in the RE engine (fixes for which could potentially be backported) and there were not merged patches waiting for review. Now the patch for atomic groups was merged and bugs was fixed (thanks to Ma Lin).
Both the C code and the Python code for the re module are distributed on few files, which lie down in directories Modules and Lib. It makes difficult to work with all related files because they are intermixed with source files of different modules.
The following changes are planned:
- Convert the re module into a package. Make sre_* modules its submodules.
- Move C sources for the _sre module into a separate directory.
- Extract the code for generating definitions of C constants from definitions of Python constants into a separate script and add it in the Tools/scripts directory (there are precedences: generate_token.py, etc).
dom1310df commented on Mar 29, 2022 animalize commented on Mar 30, 2022 Please don't merge too close to the 3.11 beta1 release date, I'll submit PRs after this merged. serhiy-storchaka commented on Mar 30, 2022 It turns out that pip uses sre_constants in its copy of pyparsing. The problem is already fixed in the upstream of pyparsing and soon should be fixed in pip. We still need to keep sre_constants and maybe other sre_* modules, but deprecate them.
Could the sre_parse and sre_constants modules be kept with public names (i.e. without the leading underscore) but within the re namespace?
It is a good idea which will allow to minimize breakage in short term. You can write "from re import sre_parse", and it would work in old and new versions because sre_parse and sre_compile were imported in the re module. This trick does not work with sre_constants, you still need try/except.
But the code that depends on these modules is fragile and can be broken by other ways.
Please don't merge too close to the 3.11 beta1 release date, I'll submit PRs after this merged.
I am going to implement step 2 only after merging your changes for bpo-23689. vstinner commented on Apr 1, 2022 sre_constants, sre_compile and sre_parse are not tested and are not documented. I don't consider them as public API currently.
If someone has good reason to use them, IMO we must clearly define which exact API is needed, properly document and test it.
If we expose something, I don't think that the API would be exposed as re.sre_xxx.xxx, but as re.xxx.
I suggest to hide sre_xxx submodules by adding an underscore to their name. Moreover, the "sre_" prefix is now redundant. I suggest renaming:
- sre_constants => re._constants
- sre_compile => re._compile
- sre_parse => re._parse
gvanrossum commented on Apr 1, 2022 I don't mind reorganizing this, but I would insist that we keep code using old undocumented things (like the sre_* modules) working for several releases, using the standard deprecation approach. serhiy-storchaka commented on Apr 1, 2022 Modules with old names are kept (deprecated). The questions are:
- Should we keep the sre_ prefix in new submodules? Should we prefix them with underscores?
- Should we keep only non-underscored names in the sre_* modules or undescored names too?
gvanrossum commented on Apr 1, 2022
- If we're reorganizing anyway, I see no reason to keep the old names.
- For maximum backwards compatibility, I'd say keep as much as you can, as long as keeping it won't interfere with the reorganization.
serhiy-storchaka commented on Apr 2, 2022
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|