GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — gh-152100: Move re compiler optimizations to Lib/re/_optimizer.py by serhiy-storchaka · Pull Request #152154 · python/cpython

Conversation

serhiy-storchaka commented Jun 25, 2026

Copy link

Copy Markdown

Member

Move the compile-time optimizations out of _compiler.py into a new Lib/re/_optimizer.py: _optimize_charset, _compile_charset, _simple, _compile_info, the literal/charset prefix helpers, _combine_flags and the related constants. _compiler.py keeps only the bytecode emitter and imports them.

The dependency is now one-directional (_compiler → _optimizer → _constants/_sre/_parser). There is no behavior change: the compiled bytecode is identical and test_re passes unchanged (one test repointed _generate_overlap_table to its new home).

This is groundwork for follow-up compile-side optimizations under gh-152100 that would otherwise accumulate in _compiler.py.

🤖 Generated with Claude Code

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

bedevere-app Bot mentioned this pull request

Jun 25, 2026

Support set operations in regular expression character classes #152100

Open

bedevere-app Bot added the awaiting core review label

Jun 25, 2026


          pythongh-152100: Move re compiler optimizations to Lib/re/_optimizer.py

586b2b8

Move the compile-time optimizations (_optimize_charset, _compile_charset,
_simple, _compile_info and the literal/charset prefix helpers) out of
_compiler.py into a new Lib/re/_optimizer.py.  _compiler.py keeps only the
bytecode emitter and imports them.  This is groundwork for a follow-up
optimization; there is no behavior change.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

serhiy-storchaka force-pushed the re-optimizer branch from 81875f4 to 586b2b8 Compare

June 25, 2026 06:58

serhiy-storchaka added the skip news label

Jun 25, 2026

serhiy-storchaka enabled auto-merge (squash)

June 25, 2026 07:13

serhiy-storchaka merged commit a00464b into python:main

Jun 25, 2026

94 of 96 checks passed

Uh oh!

There was an error while loading. Please reload this page.

bedevere-app Bot removed the awaiting core review label

Jun 25, 2026

eendebakpt reviewed

Jun 25, 2026

View reviewed changes

eendebakpt left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the PR was just merged. Two minor comments anyway

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

Lib/re/_optimizer.py

		from ._constants import *

		_CHARSET_ALL = [(NEGATE, None)]
		_UNIT_CODES = {LITERAL, NOT_LITERAL, ANY, IN, CATEGORY}

eendebakpt Jun 25, 2026

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are moving around quite some code anyway, should we change to

Suggested change

	_UNIT_CODES = {LITERAL, NOT_LITERAL, ANY, IN, CATEGORY}
	_UNIT_CODES = frozenset({LITERAL, NOT_LITERAL, ANY, IN, CATEGORY})

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

serhiy-storchaka Jun 25, 2026

Copy link

Copy Markdown

Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was a set before. frozenset does not have any advantage over set.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

Lib/re/_optimizer.py


		"""Internal support module for sre.

		Optimization passes used by the compiler: character-set optimization

eendebakpt Jun 25, 2026

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _compile_charset is not listed here (but it is imported from this module).

Since these things can get outdated, I am also fine with leaving out this docstring

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

serhiy-storchaka Jun 25, 2026

Copy link

Copy Markdown

Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be updated in follow up anyway.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

serhiy-storchaka deleted the re-optimizer branch

July 1, 2026 16:47

eendebakpt mentioned this pull request

Nov 1, 2026

Regression in re.compile() performance #161125

Open

Reviewers

eendebakpt

eendebakpt left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.