A compile-time optimization for the set operations added in gh-152100. No behaviour change -- same matches, fewer engine steps.
The parser lowers set difference [A--B] to A(?<![B]): a character class followed by a negative lookbehind. But the engine's charset() walk treats every NEGATE as a polarity toggle (see Modules/_sre/sre_lib.h), so one character set can express the difference directly -- [NEGATE] B [NEGATE] A matches A minus B in a single charset test instead of a charset match plus a lookbehind rescan.
A new pass in Lib/re/_optimizer.py performs two folds:
- Set difference
[A--B] (or an explicit A(?<![B])) into the single charset [NEGATE] B [NEGATE] A. _optimize_charset is made segment-aware so the interior NEGATE compiles correctly; the parser keeps emitting the plain A(?<![B]) form.
- A union with a non-flat operand such as
[0-9||[a-z--b]] is left by the parser as a BRANCH it cannot merge; once its alternatives are all one-character matchers, their item lists are concatenated into one IN.
Besides removing the lookbehind rescan, the fused charset also becomes the INFO prefix, so the engine can fast-skip non-matching positions. On scan-heavy input this is several times faster (findall over 50 KB of non-matching text, best of 7):
[a-z--aeiou] 5.6x
[\w--\d] 5.3x
[0-9||[a-z--b]] 9.4x
On match-dense input the per-match win is smaller, masked by match-object construction.
🤖 Generated with Claude Code