GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — gh-143658: Use `str.lower` and `replace` to further improve performance of `importlib.metadata.Prepared.normalized` by hugovk · Pull Request #144083 · python/cpython

Conversation

hugovk commented Jan 20, 2026 •

edited by bedevere-app Bot

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Member

Follow on from #143660 which replaced re.sub with str.translate.

@henryiii discovered that whilst that is an improvement on Python 3.10-3.11 and 3.14, the performance of str.translate is much worse on Python 3.12 and 3.13, and can be worse than the original re.sub.

Further, his fix in pypa/packaging#1064 to replace str.translate with str.lower and str.replace calls is better than both the other options across all Python versions, to varying degrees.

I benchmarked all three across all available Python Build Standalone versions from 3.10-3.15 on macOS:

I also benchmarked Windows and Ubuntu it's the same pattern.

Whilst the improvement isn't as large for CPython 3.15 as it is for libraries such as packaging that support a wide range of Pythons, importlib.metadata does have a backport, and I'll update python/importlib_metadata#529 once this is merged.

Issue: importlib.metadata: Use str.translate to improve performance of importlib.metadata.Prepared.normalized #143658

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hyongtao-code reacted with thumbs up emoji

All reactions

👍 1 reaction


          Use str.lower and replace to further improve performance of importlib…

b95d70b

….metadata.Prepared.normalized

Co-authored-by: Henry Schreiner <[email protected]>

hugovk requested review from jaraco and warsaw as code owners

January 20, 2026 20:09

hugovk added performance

Performance or resource usage

topic-importlib labels

Jan 20, 2026

bedevere-app Bot added the awaiting core review label

Jan 20, 2026

hugovk changed the title ~~gh-143660: Use str.lower and replace to further improve performance of importlib.metadata.Prepared.normalized~~ gh-143658: Use str.lower and replace to further improve performance of importlib.metadata.Prepared.normalized

Jan 20, 2026

bedevere-app Bot mentioned this pull request

Jan 20, 2026

importlib.metadata: Use str.translate to improve performance of importlib.metadata.Prepared.normalized #143658

Closed

hugovk mentioned this pull request

Jan 20, 2026

gh-143658: importlib.metadata: Use str.translate to improve performance of importlib.metadata.Prepared.normalized #143660

Merged

hauntsaninja approved these changes

Jan 20, 2026

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting core review labels

Jan 20, 2026

hugovk commented Jan 22, 2026

Copy link

Copy Markdown

Member Author

Attached code for benchmarking and making the chart, and the results data from my macOS machine:

bench.zip

And charts and data of benchmarking on macOS, Ubuntu and Windows via GitHub Actions:

macos-ubuntu-windows.zip

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

sergey-miryanov approved these changes

Jan 23, 2026

View reviewed changes

sergey-miryanov left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

brettcannon approved these changes

Jan 23, 2026

View reviewed changes

hugovk commented Feb 6, 2026

Copy link

Copy Markdown

Member Author

This packaging change has been in version 26.0 for 16 days now with 306,207,835 downloads no problems, let's merge :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hugovk merged commit 28fb13c into python:main

Feb 6, 2026

57 checks passed

Uh oh!

There was an error while loading. Please reload this page.

hugovk deleted the 3.15-importlib.metadata-normalized branch

February 6, 2026 17:39

bedevere-app Bot removed the awaiting merge label

Feb 6, 2026

hugovk mentioned this pull request

Feb 6, 2026

Use str.translate to improve performance of normalize python/importlib_metadata#529

Closed

bedevere-bot commented Feb 6, 2026

Copy link

Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 FreeBSD Refleaks 3.x (tier-3) has failed when building commit 28fb13c.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1613/builds/2796) and take a look at the build logs.
Check if the failure is related to this commit (28fb13c) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1613/builds/2796

Test leaking resources:

test_events: memory blocks
test_events: references

Summary of the results of the build (if available):

Click to see traceback logs

Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.ware-freebsd.refleak/build/Lib/test/support/__init__.py", line 947, in gc_collect
    gc.collect()
    ~~~~~~~~~~^^
ResourceWarning: unclosed <socket.socket fd=9, family=2, type=1, proto=6, laddr=('127.0.0.1', 20202), raddr=('127.0.0.1', 20203)>
Task was destroyed but it is pending!
task: <Task pending name='Task-2918' coro=<BaseSelectorEventLoop._accept_connection2() done, defined at /buildbot/buildarea/3.x.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py:217> wait_for=<Future pending cb=[Task.task_wakeup()]>>
Warning -- Unraisable exception
Exception ignored while calling deallocator <function _SelectorTransport.__del__ at 0x8470965d0>:
Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py", line 873, in __del__
    _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
    ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ResourceWarning: unclosed transport <_SelectorSocketTransport closing fd=9>
k


Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.ware-freebsd.refleak/build/Lib/test/support/__init__.py", line 947, in gc_collect
    gc.collect()
    ~~~~~~~~~~^^
ResourceWarning: unclosed <socket.socket fd=10, family=2, type=1, proto=6, laddr=('127.0.0.1', 14643), raddr=('127.0.0.1', 14644)>
Task was destroyed but it is pending!
task: <Task pending name='Task-98' coro=<BaseSelectorEventLoop._accept_connection2() done, defined at /buildbot/buildarea/3.x.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py:217> wait_for=<Future pending cb=[Task.task_wakeup()]>>
Warning -- Unraisable exception
Exception ignored while calling deallocator <function _SelectorTransport.__del__ at 0x8428925d0>:
Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.ware-freebsd.refleak/build/Lib/asyncio/selector_events.py", line 873, in __del__
    _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
    ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ResourceWarning: unclosed transport <_SelectorSocketTransport closing fd=10>
k

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

thunder-coding pushed a commit to thunder-coding/cpython that referenced this pull request

Feb 15, 2026


          pythongh-143658: Use str.lower and replace to further improve per…

9ecbd9f

…formance of `importlib.metadata.Prepared.normalized` (python#144083)

Co-authored-by: Henry Schreiner <[email protected]>

jaraco reviewed

Mar 20, 2026

View reviewed changes

Comment thread

Lib/importlib/metadata/__init__.py

		"""
		PEP 503 normalization plus dashes as underscores.
		"""
		# Emulates ``re.sub(r"[-_.]+", "-", name).lower()`` from PEP 503

jaraco Mar 20, 2026

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change removes this important note that links the PEP 503 prescription from the actual implementation.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

jaraco reviewed

Mar 20, 2026

View reviewed changes

Comment thread

Lib/importlib/metadata/__init__.py

		# Emulates ``re.sub(r"[-_.]+", "-", name).lower()`` from PEP 503
		# About 3x faster, safe since packages only support alphanumeric characters
		value = name.translate(_normalize_table)
		# Much faster than re.sub, and even faster than str.translate

jaraco Mar 20, 2026

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is now out of place. It's commenting on historical content.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

jaraco commented Mar 20, 2026

Copy link

Copy Markdown

Member

These changes also need to be ported to importlib_metadata to avoid being overwritten when syncing that repo.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

hugovk commented Mar 20, 2026

Copy link

Copy Markdown

Member Author

This change needs to be ported to importlib_metadata to avoid getting overridden when syncing changes from that repo.

Thanks for the review, please see PR python/importlib_metadata#529, and let's fix the comments over there.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ljfp pushed a commit to ljfp/cpython that referenced this pull request

Apr 25, 2026


          pythongh-143658: Use str.lower and replace to further improve per…

9e04a8f

…formance of `importlib.metadata.Prepared.normalized` (python#144083)

Co-authored-by: Henry Schreiner <[email protected]>

jaraco mentioned this pull request

Jul 14, 2026

Revert normalize() performance changes (gh-143658) python/importlib_metadata#541

Merged

Reviewers

jaraco

jaraco left review comments

brettcannon

brettcannon approved these changes

sergey-miryanov

sergey-miryanov approved these changes

hauntsaninja

hauntsaninja approved these changes

warsaw

Awaiting requested review from warsaw

warsaw is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

performance

Performance or resource usage

topic-importlib

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

hugovk commented Jan 20, 2026 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hugovk commented Jan 22, 2026

Uh oh!

sergey-miryanov left a comment

Choose a reason for hiding this comment

Uh oh!

hugovk commented Feb 6, 2026

Uh oh!

Uh oh!

bedevere-bot commented Feb 6, 2026

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

jaraco Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

jaraco Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

jaraco commented Mar 20, 2026

Uh oh!

hugovk commented Mar 20, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

hugovk commented Jan 20, 2026 •

edited by bedevere-app Bot

Loading

Uh oh!

There was an error while loading. Please reload this page.