GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-130167: Improve speed of `inspect.formatannotation` by replacing `re` by donbarbos · Pull Request #130242 · python/cpython

Conversation

@donbarbos

@donbarbos donbarbos commented Feb 18, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

timeit benchmark with my script:

inspect_bench.py:

import re
import timeit
from typing import List, Dict, Any, Union

def old_format_annotation(annotation):
    if getattr(annotation, "__module__", None) == "typing":
        def repl(match):
            text = match.group()
            return text.removeprefix("typing.")
        return re.sub(r"[\w\.]+", repl, repr(annotation))
    return repr(annotation)

def new_format_annotation(annotation):
    if getattr(annotation, "__module__", None) == "typing":
        return (repr(annotation)
                .replace(".typing.", "@TYPING@")
                .replace("typing.", "")
                .replace("@TYPING@", ".typing."))
    return repr(annotation)

old_time = timeit.timeit(lambda: old_format_annotation(Union[List[str], Dict[str, Any]]), number=100_000)
new_time = timeit.timeit(lambda: new_format_annotation(Union[List[str], Dict[str, Any]]), number=100_000)

print(f"Old version (re.sub): {old_time:.6f}s")
print(f"New version (.replace()): {new_time:.6f}s")
print(f"Difference: {old_time / new_time:.2f}x")

Result: 2.0s -> 1.18s = x1.74 as fast

$ ./python -B inspect_bench.py
Old version (re.sub): 2.043900s
New version (.replace()): 1.175867s
Difference: 1.74x

✔️ This was also the only use of re in module so i got rid of the import

  • Issue: Improve speed of stdlib functions by replacing re uses #130167

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@donbarbos
Improve speed of inspect.formatannotation by replacing re
ef19ae6
@bedevere-app bedevere-app Bot added the awaiting review label Feb 18, 2025
@donbarbos donbarbos changed the title Improve speed of inspect.formatannotation by replacing re gh-130167: Improve speed of inspect.formatannotation by replacing re Feb 18, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Feb 18, 2025
Improve speed of stdlib functions by replacing re uses #130167
Open
donbarbos added 5 commits February 18, 2025 04:54
@donbarbos
Merge branch 'main' into improve-inspect-speed
7dc08b7
@donbarbos
Add blurb
d9480c7
@donbarbos
Update NEWS.d entry markup
ab85241
@donbarbos
Merge branch 'main' into improve-inspect-speed
f48b2ed
@donbarbos
Update rst syntax in News entry
3c1c4b7
hugovk
hugovk reviewed Mar 27, 2025
View reviewed changes
Comment thread Misc/NEWS.d/next/Library/2025-02-18-05-04-13.gh-issue-130167.aOgAz_.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

donbarbos and others added 2 commits March 28, 2025 01:58
@donbarbos @hugovk
Update 2025-02-18-05-04-13.gh-issue-130167.aOgAz_.rst
1bfbe05
Co-authored-by: Hugo van Kemenade <[email protected]>
@donbarbos
Remove implementation detail from News entry
0ecf16f
AA-Turner
AA-Turner reviewed May 1, 2025
View reviewed changes

@AA-Turner AA-Turner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm less sure this is worth it, the replacement isn't clearly better maintenence-wise.

What do the benchmarks look like if you extract repl to a module-level _formatannotation_repl and use .sub() on a pre-compiled pattern?

A

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@donbarbos

donbarbos commented May 1, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

sorry, I forgot how python was compiled then :( but I'm pretty sure I compiled it with gcc last time

so I reran the test with the following config args '--enable-optimizations' '--with-lto' 'CC=/usr/bin/clang-18' and got a new result for my bench:

Old version (re.sub): 0.982520s
New version (.replace()): 0.701049s
Difference: 1.40x # sometimes it's 1.35x

and if we rewrite old version of benchmark as you suggested:

_pattern = re.compile(r"[\w\.]+")

def _repl(match):
    text = match.group()
    return text.removeprefix("typing.")

def old_format_annotation(annotation):
    if getattr(annotation, "__module__", None) == "typing":
        return re.sub(_pattern, _repl, repr(annotation))
    return repr(annotation)

we will get a bigger difference:

Old version (re.sub): 1.027774s
New version (.replace()): 0.698902s
Difference: 1.47x # the result can drop to 1.40

P.S.: another reason i like the PR version is because we get rid of the re import that is used once here

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Engelbarts

Engelbarts commented Sep 3, 2025

Copy link
Copy Markdown

Out of curiosity, why use the little replace dance in the refactored code instead of just return repr(annotation).removeprefix("typing.")?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@donbarbos
donbarbos requested a review from AA-Turner October 22, 2025 13:49
@charmander

charmander commented Nov 27, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Out of curiosity, why use the little replace dance in the refactored code instead of just return repr(annotation).removeprefix("typing.")?

- Union[typing.List[str], typing.Dict[str, typing.Any]]
+ Union[List[str], Dict[str, Any]]

The replace dance is pretty messy, though, and doesn’t preserve behaviour for footyping.Bar – not that applying a regex to repr output is perfectly reliable to begin with (def foo() -> Literal["typing."]: ...).

For what it’s worth:

pat = re.compile(r"\b(?<!\.)typing\.")
pat.sub("", repr(annotation))
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions

github-actions Bot commented Apr 21, 2026

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 21, 2026
@eendebakpt

eendebakpt commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

The PR is stale and has some issues, so I am closing it.

@charmander (or others) Your suggestion looks good (I tested it locally, performance gain is good). Would you like to open a PR?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@eendebakpt eendebakpt closed this Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hugovk hugovk hugovk left review comments
@AA-Turner AA-Turner Awaiting requested review from AA-Turner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review stale Stale PR or inactive for long period of time.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@donbarbos @Engelbarts @charmander @eendebakpt @hugovk @AA-Turner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.