GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-118761: substitute `re` import in `base64.b16decode` for a more efficient alternative by picnixz · Pull Request #128736 · python/cpython

Conversation

@picnixz

@picnixz picnixz commented Jan 11, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Benchmarks are on a RELEASE build (no PGO, no LTO).

See #128736 (comment) for the runtime performance improvements as well.

PR

$ ./python -I -X importtime -c 'import base64'
import time: self [us] | cumulative | imported package
...
import time:       179 |        179 | linecache
import time:       205 |        205 |     _struct
import time:       442 |        646 |   struct
import time:       227 |        227 |   binascii
import time:       298 |       1169 | base64
$ hyperfine --warmup 16 "./python -c 'import base64'"
Benchmark 1: ./python -c 'import base64'
  Time (mean ± σ):       5.7 ms ±   0.5 ms    [User: 4.8 ms, System: 1.1 ms]
  Range (min … max):     5.2 ms …  11.9 ms    455 runs

Main

$ ./python -I -X importtime -c 'import base64'
import time: self [us] | cumulative | imported package
...
import time:       180 |        180 | linecache
import time:       312 |        312 |       types
import time:      1609 |       1921 |     enum
import time:        87 |         87 |       _sre
import time:       250 |        250 |         re._constants
import time:       388 |        637 |       re._parser
import time:        89 |         89 |       re._casefix
import time:       339 |       1150 |     re._compiler
import time:       103 |        103 |         itertools
import time:        83 |         83 |         keyword
import time:        58 |         58 |           _operator
import time:       206 |        264 |         operator
import time:       129 |        129 |         reprlib
import time:        49 |         49 |         _collections
import time:       785 |       1411 |       collections
import time:        43 |         43 |       _functools
import time:       543 |       1997 |     functools
import time:       122 |        122 |     copyreg
import time:       826 |       6015 |   re
import time:       115 |        115 |     _struct
import time:        74 |        189 |   struct
import time:       122 |        122 |   binascii
import time:       279 |       6603 | base64
$ hyperfine --warmup 16 "./python -c 'import base64'"
Benchmark 1: ./python -c 'import base64'
  Time (mean ± σ):       9.5 ms ±   0.4 ms    [User: 8.2 ms, System: 1.3 ms]
  Range (min … max):     9.1 ms …  13.5 ms    295 runs
  • Issue: Improve import time of various stdlib modules #118761

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
picnixz added 2 commits January 11, 2025 12:59
@picnixz
Improve import time of the base64 module.
3a92fce
Importing `base64` is now up to six times faster.

The `re` module is now locally imported by `base64.b16decode`
and is no more implicitly exposed as `base64.re`.
@picnixz
blurb
0b52712
@bedevere-app bedevere-app Bot mentioned this pull request Jan 11, 2025
Improve import time of various stdlib modules #118761
Closed
@bedevere-app bedevere-app Bot added the awaiting core review label Jan 11, 2025
@picnixz picnixz added the performance Performance or resource usage label Jan 11, 2025
@picnixz
picnixz requested a review from hugovk January 11, 2025 16:30
@picnixz

picnixz commented Jan 11, 2025

Copy link
Copy Markdown
Member Author

@hugovk I'm requesting your review since you've commented on the issue. There are also #128732 and #128738 to review FYI.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

AA-Turner
AA-Turner reviewed Jan 12, 2025
View reviewed changes
Comment thread Lib/base64.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

picnixz added 2 commits January 13, 2025 13:23
@picnixz
Improve base64.b16decode() by a factor 10.
d7b6253
This entirely removes the needs to a regex.
@picnixz
update NEWS
9e8fe5d
hugovk
hugovk approved these changes Jan 13, 2025
View reviewed changes

@hugovk hugovk left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Misc/NEWS.d/next/Library/2025-01-10-13-06-54.gh-issue-118761.f8oADD.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Jan 13, 2025
@picnixz @hugovk
Update Misc/NEWS.d/next/Library/2025-01-10-13-06-54.gh-issue-118761.f…
5be6178
…8oADD.rst

Co-authored-by: Hugo van Kemenade <[email protected]>
@picnixz picnixz changed the title gh-118761: improve import time for base64 gh-118761: substitute re import in base64.b16decode for a more efficient alternative Jan 13, 2025
@picnixz
add What's New entry
b36d915
@picnixz
picnixz requested a review from AA-Turner January 14, 2025 12:26
AA-Turner
AA-Turner reviewed Jan 14, 2025
View reviewed changes
Comment thread Doc/whatsnew/3.14.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Misc/NEWS.d/next/Library/2025-01-10-13-06-54.gh-issue-118761.f8oADD.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

picnixz and others added 2 commits January 14, 2025 13:43
@picnixz @AA-Turner
Update Doc/whatsnew/3.14.rst
5562c81
Co-authored-by: Adam Turner <[email protected]>
@picnixz @AA-Turner
Update Misc/NEWS.d/next/Library/2025-01-10-13-06-54.gh-issue-118761.f…
82d18d7
…8oADD.rst

Co-authored-by: Adam Turner <[email protected]>
@picnixz picnixz mentioned this pull request Jan 14, 2025
gh-118761: improve import time for secrets #128738
Closed
@picnixz
picnixz requested review from AA-Turner and hugovk January 14, 2025 13:19
AA-Turner
AA-Turner approved these changes Jan 14, 2025
View reviewed changes
@AA-Turner
AA-Turner merged commit bbd3300 into python:main Jan 14, 2025
@bedevere-app bedevere-app Bot removed the awaiting merge label Jan 14, 2025
@AA-Turner

AA-Turner commented Jan 14, 2025

Copy link
Copy Markdown
Member

Thanks!

A

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz
picnixz deleted the perf/import/base64-118761 branch January 14, 2025 13:49
@vstinner

vstinner commented Jan 14, 2025

Copy link
Copy Markdown
Member

@AA-Turner merged commit bbd3300 into python:main Jan 14, 2025

FYI @picnixz was recently promoted as a core dev and so can merge his own changes.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz

picnixz commented Jan 14, 2025

Copy link
Copy Markdown
Member Author

(I don't mind others merging my PRs by the way)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@chris-eibl

chris-eibl commented Jan 14, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

and reduce the import time of :mod:base64 by up to six times.

9.5 ms down to 5.7 ms is impressive but not six times - rather 60%?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz

picnixz commented Jan 14, 2025

Copy link
Copy Markdown
Member Author

hyperfine benchmarks also take into account interpreter's startup and so, while -X importtime only takes into account the time needed to import the module itself (hence the differences).

chris-eibl reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@chris-eibl

chris-eibl commented Jan 14, 2025

Copy link
Copy Markdown
Member

Sure, my bad. Importtime itself is of course six times faster.

Thank you all for the great work here in cPython!

picnixz reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@picnixz picnixz mentioned this pull request Jan 17, 2025
gh-118761: Reduce import time of gettext.py by delaying re import #128898
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AA-Turner AA-Turner AA-Turner approved these changes
@hugovk hugovk Awaiting requested review from hugovk
+1 more reviewer
@effigies effigies effigies left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

performance Performance or resource usage

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@picnixz @AA-Turner @vstinner @chris-eibl @effigies @hugovk
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.