GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — [3.12] gh-117983: Defer import of threading for lazy module loading (GH-120233) by miss-islington · Pull Request #121350 · python/cpython

Conversation

@miss-islington

@miss-islington miss-islington commented Jul 3, 2024

Copy link
Copy Markdown
Contributor

As noted in gh-117983, the import importlib.util can be triggered at
interpreter startup under some circumstances, so adding threading makes
it a potentially obligatory load.
Lazy loading is not used in the stdlib, so this removes an unnecessary
load for the majority of users and slightly increases the cost of the
first lazily loaded module.

An obligatory threading load breaks gevent, which monkeypatches the
stdlib. Although unsupported, there doesn't seem to be an offsetting
benefit to breaking their use case.

For reference, here are benchmarks for the current main branch:

❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
  Time (mean ± σ):       9.7 ms ±   0.7 ms    [User: 7.7 ms, System: 1.8 ms]
  Range (min … max):     8.4 ms …  13.1 ms    313 runs

And with this patch:

❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
  Time (mean ± σ):       8.4 ms ±   0.7 ms    [User: 6.8 ms, System: 1.4 ms]
  Range (min … max):     7.2 ms …  11.7 ms    352 runs

Compare to:

❯ hyperfine -w 8 './python -c pass'
Benchmark 1: ./python -c pass
  Time (mean ± σ):       7.6 ms ±   0.6 ms    [User: 5.9 ms, System: 1.6 ms]
  Range (min … max):     6.7 ms …  11.3 ms    390 runs

This roughly halves the import time of importlib.util.
(cherry picked from commit 94f50f8)

Co-authored-by: Chris Markiewicz [email protected]

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@effigies @miss-islington
pythongh-117983: Defer import of threading for lazy module loading (p…
38da7b7
…ythonGH-120233)

As noted in pythongh-117983, the import importlib.util can be triggered at
interpreter startup under some circumstances, so adding threading makes
it a potentially obligatory load.
Lazy loading is not used in the stdlib, so this removes an unnecessary
load for the majority of users and slightly increases the cost of the
first lazily loaded module.

An obligatory threading load breaks gevent, which monkeypatches the
stdlib. Although unsupported, there doesn't seem to be an offsetting
benefit to breaking their use case.

For reference, here are benchmarks for the current main branch:

```
❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
  Time (mean ± σ):       9.7 ms ±   0.7 ms    [User: 7.7 ms, System: 1.8 ms]
  Range (min … max):     8.4 ms …  13.1 ms    313 runs
```

And with this patch:

```
❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
  Time (mean ± σ):       8.4 ms ±   0.7 ms    [User: 6.8 ms, System: 1.4 ms]
  Range (min … max):     7.2 ms …  11.7 ms    352 runs
```

Compare to:

```
❯ hyperfine -w 8 './python -c pass'
Benchmark 1: ./python -c pass
  Time (mean ± σ):       7.6 ms ±   0.6 ms    [User: 5.9 ms, System: 1.6 ms]
  Range (min … max):     6.7 ms …  11.3 ms    390 runs
```

This roughly halves the import time of importlib.util.
(cherry picked from commit 94f50f8)

Co-authored-by: Chris Markiewicz <[email protected]>
@miss-islington
miss-islington requested review from brettcannon, encukou, ericsnowcurrently, ncoghlan and warsaw as code owners July 3, 2024 20:51
This was referenced Jul 3, 2024
GH-114781 potentially breaks gevent: threading becomes pre-imported at startup #117983
Closed
gh-117983: Defer import of threading for lazy module loading #120233
Merged
@bedevere-app bedevere-app Bot added the awaiting review label Jul 3, 2024
brettcannon
brettcannon approved these changes Jul 3, 2024
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Jul 3, 2024
@brettcannon
brettcannon enabled auto-merge (squash) July 3, 2024 20:54
@brettcannon brettcannon self-assigned this Jul 3, 2024
@brettcannon
brettcannon merged commit 81f5754 into python:3.12 Jul 3, 2024
@bedevere-app bedevere-app Bot removed the awaiting merge label Jul 3, 2024
@miss-islington
miss-islington deleted the backport-94f50f8-3.12 branch January 2, 2026 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@brettcannon brettcannon brettcannon approved these changes
@encukou encukou Awaiting requested review from encukou encukou is a code owner
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner
@ncoghlan ncoghlan Awaiting requested review from ncoghlan ncoghlan is a code owner
@warsaw warsaw Awaiting requested review from warsaw warsaw is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@brettcannon brettcannon

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@miss-islington @brettcannon @effigies
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.