GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-104212: Explain how to port imp code to importlib by vstinner · Pull Request #105905 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Jun 19, 2023 •
edited by github-actions Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Explain in What's New in Python 3.12 how to port existing code using the removed imp to the importlib module.

  • Issue: Improve the docs regarding the migration from imp to importlib  #104212

📚 Documentation preview 📚: https://cpython-previews--105905.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting core review docs Documentation in the Doc dir skip news labels Jun 19, 2023
@bedevere-bot bedevere-bot mentioned this pull request Jun 19, 2023
Improve the docs regarding the migration from imp to importlib #104212
Closed
@vstinner

vstinner commented Jun 19, 2023

Copy link
Copy Markdown
Member Author

Replacing load_dynamic() is non-trivial:

def load_dynamic(name, filename):
    loader = importlib.machinery.ExtensionFileLoader(name, filename)
    spec = importlib.util.spec_from_loader(name, loader)
    module = importlib.util.module_from_spec(spec)
    sys.modules[module.__name__] = module
    loader.exec_module(module)
    return module

If you omit loader.exec_module(), you get an uninitialized extension which lead to funny crashes :-)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 19, 2023

Copy link
Copy Markdown
Member Author

Replace init_builtin():

def init_builtin(name):
    spec = importlib.machinery.BuiltinImporter.find_spec(name)
    if spec is None:
        raise ImportError(f'no built-in module named {name!r}')
    return importlib.util.module_from_spec(spec)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 19, 2023

Copy link
Copy Markdown
Member Author

For removed imp.load_source(), I proposed adding importlib.util.load_source_path(): PR #105755.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

arhadthedev
arhadthedev reviewed Jun 19, 2023
View reviewed changes
Comment thread Doc/whatsnew/3.12.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 19, 2023

Copy link
Copy Markdown
Member Author

Replace imp.load_package():

import sys
import os.path
import importlib

def load_package(name, path):
    old_path = list(sys.path)
    try:
        sys.path.insert(0, os.path.dirname(path))
        return importlib.import_module(name)
    finally:
        sys.path.clear()
        sys.path.extend(old_path)

I'm not sure about this one. It uses high-level API to call indirectly PathFinder and FileFinder API. I would prefer to not expose them since I consider them as the internal API.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 19, 2023

Copy link
Copy Markdown
Member Author

For removed imp.load_source(), I proposed adding importlib.util.load_source_path(): PR #105755.

I suppose that it can be used to replace imp.load_cached().

Otherwise, here is a recipe to replace imp.load_cached():

import importlib.util
import sys

def load_compiled(name, filename):
    spec = importlib.util.spec_from_file_location(name, filename)
    module = importlib.util.module_from_spec(spec)
    sys.modules[module.__name__] = module
    spec.loader.exec_module(module)
    return module

Test:

import os.path

def create_pyc():
    name = 'script'
    filename = f'/tmp/{name}.py'
    with open(filename, "w") as fp:
        print("print('hello')", file=fp)

    sys.path.insert(0, os.path.dirname(filename))
    mod = importlib.import_module(name)

    del sys.path[0]
    del sys.modules[name]
    os.unlink(filename)
    mod = None

    return importlib.util.cache_from_source(filename)

print("create PYC")
pyc = create_pyc()

print(f"load PYC: {pyc}")
mod = load_compiled('script', pyc)
print(mod)
print(mod.__file__)
print(mod.__cached__)
os.unlink(pyc)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Jun 19, 2023

Copy link
Copy Markdown
Member Author

My recipes always create a new module: they don't try to get a cached module from sys.modules, whereas the imp module respected sys.modules cache.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
pythongh-104212: Explain how to port imp code to importlib
fab77c8
Explain in What's New in Python 3.12 how to port existing code using
the removed imp to the importlib module.
@vstinner
vstinner force-pushed the port_imp branch from 5a2cb98 to fab77c8 Compare June 19, 2023 12:57
vstinner added 2 commits June 19, 2023 15:14
@vstinner
Fix references
f856e57
@vstinner
Remove dot
2d7a01a
@vstinner
vstinner merged commit 7a56a41 into python:main Jun 19, 2023
@bedevere-bot bedevere-bot removed the awaiting core review label Jun 19, 2023
@vstinner
vstinner deleted the port_imp branch June 19, 2023 14:13
@vstinner vstinner added the needs backport to 3.12 only security fixes label Jun 20, 2023
@miss-islington

miss-islington commented Jun 20, 2023

Copy link
Copy Markdown
Contributor

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Jun 20, 2023

Copy link
Copy Markdown

GH-105952 is a backport of this pull request to the 3.12 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.12 only security fixes label Jun 20, 2023
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 20, 2023
@vstinner @miss-islington
pythongh-104212: Explain how to port imp code to importlib (pythonGH-…
4121570
…105905)

(cherry picked from commit 7a56a41)

Co-authored-by: Victor Stinner <[email protected]>
vstinner added a commit that referenced this pull request Jun 20, 2023
@miss-islington @vstinner
[3.12] gh-104212: Explain how to port imp code to importlib (GH-105905)…
5729b41
… (#105952)

gh-104212: Explain how to port imp code to importlib (GH-105905)
(cherry picked from commit 7a56a41)

Co-authored-by: Victor Stinner <[email protected]>
@AA-Turner AA-Turner mentioned this pull request Sep 22, 2023
GH-109190: Copyedit 3.12 What's New: Update the imp porting guidance #109755
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@arhadthedev arhadthedev arhadthedev left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

docs Documentation in the Doc dir skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@vstinner @miss-islington @bedevere-bot @arhadthedev
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.