GeezerVison is showing github.com — Improve the docs regarding the migration from imp to importlib · Issue #104212 · python/cpython
Improve the docs regarding the migration from imp to importlib #104212DescriptionNow that the imp removal has landed on main, users migrating to Python3.12 will likely need help to move to importlib.
The current imp docs have tips on how to do just that. Great!
One caveat: the imp.load_source has been removed from the docs a long time ago, now it is only visible is the Python2 version of the docs. So users of imp.load_source cannot rely on the docs to help them migrate, and have to Google this. The first results on stackoverflow are a bit wrong:
- many point to
SourceFileLoader(...).load_module(), but this is also deprecated and slated for removal in 3.12 (according to the warning)
- some solutions point to
importlib.util.spec_from_file_location, but this does not work with files that do not end with ".py"
The solution that I think is the best translation:
def imp_load_source(module_name, module_path):
loader = SourceFileLoader(module_name, module_path)
module = types.ModuleType(loader.name)
loader.exec_module(module)
return moduleI think it would be beneficial to have that kind of information in the docs. Unfortunately, imp.load_source is not officially documented, but there are several GitHub issues and SO threads discussing how to migrate code to importlib. IMO, we should do one of:
- add it back to the docs, with explanations on how to migrate it to
importlib
- just document the migration in the release notes, in the section "Porting to Python 3.12"
What do you think?
Linked PRs
Reacted by Michael R. Crusoe Activitywarsaw commented on May 5, 2023 Maybe we need a porting guide in the importlib docs for porting older imp and other APIs to importlib? I don't like that imp module has been deleted but the docs are still there. Reacted by Hugo van Kemenade hugovk commented on May 5, 2023 Good idea!
In fact we were just discussing this earlier today and thought about putting a migration guide in both the imp and importlib docs and in the 3.12 porting guide.
cc @vstinner terryjreedy commented on May 5, 2023 In current 3.12, 'imp' is no longer present in the module index, which suggests that it is gone. vstinner commented on May 5, 2023 I don't know the importlib design, so maybe it doesn't make sense, but... Would it make sense to add a imp_load_source() function to importlib? Maybe under a different name. vstinner commented on May 5, 2023 CAM-Gerlach commented on May 6, 2023
Would it make sense to add a imp_load_source() function to importlib?
Just to note, see, e.g., #58756 for some previous discussion of this. vstinner commented on May 24, 2023 The recipe didn't work for me:
- (1) The module has no
__file__ variable
- (2) The module was no registered in
sys.modules
imp.load_source() defines __file__ and adds the module to sys.modules.
I used this recipe instead:
def load_module(module_name, filename):
loader = importlib.machinery.SourceFileLoader(module_name, filename)
module = types.ModuleType(loader.name)
module.__file__ = filename
sys.modules[module.__name__] = module
loader.exec_module(module)
return moduleReacted by Alex Prengère, romainsc and mroelandts vstinner commented on Jun 13, 2023 Minor step forward: I documented how to replace removed imp.new_module(): 457a459 brettcannon commented on Jun 16, 2023
- some solutions point to
importlib.util.spec_from_file_location, but this does not work with files that do not end with ".py"
That's actually only true if you don't pass in the loader you need to use as the code has to guess as what you're after. You can use either importlib.util.spec_from_loader() or importlib.util.spec_from_file_location(), they just need different specifics to do the right thing. brettcannon commented on Jun 23, 2023
I'm not sure neither how find_module() was used with load_module() to load modules. Is the use case to specify a search path which is not in sys.path?
I think so.
Why not putting the path in sys.path and simply use importlib.import_module()?
Worried they would forget to clean up sys.path after importing the one file they wanted? vstinner commented on Jun 25, 2023
add it back to the docs, with explanations on how to migrate it to importlib
just document the migration in the release notes, in the section "Porting to Python 3.12"
That's now done in What's New in Python 3.12, but under the "Removed" section, where removals are mentioned. I dislike documenting the same removal at two places.
https://docs.python.org/dev/whatsnew/3.12.html#removed
I just wrote a 3.12 backport for my second doc change: PR #106083. It'ss going to be merged soon.
I added a recipe to replace imp.load_source(). I consider that this documentation issue is now solved and I close the issue.
If someone wants a more complete explanation for a specific removed imp function, please open a new issue.
Thanks @alexprengere for the bug report, thanks @brettcannon and @arhadthedev for the reviews. Hoeze commented on Jan 30, 2026 The docs now state to replace imp.load_source() with this:
import importlib.util
import importlib.machinery
def load_source(modname, filename):
loader = importlib.machinery.SourceFileLoader(modname, filename)
spec = importlib.util.spec_from_file_location(modname, filename, loader=loader)
module = importlib.util.module_from_spec(spec)
# The module is always executed and not cached in sys.modules.
# Uncomment the following line to cache the module.
# sys.modules[module.__name__] = module
loader.exec_module(module)
return moduleIs there a reason why this function was not directly put into importlib.util? brettcannon commented on Feb 2, 2026
Is there a reason why this function was not directly put into importlib.util?
- There is a lot of variance people want for similar functionality as the code comment hints at.
- It avoids the import system while stuff in
importlib.util is meant to help use it in some advanced ways.
- Not everything needs to be in the stdlib.
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|