GeezerVison is showing github.com — Un-deprecate functional API for importlib resources & add subdirectory support · Issue #116608 · python/cpython
Un-deprecate functional API for importlib resources & add subdirectory support #116608DescriptionFeature or enhancement
Proposal:
The importlib.resources functions {open,read}_{text,binary}, path, is_resource and contents, deprecated in 3.11 and removed in 3.13 alphas, are, anecdotally, missed by quite a few users.
They provide a simple API for simple tasks, while the full-featured Traversable API is better suited for complex ones -- especially for implementing new resources-aware loaders.
I'm now in a position where I can add these functions back and support them.
Their main drawback -- not allowing subdirectories -- can be solved by taking multiple path components as positional arguments, for example:
importlib.resources.read_text('modulename', 'subdirectory', 'subsubdir', 'resource.txt')The additional arguments (encoding and errors) would become keyword-only.
There is a wrinkle in this: in Python 3.9-3.11, the above would mean:
importlib.resources.read_text(
'modulename', 'subdirectory',
encoding='subsubdir',
errors='resource.txt',
)I believe that this is acceptable, since:
pragmatically: typical file names do not match typical encoding/errorhandler nameslawyerly: the functions have already been deprecated for 2 releases; no one is using them now, right?
However, if this is a problem, I can
[edit: This is solved by:]
- make the
encoding argument required if a text-reading function more than one path component is given.
- plan to lift this limitation around 3.15.
importlib.resources.read_text(
'modulename', 'subdirectory', 'subsubdir', 'resource.txt',
encoding='utf-8',
)
importlib.resources.read_text('modulename', 'resource.txt') # OK
importlib.resources.read_text('modulename', 'subdirectory', 'utf-8') # errorHas this already been discussed elsewhere?
I have already discussed this feature proposal on Discourse
Links to previous discussion of this feature:
https://discuss.python.org/t/deprecating-importlib-resources-legacy-api/11386/29
Linked PRs
Reacted by Jason R. Coombs and Shahriar Heidrich Reacted by Miro Hrončok and Tim Hutt ActivityFFY00 commented on Mar 11, 2024 Wouldn't this be a bit late for that? We already went through the deprecation period, and removed the feature in the alpha releases, bringing them back now would be a bit confusing.
The importlib.resources functions {open,read}_{text,binary}, path, is_resource and contents, deprecated in 3.11 and removed in 3.13 alphas, are, anecdotally, missed by quite a few users.
Can you actually show a couple examples of this affecting users downstream? I think that's the most viable argument to bring that API back. zooba commented on Mar 11, 2024
taking multiple path components as positional arguments
Why not just take multiple components with separators in a single argument? It's easy enough to require forward slash, disallow .. and even to normalise backslashes on Windows if you feel like it.
If they didn't allow subdirectories before (I never noticed, tbh), then presumably using a slash here would have either failed completely or worked. Either way, we can enable them in a new release.
(And add me to the anecdotal list of people who missed them. It's easy enough to add a few lines of code to bring them back, which is how I have been handling it so far, but I'd be happier to have those few lines in the stdlib.) jaraco commented on Mar 11, 2024 At least one audience that would like to keep the legacy APIs is in mesonbuild/meson#12401.
I admit, I prefer this approach over keeping the legacy APIs with the cruft that it still had lying around. It adds a mostly-compatible layer and restores these wrappers in a supported way.
On one hand, this approach violates the "preferably one way" to do things; users will need to decide which way works best for them, creating a variety of supported approaches. On the other hand, I do appreciate that it offers a friendlier interface for certain operations (esp. path(...)).
FFY00 and I put a lot of work into this deprecation process, so it'll be disappointing to now see this reversed at the last minute, but it does feel like the right thing to do, especially since someone else is willing to own the implementation (thanks encukuo!). We will have to backport the change to importlib_resources, but that should be fairly straightforward.
Overall, I'm +0 on the change. I'd really like to see more vocal support from other core devs before committing to this approach. Reacted by Petr Viktorin and Daniël van Noord Reacted by Wim Jeantine-Glenn encukou commented on Mar 12, 2024 I've made the encoding argument mandatory for _text functions when multiple path names are given.
Wouldn't this be a bit late for that?
Yes, sorry. Previously I couldn't commit to supporting this API.
Why not just take multiple components with separators in a single argument?
I'd rather not derail discussion on this issue. Support for separators can be added later if necessary. If they will, allowing multiple arguments will still be useful.
FFY00 and I put a lot of work into this deprecation process
Sorry to hear that. Sunk costs suck :(
This makes it seem that implementing the deprecation process was similarly (or more) time-consuming as keeping the API working. That's not a good situation to be in, especially considering all the work users need to put in to update their code. pfmoore commented on Mar 12, 2024
(And add me to the anecdotal list of people who missed them. It's easy enough to add a few lines of code to bring them back, which is how I have been handling it so far, but I'd be happier to have those few lines in the stdlib.)
I’ll add a “me too” here as well. Being able to do simple things simply is an advantage. pradyunsg commented on Mar 12, 2024 +1 from me on the (updated) proposed API as well as un-deprecating these -- for reasons that have been discussed on the d.p.o thread as well as mentioned here by others. eli-schwartz commented on Mar 12, 2024
Their main drawback -- not allowing subdirectories -- can be solved by taking multiple path components as positional arguments, for example:
importlib.resources.read_text('modulename', 'subdirectory', 'subsubdir', 'resource.txt')
I've always sort of wondered why this is a drawback at all, compared to simply doing this:
with importlib.resources.path('modulename.subdirectory.subsubdir', 'resources.txt') as f:
...I'm not objecting to the new API! It's more ergonomic than pretending everything is a namespace module. But for backwards compatibility with python < 3.13 it seems practical to use the two-argument form, and the lack of a new API doesn't seem like it should have been a killer problem before now. zooba commented on Mar 12, 2024 I don't understand the reason we can't reimplement it as:
def read_text(module, filename, *args, **kwargs): #use proper args if you want here, I just don't know them all off the top of my head
with (path(module) / filename).open("r", *args, **kwargs) as f:
return f.read()
Why do we need the module and filename as multiple args instead of just two? zooba commented on Mar 12, 2024
I'd rather not derail discussion on this issue.
How is it derailing this issue? You're bringing back an API, which I like, and changing the design in a potentially backwards-incompatible way in the process, which I don't. Why is it derailing to ask why it has to have a different design now? barneygale commented on Mar 12, 2024 Reacted by Steve Dower, Petr Viktorin and Wim Jeantine-Glenn jaraco commented on Mar 12, 2024 barneygale commented on Mar 12, 2024 IIUC:
- Support for resources in subdirectories required new APIs, e.g. something resembling
Traversible.iterdir().
- The pathlib API was considered a good fit and chosen for the task.
- The functional interface wasn't enhanced to support subdirectories; instead it was earmarked for eventual removal.
So there's perhaps four levels of support we could offer for the functional APIs:
- Deprecate + remove (@jaraco's original plan, already landed)
- Restore old APIs + functionality as it was.
- Restore old APIs, and add support for subdirectories to existing functions (this issue and @encukou's PR)
- Restore old APIs, add add support for subdirectories, including adding new functions where needed (e.g. functional equivalent of
traversable.iterdir())
Personally I'd lean towards option 2. If folks need subdirectory support they can use the OOP API - that's it's whole reason to exist! Reacted by Gregory P. Smith graingert commented on Mar 31, 2025 This was undeprecated in 3.13 but the undeprecation should be backported to 3.12 and 3.11 gamer191 commented on Aug 10, 2026 I can see this was un-deprecated by https://github.com/python/cpython/pull/132206/changes in Python 3.12.10.
Can I suggest updating the documentation (at https://docs.python.org/3/library/importlib.resources.html) to add a note next to un-deprecated functions along the lines of Changed in version 3.12.10: No longer raises DeprecationWarning so that programs targeting older Python versions know not to use those functions (especially because programs might expect compatibility with all Python 3.12 versions if they run their CI on the latest minor version of Python 3.12)
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|