GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Support non-package modules. by jaraco · Pull Request #258 · python/importlib_resources

Conversation

@jaraco

@jaraco jaraco commented Jul 22, 2022

Copy link
Copy Markdown
Member
  • Add test capturing current behavior and missed expectation. Ref Unable to load resources for modules #203.
  • Remove restriction that a 'package' cannot be a module. Allows resolution of resources from adjacent modules, even those not found in a package. Fixes Unable to load resources for modules #203.
  • Update changelog. Ref Unable to load resources for modules #203.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@jaraco
jaraco requested a review from warsaw July 22, 2022 18:21
@jaraco

jaraco commented Jul 22, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

@warsaw I'd appreciate your review of this change, especially since you were involved so much in the discussion on #60. The behavior change is entirely in the second commit ecffc07. Note how it simplifies the code, aligns with pkg_resources behavior, and enables use-cases that were otherwise unreachable (resources for non-package modules). Is there any reason we shouldn't allow users to supply a module as an indicator for where to locate resources?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jaraco

jaraco commented Jul 23, 2022

Copy link
Copy Markdown
Member Author

See jaraco/openpack@82a5d0b for an illustration of how this change simplifies and harmonizes resource loading.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

FFY00
FFY00 reviewed Jul 27, 2022
View reviewed changes
Comment thread importlib_resources/tests/test_files.py
_path.build(spec, self.site_dir)
import mod

actual = resources.files(mod).joinpath('res.txt').read_text()

@FFY00 FFY00 Jul 27, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is surprising to me. I would expect, a pathlib.Path('...', 'mod.py') to be returned, requiring resources.files(mod).parent.joinpath('res.txt'). res.txt is not a resource of mod, but rather of the parent module, which might not even exist if mod is top level.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@jaraco jaraco Jul 29, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for sharing your surprise. Surely this would need some improved documentation to illustrate how files() now accepts an Anchor (instead of Package), where the anchor indicates:

  • the container (importer) from where resources can be discovered,
  • if a module is indicated, that's equivalent to indicating the container of that module,
  • if a top-level module is indicated, that will anchor on resources reachable at the top level by that same importer.

This is the behavior indicated by pkg_resources:

Note that if a module name is used, then the resource name is relative to the package immediately containing the named module.

However, I've also observed that the pkg_resources implementation also allows for loading resources against unpackaged modules. And while that behavior may have been unintentional, it works because it's a natural consequence of relying on the importer/loader to resolve the resources from the same location as the indicated package/module. That is, because the importlib mechanism supports loading modules from outside packages, so too should resources be loadable from that location.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@jaraco jaraco Jul 29, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

res.txt is not a resource of mod, but rather of the parent module, which might not even exist if mod is top level.

A parent module may not exist, but a loader for that module does exist and can load resources adjacent to that module:

 importlib_resources feature/203-non-package-modules $ touch setup.py
 importlib_resources feature/203-non-package-modules $ .tox/python/bin/python -iq
>>> import importlib_resources as res
>>> files = res.files('setup')
>>> files
PosixPath('/Users/jaraco/code/public/importlib_resources')
>>> files.joinpath('pyproject.toml').read_text()[:10]
'[build-sys'

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@jaraco jaraco Oct 5, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In 1f3c226, I've updated the documentation to honor an Anchor type which need not be a package.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
jaraco added 3 commits October 5, 2022 10:25
@jaraco
Add test capturing current behavior and missed expectation. Ref #203.
94d59b6
@jaraco
Remove restriction that a 'package' cannot be a module. Allows resolu…
5b1281d
…tion of resources from adjacent modules, even those not found in a package. Fixes #203.
@jaraco
Update changelog. Ref #203.
a60496b
@jaraco
jaraco force-pushed the feature/203-non-package-modules branch from 1923197 to 1f3c226 Compare October 5, 2022 14:26
@jaraco
Update docs to prefer the name 'anchor' to 'package', as the anchor m…
8c3edeb
…ay be a module.
@jaraco
jaraco force-pushed the feature/203-non-package-modules branch from 1f3c226 to 8c3edeb Compare October 5, 2022 14:30
jaraco added 4 commits October 5, 2022 11:45
@jaraco
Extend type spec for clarity.
091f229
@jaraco
Correct the changelog to mention the correct function affected.
4c672fb
@jaraco
Rename the 'package' parameter to 'anchor'.
eb7f123
@jaraco
Add doctests to flesh out coverage.
4e63613
@jaraco
jaraco force-pushed the feature/203-non-package-modules branch from 2ff7b03 to 4e63613 Compare October 7, 2022 17:29
@jaraco
jaraco merged commit 19ca481 into main Oct 7, 2022
@jaraco
jaraco deleted the feature/203-non-package-modules branch October 7, 2022 23:47
@yan12125 yan12125 mentioned this pull request Dec 24, 2023
buildbot.errors.PluginDBError when importing custom www plugin buildbot/buildbot#7306
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@FFY00 FFY00 FFY00 left review comments
@warsaw warsaw Awaiting requested review from warsaw

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

Sprint
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Unable to load resources for modules

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@jaraco @FFY00
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.