GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Expand documentation for importlib.metadata custom providers to include caveats regarding backports · Issue #123976 · python/cpython

Expand documentation for importlib.metadata custom providers to include caveats regarding backports #123976

New issue
Copy link
New issue
Copy link
Open
Open
Expand documentation for importlib.metadata custom providers to include caveats regarding backports#123976
Copy link
Assignees
jaraco
Labels
docsDocumentation in the Doc dirDocumentation in the Doc dirtopic-importlib

Description

@jaraco
jaraco
opened on Sep 11, 2024
Issue body actions

Documentation

As importlib_metadata provides a backport of importlib.metadata and asserts primacy when imported, for consumers of the API, it provides a consistent interface, except when a custom provider serves objects as imported from importlib.metadata (not using the backport). For example:

  • project A implements and installs a DistributionFinder to sys.meta_path providing importlib.metadata.Distribution instances.
  • project B calls importlib_metadata.distributions() to return all distributions, including that of project A.
  • A user installs project A and project B.
  • When project B queries for the distributions, expecting it to return Iterable[importlib_metadata.Distribution] objects, their expectation is violated when the DistributionFinder installed by project A emits importlib.metadata.Distribution objects.

python/importlib_metadata#486 captures this concern. python/importlib_metadata#505 promises to correct this concern by wrapping and transforming any objects from custom providers referencing importlib.metadata to instead use objects from importlib_metadata.

However, for importlib.metadata.Distribution objects, it's not possible to replace safely the objects of an arbitrary custom provider, so the proposed implementation emits a warning when such an object is encountered.

Therefore, we'd like to add some documentation to the custom provider docs to advise custom providers on what to do to avoid their users encountering the warning. In particular:

  • Custom providers should emit importlib_metadata.Distribution objects when importlib_metadata is present, or
  • Custom providers should register their objects with importlib_metadata to bypass the warning.

Update: the protocols landed, and need syncing

python/importlib_metadata#505 merged 2026-10-23 and released in importlib_metadata 10.0.0. It takes the shared-protocol approach settled on in python/importlib_metadata#486 rather than either recommendation above:

  • IDistribution and IPackagePath protocols join the existing PackageMetadata, so providers and consumers have one interface that the classes of both libraries satisfy.
  • Objects a provider supplies from importlib.metadata are adapted to this package's classes where they can be. An arbitrary Distribution subclass cannot be, so it is passed through, and the warning now fires only where the class omits members IDistribution requires, naming them, rather than merely because it derives from importlib.metadata.

So neither recommendation above is needed: a provider implements the interface, and a conforming one draws no warning. gh-160344 drafts the documentation on that basis.

That documentation names the protocols, so this issue now also covers getting them into the stdlib. Under the development methodology, _meta.py carries the protocols and syncs downstream, while the compat and _compat packages holding localize and its warning are excluded from the port. Only the protocols arrive here, which is what gh-160344 assumes.

Lib/importlib/metadata/ was last synced at importlib_metadata 8.7 (gh-132947, 3.15.0a1). The current release is 10.1.0, so a sync spans 9.0.0, 9.0.1, 9.1.0, 10.0.0 and 10.1.0 — among them the MetadataNotFound and KeyError changes, lazy EntryPoint.value validation, the metadata directory preference added in 9.1.0 and reverted in 10.1.0, and the Prepared.normalized and metadata detection performance work. Some of those originated in CPython and have already been ported back, so the diff wants tracing rather than taking wholesale.

Tasks

  • Sync Lib/importlib/metadata/ with importlib_metadata 10.1.0
  • Document the custom provider caveats (gh-160386: Apply changes from importlib_metadata 10.0 and document the shared protocols #160344), which depends on the sync

Linked PRs

  • gh-123976: Refresh docs around custom providers. #123977
  • gh-160386: Apply changes from importlib_metadata 10.0 and document the shared protocols #160344

Activity

  1. jaraco
    added
    docsDocumentation in the Doc dirDocumentation in the Doc dir
    on Sep 11, 2024
  2. jaraco
    self-assigned this
    on Sep 11, 2024
  3. jaraco
    added
    topic-importlib
    on Sep 11, 2024
  4. jaraco commented on Sep 11, 2024

    @jaraco
    jaraco
    on Sep 11, 2024
    MemberAuthor
    More actions

    As I write that recommendation, I'm already uncomfortable with the recommendation. It's unclear what "when importlib_metadata is present" means. Does it mean the module is available for import? Does it mean it's been imported previously (e.g. in sys.modules)?

    I think it has to mean it's available for import, because the provider can't tell if it will later be imported by a consumer. But that also means the provider might activate importlib_metadata in an environment that would otherwise have never had it activated. Perhaps that's not a huge concern - if it's installed, it's likely to be activated.

    That also means that users of importlib.metadata will be getting importlib_metadata.Distribution objects, violating the type spec. But that's the case already today, so perhaps that's not so bad.

    Also, "register with importlib_metadata to bypass the warning" sounds messy at best. There probably aren't a lot of providers out there, but even so, having to make new releases that are required to support their custom provider does not sound like a lot of fun for anyone.

  5. jaraco commented on Sep 11, 2024

    @jaraco
    jaraco
    on Sep 11, 2024
    MemberAuthor
    More actions

    All of this has me re-thinking the strategy.

    Maybe instead of having two distinct class trees (importlib.metadata.Distribution and importlib_metadata.Distribution and subclasses), there should be a protocol that's shared by both versions of the library, explicitly modeling what's currently implicit - that there's an interface expected by Distribution objects (also PackagePath and Message objects), and those protocols are what the two libraries are coded against.

  6. jaraco
    added 3 commits that reference this issue on Sep 11, 2024

    pythongh-123976: Refresh docs around custom providers.

    72d6b70

    pythongh-123976: Refresh docs around custom providers.

    a6b3a49

    gh-123976: Refresh docs around custom providers. (#123977)

    a47cd21
  7. hugovk commented on Sep 15, 2024

    @hugovk
    hugovk
    on Sep 15, 2024
    Member
    More actions

    Triage: #123977 has been merged, ready to close this?

  8. jaraco commented on Sep 15, 2024

    @jaraco
    jaraco
    on Sep 15, 2024
    MemberAuthor
    More actions

    Thanks, but no. The PR only did some minor refreshing of the docs. The core concern of this issue remains.

    Reacted by Hugo van Kemenade
  9. jaraco
    added 3 commits that reference this issue on Oct 22, 2026

    pythongh-123976: Warn only on an incomplete provider

    ab57ac7

    pythongh-123976: Separate the consumer and provider caveats

    69a4d48

    pythongh-123976: Document the protocols shared with the backport

    be9a5de
  10. jaraco
    added 3 commits that reference this issue on Oct 31, 2026

    Merge remote-tracking branch 'upstream/main' into docs/pythongh-123976-…

    ce987c6

    pythongh-160386: Apply changes from importlib_metadata 10.0

    294c0df

    pythongh-123976: Move the backport caveats to the backport's docs

    cb1a2b7
  11. warsaw
    added a commit that references this issue on Nov 7, 2026

    Merge branch 'main' into docs/pythongh-123976-backport-caveats

    f234adb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

  • @jaraco
    jaraco

Labels

docsDocumentation in the Doc dirDocumentation in the Doc dirtopic-importlib

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    You are visitor 01402520 — GeezerVison does not store the pages you look up.
    Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.