Abstract

This PEP relaxes the constraint on dynamic metadata listed in the [project] section in pyproject.toml. It is now permitted to define a static portion of a dynamic metadata field in the [project] table as long as the field is a table or array. Likewise, METADATA 2.6 allows mixed static and dynamic metadata to be specified in source distribution metadata.

This allows users to opt into allowing a backend to extend metadata while still keeping the static portions of the metadata defined in the standard location in pyproject.toml, and allows inspection tools to still be able to process the static portions of the metadata.

Motivation

In the core metadata specification originally set out in PEP 621, metadata can be specified in three ways. First, it can be listed in the [project] table. This makes it statically inferable, meaning any tool (not just the build backend) can reliably compute the value. Second, a field can be listed in the project.dynamic list, which allows the build backend to compute the value. Finally, a value could be missing from both the project table and the project.dynamic list, in which case the matching metadata is guaranteed to be empty.

This system provided two important benefits to Python packaging. A standard specification that all major backends have now adopted makes teaching much easier; a single tutorial is now sufficient to cover the metadata portion of configuring any backend. Users can now switch from a general purpose backend to a specialized backend without changing their static metadata. Tooling like schema validation tools can verify and catch configuration mistakes.

The second benefit is improved support for static tools that read the source files looking for metadata. This is useful for dependency chain analysis, such as creating “used by” and “uses” graphs. It is used for code quality tooling to detect the minimum supported version of Python. It is used by cibuildwheel to automatically avoid building wheels that are not supported. It is not used, however, to avoid wheel builds when the SDist is available; that was addressed by METADATA 2.2, which added a Dynamic field in the SDist metadata that lets a tool know if the metadata can change when making a wheel - this is an easy mistake to make due to the similarity of the names.

Due to the rapidly increasing popularity of the project table, support from all major backends, and a rise of backends supporting complex compiled extensions, an issue with the restrictions applied in PEP 621 is becoming more apparent. In PEP 621, the metadata choice is all-or-nothing; metadata must be completely static, or listed in the dynamic field and completely absent from the static definition. For the most common use cases, this is fine; there is little benefit to set the version statically if you are going to override it dynamically. If you are using a custom README processor to filter or modify the README for proper display, it’s not a big deal to have to specify the configuration in a custom [tool.*] section. But there is a specific class of cases where the all-or-nothing approach is problematic: lists of items where the backend needs to add items are currently forced to be fully dynamically specified (that is, in a backend-specific configuration location). This causes both of the original benefits (standard location and static tooling support) to be lost.

Rationale

PEP 621 includes the following statement:

In an earlier version of this PEP, tools were allowed to extend data for fields. For instance, build back-ends could take the version number and add a local version for when they built the wheel. Tools could also add more trove classifiers for things like the license or supported Python versions.

In the end, though, it was thought better to start out stricter and contemplate loosening how static the data could be considered based on real-world usage.

In this PEP, we are proposing a limited and explicit loosening of the restrictions on the [project] table and project.dynamic list.

Every list and every table with arbitrary keys will now be allowed to be specified both statically, in the [project] table, and in the project.dynamic list. If it is present in both places, the build backend can extend list items and add new keys, but not modify existing entries.

[project]
dynamic = ["dependencies"]

[tool.my-build-backend]
original-dependencies = ["torch", "packaging"]
pin-to-build-versions = ["torch=={exact}"]

Dynamic: Requires-Dist
Requires-Dist: packaging
Requires-Dist: torch

Requires-Dist: packaging
Requires-Dist: torch
Requires-Dist: torch==2.8.0

[project]
dependencies = ["torch", "packaging"]
dynamic = ["dependencies"]

[tool.my-build-backend]
pin-to-build-versions = ["torch=={exact}"]

Terminology

The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

Specification

Any field that is comprised of a list or a table with arbitrary entries will now be allowed to be present in both the [project] table and the project.dynamic list. If a field is present in both places, then the build backend is allowed to insert entries into the list or table, but not remove entries, reorder entries, or modify the entries. Tables of arrays allow adding a new table entry or extending an existing array according to the rules above.

The fields that are arrays or tables with arbitrary entries are:

• authors, maintainers: New author tables can be added to the list. Existing authors cannot be modified (list of tables with pre-defined keys).
• classifiers: Classifiers can be added to the list.
• dependencies: New dependencies can be added, including more tightly constrained existing dependencies.
• entry-points: Entry points can be added, to either new or existing groups. Existing entry points cannot be changed or removed.
• scripts, gui-scripts: New scripts can be added. Existing ones cannot be changed or removed.
• keywords: Keywords can be added to the list.
• license-files: Files can be added to the list.
• optional-dependencies: A new extra or new items can be added to an existing extra.
• urls: New urls can be added. Existing ones cannot be changed or removed.
• import-names, import-namespaces: New import names or namespaces can be added. Existing ones cannot be modified or removed.

To add items, users must opt in by listing the field in dynamic; without that, the metadata continues to be entirely static.

A backend SHOULD error if a field is specified and it does not support extending that field, to protect against possible user error. We recommend being as strict as possible to avoid unnecessary entries in the dynamic list.

Static analysis tools, when detecting a field is both specified and in the project.dynamic array, SHOULD assume the field is incomplete, allowing for new entries to be present when the package is built.

The Dynamic field, as specified in PEP 643, is unaffected by this PEP, and backends can continue to fill it as they choose. However, a backend MUST ensure that both the SDist and the wheel metadata include the static metadata portion of the project table.

In METADATA 2.2 to 2.5, there are no constraints on entries listed as Dynamic (which means a wheel can have different metadata than the SDist for that field). Now, metadata fields specified in the SDist are guaranteed to also be in the wheel, even if Dynamic is present. The METADATA version will be incremented to 2.6. Given this example:

Dynamic: Requires-Dist
Requires-Dist: packaging

Before METADATA 2.6, there are no constraints on a field if it appeared in Dynamic, so the wheel could contain anything; now in 2.6, any values here are guaranteed to be also present in wheels, so the wheel will contain Requires-Dist: packaging; it may contain more Requires-Dist, but it will contain at least that one.

This new point will be added to the guidelines:

• If a multiple use field is present in a source distribution and also marked Dynamic, a wheel can add values, but it must include the one(s) present in the SDist. Keywords, Author-Email, and Maintainer-Email are comma-separated lists, and can likewise be extended if present.

Reference Implementation

The choice to support dynamic metadata for each field is already left up to backends, and this PEP simply relaxes restrictions on what a backend is allowed to do with dynamic metadata.

The pyproject-metadata project, which is used by several build backends, will need to modify the correctness check to account for the possible extensions; this is in a draft PR.

The dynamic-metadata project, which provides a plugin system that backends can use to share dynamic metadata plugins, was designed to allow this possibility, and a similar PR to the one above will allow additive metadata.

Backwards Compatibility

This does not affect any existing pyproject.toml files, since this was strictly not allowed before this PEP.

When users adopt this in a pyproject.toml, the backend must support it; an error will be correctly generated if it doesn’t, following the previous standard. Frontends were never required to throw an error, though some frontends may need to be updated to benefit from the partially static metadata. Some frontends and other tooling may need updating, such as schema validators, just like other pyproject.toml PEPs.

Static analysis tools may require updating to handle this change. Tools should check the dynamic table first, like this:

match pyproject["project"]:
    # New in PEP 808
    case {my.key: value, "dynamic": dyn} if my.key in dyn:
        print(f"Partial {my.key}: {value}")
    case {"dynamic": dyn} if my.key in dyn:
        print(f"Fully dynamic {my.key}")
    case {my.key: value}:
        print(f"Fully static {my.key}: {value}")
    case _:
        print(f"No metadata for {my.key}")

Before this PEP, tools could reverse the order of the dynamic and static blocks, assuming that an entry in the project table meant it could not be dynamic. If they do this, they will now incorrectly assume they have all the metadata for a field, when they in fact only have part of it.

Security Implications

There are no security concerns that are not already present, as this just adds a static component to existing dynamic metadata support.

How to Teach This

If you currently have dynamic metadata, but some list or table entries are known statically, you can now make that explicit by adding the static entry in the [project] table, while also keeping the entry in the project.dynamic list to allow the dynamic portion to be added by your build backend.

The current guides that state metadata must not be listed in both [project] and project.dynamic can be updated to say that lists and tables marked with project.dynamic can still have static entries. Since dynamic metadata is already an advanced concept, this will likely not affect most existing tutorial material aimed at introductory packaging.

The pyproject.toml specification will be updated to include the behavior of fields when specified and also listed in the dynamic field.

It should also be noted that specifying something in dynamic will require any tool that needs the full metadata to invoke the backend even if it is partially statically specified. So, it should not be used unless necessary, just like any other dynamic metadata.

Rejected Ideas

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.