GeezerVison

Python 3.12’s beta’s are out, which means the features are locked in. The theme this year has been cleanup and typing. distutils has been removed, and setuptools is no longer present in default environments.

Faster CPython

The faster CPython project is still working on features, though most of the changes this time around don’t seem to affect daily performance as much as 3.11 did. A lot of work went into the per interpreter GIL, which will be covered in its own section later. There still are some nice user-facing improvements though.

Comprehensions are now inlined, making them up to 2x faster. This does affect scoping inside comprehensions; just in case you are using stacklevel=n to go up through a comprehension in a warning, you’d need to reduce n by 1 in 3.12.

inspect.getattr_static has been made much faster (2-6x!); and this now is used in the implementation of typing.runtime_checkable, so that checking isinstance on a Protocol is much faster for smallish Protocols. This is a huge win for readability and typing, as this matters in tight loops in libraries like Rich; this should reduce the need for the manual performance-oriented workarounds they employ today.

os.stat() on Windows is more accurate, and also faster. asyncio.current_task is 4-6x faster. And f-strings tokenize faster, as well, though I’ll cover f-strings in more detail below.

Error messages

The suggestion features keep getting better, based heavily on reports from people teaching Python about common mistakes. Trying to use a stdlib module without importing it, reversing the order of from and import, and forgetting self. in front of an attempted attribute access now have customized error messages. And now when you try to import something that doesn’t exist from a module, you’ll get suggestions based on what’s actually in the module! F-strings also get much better error messages, but more on those below.

Typing

Generics

There were a lot of typing improvements this round, including a brand new syntax for generics! This is common in other statically-typed languages, but might seem a bit odd if you’ve not seen one of those before. Here’s an example, compared with the classic TypeVar method, and C++:

def f[T](x: T) -> T:
    return 2 * x

from typing import TypeVar

T = TypeVar("T")


def f(x: T) -> T:
    return 2 * x

template<typename T>
T f(T x) {
    return 2*x;
}

Bounds and constraints are supported too:

def f[T: numbers.Real](x: T) -> T:
    return 2 * x

from typing import TypeVar

T = TypeVar("T", bound=numbers.Real)


def f(x: T) -> T:
    return 2 * x

template <typename Data>
concept Numeric = std::is_arithmetic_v<Data>;

template<Numeric T>
T f(T x) {
    return 2*x;
}

And you can also make generic type aliases in one line:

type Vector3D[T] = tuple[T, T, T]

This also works if the type alias is not generic, so most usage of TypeVar and TypeAlias (and ParamSpec and TypeVarTuple, as * and ** are both supported) are no longer needed.

Unlike most typing improvements, this is not available in older Python versions, even with from __future__ import annotations.

Other typing improvements

You can now use a TypedDict for **kwargs! To do so, you need the new typing.Unpack[T] wrapper, since the normal syntax for **kwargs just includes the values.

There’s now a typing.override decorator, to indicate that you intended to override a method.

array.array is now Generic.

Native f-strings

F-strings are now a native part of the syntax rather than a thousand-plus line hack on strings. This means the following is now valid:

msg = f"{x["y"]}"

The nested quotes are fine, because the stuff inside the brackets is normally parsed Python! Multiple lines, escape codes, all that stuff now works like normal Python, and not like string contents. This also means error messages now look like normal Python and can access the correct places in the line(s). And tokenizing the f-strings for parsing is 64% faster!

Per-interpreter GIL

One of the biggest features, and one with possibly the most performance potential, is the per-interpreter GIL. This is a rather odd new feature, though, because it landed without a Python interface; you have to use the C API for now. Though don’t worry, there will be a first-party PyPI module providing a Python API, and that will help guide the development of a proper standard library API, probably in 3.13.

The API will probably look something like this:

# Get "interpreters" from somewhere for now
interp = interpreters.create()
script = "print('Hello world')"
interp.run(script)

This would also integrate with threading:

t = Thread(target=interp.run, args=(script,))

The interpreters-3-12 module on PyPI contains initial work for this. There are other plans and hopes, like a dedicated API for data passing, but that’s the core idea for now. These interpreters each have their own GIL, so that means that with effort, you can run Python fully multithreaded in a single process! See the examples in ericsnowcurrently/interpreters for more.

Other features

The buffer protocol can now be used from Python, with __buffer__, making it a proper (and static-typable!) Protocol (available as collections.abc.Buffer). And a few other things:

• New calendar.Month and calendar.Day enums.
• New itertools.batched() which collects into batches, where the last one could be shorter.
• New math.sumprod, sum of products.
• is_junction and better access to Windows drives/mounts/volumes in os and pathlib.
• Path/PurePath finally subclassable
• Path.walk() finally added.
• pathlib globbing and matching now has a case_sensitive option.
• shutil.which improved on Windows
• New command line interface to sqlite3 and a few new features.
• types.get_original_bases() to help with inspecting generics.
• --durations added to unittest’s command line option (a rare addition to unittest). Lots of removals from unittest.

Removals

Distutils has been removed from the standard library. Setuptools is no longer installed by default when you make a new venv (or use the third-party virtualenv on Python 3.12), or when you run ensurepip. You really should be providing at least a pyproject.toml with the following content:

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

Dropping this in with your current setup.py is enough to be ready for the future, though you should also look into modern Python backends like hatchling or C++ backends like scikit-build-core for a simpler and more elegant packaging experience.

Other removed libraries include asynchat and asyncore. There were also removals from unittest, configparser, sqlite, importlib, and more.

Other Developer changes

As usual, there are some more technical changes that will excite some people:

• Linux perf is now supported (-X perf or PYTHONPERFSUPPORT)
• Tarfile now supports a filter option, with a new safer default coming in 3.14.
• Slices are now hashable (and therefore can be set items or dict keys!)
• Sum uses Neumaier summation for more accurate floating point sums
• Moving to using a single exc argument in various places, getting rid of (typ, exc, tb) tuples. Move over before 3.14 removes the old way of doing this!
• Reduced the size of unicode by removing wstr/wstr_length.
• Added a new Unstable C-API, which can change between versions, intended for JIT compilers and debuggers.
• Added PyType_FromMetaclass and support for vectorcall in the Limited API - as a result, 3.12 is the first version Nanobind supports in Limited API / Stable ABI mode!
• Immortal objects added.

Final Words

If you are using GitHub Actions, the new and best way to add 3.12 is to use this:

- uses: actions/setup-python@v4
  with:
    python-version: "3.12"
    allow-prereleases: true

This works in a matrix, etc. too. Also, note that setup-python recently started supporting setting up multiple Python’s at once, with a range - very useful if you are using nox or tox, for example.

And if you are using cibuildwheel, we’ve supported this since beta 1 with the following flag:

- uses: pypa/[email protected]
  with:
    CIBW_ALLOW_PRERELEASES: true

If you are using NumPy, you might be in for a wait. The just-released version of numpy (1.25) does not support Python 3.12; the next release (1.26) will complete the migration to a new build backend, and is supposed to come up “when 3.12 is released”. I don’t know if that means final release, RC 1, or some future beta.

Sources and other links

• Official docs