GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — `_py_abc` Python implementation of `abc` is not thread-safe · Issue #130095 · python/cpython

_py_abc Python implementation of abc is not thread-safe #130095

New issue
Copy link
New issue
Copy link
Open
Open
_py_abc Python implementation of abc is not thread-safe#130095
Copy link
Labels
stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Description

@colesbury
colesbury
opened on Feb 13, 2025
Issue body actions

Bug report

The update to the invalidation counter is not thread-safe and can lose updates in some Python implementations:

Failures seen on:

  • Python 3.14t
  • Python 3.9
  • pypy3.10
  • pypy3.11

But not on Python 3.10-3.14 with GIL due to limited GIL switch opportunities.

cpython/Lib/_py_abc.py

Lines 54 to 70 in 05e89c3

def register(cls, subclass):
"""Register a virtual subclass of an ABC.
Returns the subclass, to allow usage as a class decorator.
"""
if not isinstance(subclass, type):
raise TypeError("Can only register classes")
if issubclass(subclass, cls):
return subclass # Already a subclass
# Subtle: test for cycles *after* testing for "already a subclass";
# this means we allow X.register(X) and interpret it as a no-op.
if issubclass(cls, subclass):
# This would create a cycle, which is bad for the algorithm below
raise RuntimeError("Refusing to create an inheritance cycle")
cls._abc_registry.add(subclass)
ABCMeta._abc_invalidation_counter += 1 # Invalidate negative cache
return subclass

For example, consider the following repro, adapted from test_abc.test_registration_basics:

import _py_abc as abc # Use Python implementation of ABCs!!
import threading
import os
import sys

sys.setswitchinterval(1e-6)

N = 5

def run(b):
    b.wait()

    class A(metaclass=abc.ABCMeta):
        pass
    A.register(int)
    if not isinstance(42, A):
        print("Oops!")
        os._exit(1)

def main():
    for _ in range(10000):
        threads = []
        b = threading.Barrier(N)
        for _ in range(N):
            t = threading.Thread(target=run, args=(b,))
            threads.append(t)
            t.start()
        for t in threads:
            t.join()


if __name__ == "__main__":
    main()

Linked PRs

  • gh-130095: Mark _py_abc tests as not thread-safe #130131

Activity

  1. colesbury
    added
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    on Feb 13, 2025

  2. colesbury commented on Feb 13, 2025

    @colesbury
    colesbury
    on Feb 13, 2025
    ContributorAuthor
    More actions

    I'm not really sure what to do here. We could add a threading.Lock() to protect _abc_invalidation_counter, but I'm not sure we want to import the threading module in _py_abc.py.

    cc @cfbolz in case you have thoughts on this.

  3. encukou
    added
    stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory
    on Feb 14, 2025

  4. picnixz commented on Feb 14, 2025

    @picnixz
    picnixz
    on Feb 14, 2025
    Member
    More actions

    Not sure if this is relevant: https://github.com/python/cpython/actions/runs/13335219139/job/37248893358?pr=129175 (I don't know if the test is testing the Python or the C implementation).

    EDIT: I think this is the reproducer you had. But now we caught it in the CI

    Reacted by Sam Gross
  5. colesbury
    added a commit that references this issue on Feb 14, 2025

    pythongh-130095: Mark _py_abc tests as not thread-safe

    e880c6e

  6. colesbury commented on Feb 14, 2025

    @colesbury
    colesbury
    on Feb 14, 2025
    ContributorAuthor
    More actions

    Thanks @picnixz. I put up a PR to avoid running them with --parallel-threads for now until we figure out if and how to make _py_abc thread-safe.

  7. cfbolz commented on Feb 14, 2025

    @cfbolz
    cfbolz
    on Feb 14, 2025
    Contributor
    More actions

    yeah, importing threading in _py_abc leads to a circular import problem if _abc is not available. What about using itertools.count? that's supposed to be thread-safe, isn't it?

  8. cfbolz commented on Feb 14, 2025

    @cfbolz
    cfbolz
    on Feb 14, 2025
    Contributor
    More actions

    hm, doesn't quite work, because itertools.count has no way to read the current value, only a way to increase the count and get the next value.

  9. sergey-miryanov commented on Feb 14, 2025

    @sergey-miryanov
    sergey-miryanov
    on Feb 14, 2025
    Contributor
    More actions

    If I'm right, the problem comes from three bytecode operations: LOAD_ATTR, BINARY_OP, and STORE_ATTR. I think there are two ways to fix this: using a specialized operation for atomic increment/update or the same built-in function. Current main branch has only three uses of _Py_atomic_add_uint64 - qsbr, _asynciomodule and _abc, so there are no existing ways to do atomic update from python. I believe the quickest way to add private built-in for this, but the right one to add atomic ops.

    Sorry if I'm obvious.

  10. colesbury commented on Feb 14, 2025

    @colesbury
    colesbury
    on Feb 14, 2025
    ContributorAuthor
    More actions

    The pure Python companion modules like _py_abc aren't really used by CPython in normal circumstances outside of tests. They are primarily used by other Python implementations like PyPy. I don't think it makes much sense to consider CPython specific changes to bytecode.

    https://peps.python.org/pep-0399/

    Reacted by Sergey Miryanov

  11. sergey-miryanov commented on Feb 14, 2025

    @sergey-miryanov
    sergey-miryanov
    on Feb 14, 2025
    Contributor
    More actions

    Thanks for the link - it is useful for my understanding.
    I’m stick to atomic ops, but I’m sure you have much more and deeper understanding.

  12. colesbury
    added a commit that references this issue on Feb 14, 2025

    gh-130095: Mark `_py_abc` tests as not thread-safe (gh-130131)

    39cd972

  13. cfbolz commented on Feb 14, 2025

    @cfbolz
    cfbolz
    on Feb 14, 2025
    Contributor
    More actions

    @colesbury if you are specifically concerned about pypy, we don't use _py_abc.py anymore either.

  14. colesbury commented on Feb 14, 2025

    @colesbury
    colesbury
    on Feb 14, 2025
    ContributorAuthor
    More actions

    Thanks for the clarification. I also noticed a failure when I substituted abc for _py_abc with pypy311, so there might be a thread-safety issue there as well.

    If neither pypy nor CPython use _py_abc outside of tests, I'm not going to spend much more time on this.

  15. sergey-miryanov commented on Oct 20, 2025

    @sergey-miryanov
    sergey-miryanov
    on Oct 20, 2025
    Contributor
    More actions

    Do we want to fix this or may be deprecate it (if it is not used anymore by pypy and CPython)?

    Usage over 15_000 pypi packages 
     python ../../misc/cpython/search_pypi_top.py -q . "_py_abc"
.\cx_freeze-8.3.0.tar.gz: cx_freeze-8.3.0/cx_Freeze/hooks/__init__.py: module.exclude_names.add("_py_abc")
.\cx_freeze-8.3.0.tar.gz: cx_freeze-8.3.0/cx_Freeze/hooks/__init__.py: finder.include_module("_py_abc")
.\cx_freeze-8.4.1.tar.gz: cx_freeze-8.4.1/cx_Freeze/hooks/__init__.py: module.exclude_names.add("_py_abc")
.\cx_freeze-8.4.1.tar.gz: cx_freeze-8.4.1/cx_Freeze/hooks/__init__.py: finder.include_module("_py_abc")
.\contrast_agent-10.11.0.tar.gz: contrast_agent-10.11.0/src/contrast_vendor/stdlib_list/lists/3.9.txt: _py_abc
.\contrast_agent-10.17.0.tar.gz: contrast_agent-10.17.0/src/contrast_vendor/stdlib_list/lists/3.9.txt: _py_abc
.\astroid-3.3.10.tar.gz: astroid-3.3.10/astroid/_backport_stdlib_names.py: "_py_abc",
.\astroid-3.3.10.tar.gz: astroid-3.3.10/tests/test_scoped_nodes.py: self.assertIn(metaclass.qname(), ("abc.ABCMeta", "_py_abc.ABCMeta"))
.\basedmypy-2.10.0.tar.gz: basedmypy-2.10.0/basedmypy.egg-info/SOURCES.txt: mypy/typeshed/stdlib/_py_abc.pyi
.\astroid-4.0.1.tar.gz: astroid-4.0.1/tests/test_scoped_nodes.py: self.assertIn(metaclass.qname(), ("abc.ABCMeta", "_py_abc.ABCMeta"))
.\basedmypy-2.10.0.tar.gz: basedmypy-2.10.0/mypy/typeshed/stdlib/VERSIONS: _py_abc: 3.7-
.\basedpyright-1.29.5.tar.gz: basedpyright-1.29.5/basedpyright/dist/typeshed-fallback/stdlib/VERSIONS: _py_abc: 3.7-
.\basedpyright-1.29.5.tar.gz: basedpyright-1.29.5/docstubs/stdlib/VERSIONS: _py_abc: 3.7-
.\basedpyright-1.29.5.tar.gz: basedpyright-1.29.5/packages/pyright-internal/typeshed-fallback/stdlib/VERSIONS: _py_abc: 3.7-
.\basedpyright-1.29.5.tar.gz: basedpyright-1.29.5/packages/pyright/dist/typeshed-fallback/stdlib/VERSIONS: _py_abc: 3.7-
.\basedpyright-1.31.7.tar.gz: basedpyright-1.31.7/basedpyright/dist/typeshed-fallback/stdlib/VERSIONS: _py_abc: 3.7-
.\basedpyright-1.31.7.tar.gz: basedpyright-1.31.7/docstubs/stdlib/VERSIONS: _py_abc: 3.7-
.\basedpyright-1.31.7.tar.gz: basedpyright-1.31.7/packages/pyright-internal/typeshed-fallback/stdlib/VERSIONS: _py_abc: 3.7-
.\basedpyright-1.31.7.tar.gz: basedpyright-1.31.7/packages/pyright/dist/typeshed-fallback/stdlib/VERSIONS: _py_abc: 3.7-
.\jedi-0.19.2.tar.gz: jedi-0.19.2/jedi.egg-info/SOURCES.txt: jedi/third_party/typeshed/stdlib/3.7/_py_abc.pyi
.\pipreqs-0.5.0.tar.gz: pipreqs-0.5.0/pipreqs/stdlib: _py_abc
.\pipreqs_fivetran-1.0.1.tar.gz: pipreqs_fivetran-1.0.1/pipreqs/stdlib: _py_abc
.\mypy-1.16.1.tar.gz: mypy-1.16.1/mypy/typeshed/stdlib/VERSIONS: _py_abc: 3.7-
.\mypy-1.16.1.tar.gz: mypy-1.16.1/mypy.egg-info/SOURCES.txt: mypy/typeshed/stdlib/_py_abc.pyi
.\mypy-1.18.2.tar.gz: mypy-1.18.2/mypy/typeshed/stdlib/VERSIONS: _py_abc: 3.7-
.\mypy-1.18.2.tar.gz: mypy-1.18.2/mypy.egg-info/SOURCES.txt: mypy/typeshed/stdlib/_py_abc.pyi
.\pylint-3.3.7.tar.gz: pylint-3.3.7/pylint/checkers/base/basic_error_checker.py: ABC_METACLASSES = {"_py_abc.ABCMeta", "abc.ABCMeta"}  # Python 3.7+,
.\pylint-3.3.7.tar.gz: pylint-3.3.7/pylint/checkers/utils.py: ABC_MODULES = {"abc", "_py_abc"}
.\pylint-3.3.7.tar.gz: pylint-3.3.7/pylint/checkers/variables.py: METACLASS_NAME_TRANSFORMS = {"_py_abc": "abc"}
.\pylint-4.0.1.tar.gz: pylint-4.0.1/pylint/checkers/base/basic_error_checker.py: ABC_METACLASSES = {"_py_abc.ABCMeta", "abc.ABCMeta"}  # Python 3.7+,
.\pylint-4.0.1.tar.gz: pylint-4.0.1/pylint/checkers/utils.py: ABC_MODULES = {"abc", "_py_abc"}
.\pylint-4.0.1.tar.gz: pylint-4.0.1/pylint/checkers/variables.py: METACLASS_NAME_TRANSFORMS = {"_py_abc": "abc"}
.\pytest-archon-0.0.6.tar.gz: pytest-archon-0.0.6/src/pytest_archon/assets/core-module-lists/3.7.txt: _py_abc
.\pytest-archon-0.0.6.tar.gz: pytest-archon-0.0.6/src/pytest_archon/assets/core-module-lists/3.8.txt: _py_abc
.\pytest-archon-0.0.6.tar.gz: pytest-archon-0.0.6/src/pytest_archon/assets/core-module-lists/3.9.txt: _py_abc
.\pytest_archon-0.0.7.tar.gz: pytest_archon-0.0.7/src/pytest_archon/assets/core-module-lists/3.7.txt: _py_abc
.\pytest_archon-0.0.7.tar.gz: pytest_archon-0.0.7/src/pytest_archon/assets/core-module-lists/3.8.txt: _py_abc
.\pytest_archon-0.0.7.tar.gz: pytest_archon-0.0.7/src/pytest_archon/assets/core-module-lists/3.9.txt: _py_abc
.\pyre-check-0.9.23.tar.gz: pyre_check-0.9.23/pyre_check.egg-info/SOURCES.txt: typeshed/stdlib/_py_abc.pyi
.\pyre-check-0.9.25.tar.gz: pyre_check-0.9.25/pyre_check.egg-info/SOURCES.txt: typeshed/stdlib/_py_abc.pyi
.\pyre-check-nightly-0.0.101751800363.tar.gz: pyre_check_nightly-0.0.101751800363/pyre_check_nightly.egg-info/SOURCES.txt: typeshed/stdlib/_py_abc.pyi
.\pyright-1.1.402.tar.gz: pyright-1.1.402/src/pyright/dist/dist/typeshed-fallback/stdlib/VERSIONS: _py_abc: 3.7-
.\pyright-1.1.402.tar.gz: pyright-1.1.402/src/pyright.egg-info/SOURCES.txt: src/pyright/dist/dist/typeshed-fallback/stdlib/_py_abc.pyi
.\pyright-1.1.406.tar.gz: pyright-1.1.406/src/pyright/dist/dist/typeshed-fallback/stdlib/VERSIONS: _py_abc: 3.7-
.\pyright-1.1.406.tar.gz: pyright-1.1.406/src/pyright.egg-info/SOURCES.txt: src/pyright/dist/dist/typeshed-fallback/stdlib/_py_abc.pyi
.\rtest-0.0.36.tar.gz: rtest-0.0.36/ruff/crates/ruff_python_stdlib/src/sys/known_stdlib.rs: | "_py_abc"
.\ruff-0.12.2.tar.gz: ruff-0.12.2/crates/ruff_python_stdlib/src/sys/known_stdlib.rs: | "_py_abc"
.\ruff-0.12.2.tar.gz: ruff-0.12.2/crates/ty_vendored/vendor/typeshed/stdlib/VERSIONS: _py_abc: 3.7-
.\ruff-0.14.1.tar.gz: ruff-0.14.1/crates/ty_vendored/vendor/typeshed/stdlib/VERSIONS: _py_abc: 3.7-
.\ruff-0.14.1.tar.gz: ruff-0.14.1/crates/ruff_python_stdlib/src/sys/known_stdlib.rs: | "_py_abc"
.\typeshed_client-2.7.0.tar.gz: typeshed_client-2.7.0/typeshed_client/typeshed/VERSIONS: _py_abc: 3.7-
.\typeshed_client-2.7.0.tar.gz: typeshed_client-2.7.0/typeshed_client.egg-info/SOURCES.txt: typeshed_client/typeshed/_py_abc.pyi
.\typeshed_client-2.8.2.tar.gz: typeshed_client-2.8.2/typeshed_client/typeshed/VERSIONS: _py_abc: 3.7-
.\typeshed_client-2.8.2.tar.gz: typeshed_client-2.8.2/typeshed_client.egg-info/SOURCES.txt: typeshed_client/typeshed/_py_abc.pyi
.\stdlib_list-0.11.1.tar.gz: stdlib_list-0.11.1/stdlib_list/lists/3.10.txt: _py_abc
.\stdlib_list-0.11.1.tar.gz: stdlib_list-0.11.1/stdlib_list/lists/3.11.txt: _py_abc
.\stdlib_list-0.11.1.tar.gz: stdlib_list-0.11.1/stdlib_list/lists/3.12.txt: _py_abc
.\stdlib_list-0.11.1.tar.gz: stdlib_list-0.11.1/stdlib_list/lists/3.13.txt: _py_abc
.\stdlib_list-0.11.1.tar.gz: stdlib_list-0.11.1/stdlib_list/lists/3.7.txt: _py_abc
.\stdlib_list-0.11.1.tar.gz: stdlib_list-0.11.1/stdlib_list/lists/3.8.txt: _py_abc
.\stdlib_list-0.11.1.tar.gz: stdlib_list-0.11.1/stdlib_list/lists/3.9.txt: _py_abc
.\stdlibs-2025.5.10.tar.gz: stdlibs-2025.5.10/README.md: ['_abc', '_contextvars', '_py_abc', '_queue', '_uuid', '_xxtestfuzz', 'contextvars', 'dataclasses']
.\stdlibs-2025.5.10.tar.gz: stdlibs-2025.5.10/stdlibs/py.py: "_py_abc",
.\stdlibs-2025.5.10.tar.gz: stdlibs-2025.5.10/stdlibs/py3.py: "_py_abc",
.\stdlibs-2025.5.10.tar.gz: stdlibs-2025.5.10/stdlibs/py310.py: "_py_abc",
.\stdlibs-2025.5.10.tar.gz: stdlibs-2025.5.10/stdlibs/py311.py: "_py_abc",
.\stdlibs-2025.5.10.tar.gz: stdlibs-2025.5.10/stdlibs/py312.py: "_py_abc",
.\stdlibs-2025.5.10.tar.gz: stdlibs-2025.5.10/stdlibs/py313.py: "_py_abc",
.\stdlibs-2025.5.10.tar.gz: stdlibs-2025.5.10/stdlibs/py314.py: "_py_abc",
.\stdlibs-2025.5.10.tar.gz: stdlibs-2025.5.10/stdlibs/py37.py: "_py_abc",
.\stdlibs-2025.5.10.tar.gz: stdlibs-2025.5.10/stdlibs/py38.py: "_py_abc",
.\stdlibs-2025.5.10.tar.gz: stdlibs-2025.5.10/stdlibs/py39.py: "_py_abc",
.\stdlibs-2025.5.10.tar.gz: stdlibs-2025.5.10/PKG-INFO: ['_abc', '_contextvars', '_py_abc', '_queue', '_uuid', '_xxtestfuzz', 'contextvars', 'dataclasses']
.\temporalio-1.13.0.tar.gz: temporalio-1.13.0/temporalio/worker/workflow_sandbox/_restrictions.py: "_pickle,_posixshmem,_posixsubprocess,_py_abc,_pydecimal,_pyio,_queue,_random,_scproxy,"
.\temporalio-1.18.1.tar.gz: temporalio-1.18.1/temporalio/worker/workflow_sandbox/_restrictions.py: "_pickle,_posixshmem,_posixsubprocess,_py_abc,_pydecimal,_pyio,_queue,_random,_scproxy,"
.\pytype-2024.10.11.tar.gz: pytype-2024.10.11/pytype/typeshed/stdlib/VERSIONS: _py_abc: 3.7-
.\pytype-2024.10.11.tar.gz: pytype-2024.10.11/pytype.egg-info/SOURCES.txt: pytype/typeshed/stdlib/_py_abc.pyi
.\shed-2025.6.1.tar.gz: shed-2025.6.1/src/shed/_stdlib_module_names/py38.py: "_py_abc",
.\shed-2025.6.1.tar.gz: shed-2025.6.1/src/shed/_stdlib_module_names/py39.py: "_py_abc",
.\ty-0.0.1a13.tar.gz: ty-0.0.1a13/ruff/crates/ty_vendored/vendor/typeshed/stdlib/VERSIONS: _py_abc: 3.7-
.\ty-0.0.1a13.tar.gz: ty-0.0.1a13/ruff/crates/ruff_python_stdlib/src/sys/known_stdlib.rs: | "_py_abc"
.\wrapt-1.17.2.tar.gz: wrapt-1.17.2/tests/test_inheritance_py37.py: import _py_abc
.\wrapt-1.17.2.tar.gz: wrapt-1.17.2/tests/test_inheritance_py37.py: def test_py_abc_inheritance(self):
.\wrapt-1.17.2.tar.gz: wrapt-1.17.2/tests/test_inheritance_py37.py: class A1(metaclass=_py_abc.ABCMeta):
.\wrapt-1.17.3.tar.gz: wrapt-1.17.3/tests/test_inheritance_py37.py: import _py_abc
.\wrapt-1.17.3.tar.gz: wrapt-1.17.3/tests/test_inheritance_py37.py: def test_py_abc_inheritance(self):
.\wrapt-1.17.3.tar.gz: wrapt-1.17.3/tests/test_inheritance_py37.py: class A1(metaclass=_py_abc.ABCMeta):
.\ty-0.0.1a23.tar.gz: ty-0.0.1a23/ruff/crates/ruff_python_stdlib/src/sys/known_stdlib.rs: | "_py_abc"
.\ty-0.0.1a23.tar.gz: ty-0.0.1a23/ruff/crates/ty_vendored/vendor/typeshed/stdlib/VERSIONS: _py_abc: 3.7-
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions
      You are visitor 01402503 — GeezerVison does not store the pages you look up.
      Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.