GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — refcycles in exceptions raised from asyncio.TaskGroup · Issue #124958 · python/cpython

refcycles in exceptions raised from asyncio.TaskGroup #124958

New issue
Copy link
New issue
Copy link
Closed
Closed
refcycles in exceptions raised from asyncio.TaskGroup#124958
Copy link
Labels
stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytopic-asynciotype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Description

@graingert
graingert
opened on Oct 4, 2024
Issue body actions

Bug report

Bug description:

asyncio.TaskGroup attempts to avoid refcycles in raised exceptions by deleting self._errors but when I reviewed the code it doesn't actually achieve this:

see

cpython/Lib/asyncio/taskgroups.py

Lines 152 to 156 in 5e9e506

try:
me = BaseExceptionGroup('unhandled errors in a TaskGroup', self._errors)
raise me from None
finally:
self._errors = None

There's a refcycle in me is me.__traceback__.tb_next.tb_frame.f_locals["me"]

I wrote a few tests to route out all the refcycles in tracebacks

import asyncio
import gc
import unittest

class TestTaskGroup(unittest.IsolatedAsyncioTestCase):

    async def test_exception_refcycles_direct(self):
        """Test that TaskGroup doesn't keep a reference to the raised ExceptionGroup"""
        tg = asyncio.TaskGroup()
        exc = None

        class _Done(Exception):
            pass

        try:
            async with tg:
                raise _Done
        except ExceptionGroup as e:
            exc = e

        self.assertIsNotNone(exc)
        self.assertListEqual(gc.get_referrers(exc), [])


    async def test_exception_refcycles_errors(self):
        """Test that TaskGroup deletes self._errors, and __aexit__ args"""
        tg = asyncio.TaskGroup()
        exc = None

        class _Done(Exception):
            pass

        try:
            async with tg:
                raise _Done
        except* _Done as excs:
            exc = excs.exceptions[0]

        self.assertIsInstance(exc, _Done)
        self.assertListEqual(gc.get_referrers(exc), [])


    async def test_exception_refcycles_parent_task(self):
        """Test that TaskGroup deletes self._parent_task"""
        tg = asyncio.TaskGroup()
        exc = None

        class _Done(Exception):
            pass

        async def coro_fn():
            async with tg:
                raise _Done

        try:
            async with asyncio.TaskGroup() as tg2:
                tg2.create_task(coro_fn())
        except* _Done as excs:
            exc = excs.exceptions[0].exceptions[0]

        self.assertIsInstance(exc, _Done)
        self.assertListEqual(gc.get_referrers(exc), [])

    async def test_exception_refcycles_propagate_cancellation_error(self):
        """Test that TaskGroup deletes propagate_cancellation_error"""
        tg = asyncio.TaskGroup()
        exc = None

        try:
            async with asyncio.timeout(-1):
                async with tg:
                    await asyncio.sleep(0)
        except TimeoutError as e:
            exc = e.__cause__

        self.assertIsInstance(exc, asyncio.CancelledError)
        self.assertListEqual(gc.get_referrers(exc), [])

    async def test_exception_refcycles_base_error(self):
        """Test that TaskGroup deletes self._base_error"""
        class MyKeyboardInterrupt(KeyboardInterrupt):
            pass

        tg = asyncio.TaskGroup()
        exc = None

        try:
            async with tg:
                raise MyKeyboardInterrupt
        except MyKeyboardInterrupt as e:
            exc = e

        self.assertIsNotNone(exc)
        self.assertListEqual(gc.get_referrers(exc), [])

in writing all these tests I noticed refcycles in PyFuture:

cpython/Lib/asyncio/futures.py

Lines 197 to 198 in 58f7763

exc = self._make_cancelled_error()
raise exc

cpython/Lib/asyncio/futures.py

Lines 215 to 216 in 58f7763

exc = self._make_cancelled_error()
raise exc

class BaseFutureTests:
    def test_future_cancelled_result_refcycles(self):
        f = self._new_future(loop=self.loop)
        f.cancel()
        exc = None
        try:
            f.result()
        except asyncio.CancelledError as e:
            exc = e
        self.assertIsNotNone(exc)
        self.assertListEqual(gc.get_referrers(exc), [])

    def test_future_cancelled_exception_refcycles(self):
        f = self._new_future(loop=self.loop)
        f.cancel()
        exc = None
        try:
            f.exception()
        except asyncio.CancelledError as e:
            exc = e
        self.assertIsNotNone(exc)
        self.assertListEqual(gc.get_referrers(exc), [])

CPython versions tested on:

3.12, 3.13

Operating systems tested on:

Linux

Linked PRs

  • gh-124958: fix asyncio.TaskGroup and _PyFuture refcycles #124959
  • [3.13] gh-124958: fix asyncio.TaskGroup and _PyFuture refcycles (GH-124959) #125463
  • [3.12] gh-124958: fix asyncio.TaskGroup and _PyFuture refcycles (#124959) #125466
  • gh-124958: Revert "gh-125472: Revert "gh-124958: fix asyncio.TaskGroup and _PyFuture refcycles ... #125486

Activity

  1. graingert
    added
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    on Oct 4, 2024
  2. graingert
    added a commit that references this issue on Oct 4, 2024

    pythongh-124958: fix asyncio.TaskGroup and _PyFuture refcycles

    1a4f2b5
  3. 1st1
    added a commit that references this issue on Oct 14, 2024

    gh-124958: fix asyncio.TaskGroup and _PyFuture refcycles (#124959)

    d5dbbf4
  4. miss-islington
    added a commit that references this issue on Oct 14, 2024

    pythongh-124958: fix asyncio.TaskGroup and _PyFuture refcycles (pytho…

    e754fd8
  5. 1st1
    added a commit that references this issue on Oct 14, 2024

    [3.13] gh-124958: fix asyncio.TaskGroup and _PyFuture refcycles (GH-1…

    4e08c98
  6. picnixz
    added
    topic-asyncio
    on Oct 14, 2024
  7. github-project-automation
    added this to asyncioon Oct 14, 2024
  8. picnixz
    added
    stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory
    on Oct 14, 2024
  9. github-project-automation
    moved this to Todo in asyncioon Oct 14, 2024
  10. Eclips4
    added a commit that references this issue on Oct 14, 2024

    Revert "pythongh-124958: fix asyncio.TaskGroup and _PyFuture refcycles (

    b8cc4ca
  11. kumaraditya303
    added a commit that references this issue on Oct 14, 2024

    gh-125472: Revert "gh-124958: fix asyncio.TaskGroup and _PyFuture ref…

    e99650b
  12. miss-islington
    added a commit that references this issue on Oct 14, 2024

    pythongh-125472: Revert "pythongh-124958: fix asyncio.TaskGroup and _…

    874bfad
  13. graingert
    added a commit that references this issue on Oct 14, 2024

    Revert "pythongh-125472: Revert "pythongh-124958: fix asyncio.TaskGro…

    050cef9
  14. 1st1
    added 2 commits that reference this issue on Oct 14, 2024

    gh-124958: Revert "gh-125472: Revert "gh-124958: fix asyncio.TaskGrou…

    0b28ea4

    [3.12] gh-124958: fix asyncio.TaskGroup and _PyFuture refcycles (#124959

    32d4579
  15. graingert commented on Oct 18, 2024

    @graingert
    graingert
    on Oct 18, 2024
    ContributorAuthor
    More actions

    This is merged and backported now

  16. graingert
    closed this as completedon Oct 18, 2024
  17. github-project-automation
    moved this from Todo to Done in asyncioon Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytopic-asynciotype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

    Projects

    • asyncio
      Status
      Done
      Show more project fields

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    You are visitor 01402503 — GeezerVison does not store the pages you look up.
    Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.