Description

assignee = None
closed_at = None
created_at = <Date 2021-05-19.05:04:33.179>
labels = ['type-bug', '3.8', '3.9', '3.10', '3.7', 'library', 'expert-asyncio']
title = 'asyncio.as_completed() raises TypeError when the first supplied parameter is a generator that yields awaitables'
updated_at = <Date 2021-05-19.05:53:09.349>
user = 'https://github.com/alexdelorenzo'

activity = <Date 2021-05-19.05:53:09.349>
actor = 'alexdelorenzo'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'asyncio']
creation = <Date 2021-05-19.05:04:33.179>
creator = 'alexdelorenzo'
dependencies = []
files = ['50052']
hgrepos = []
issue_num = 44176
keywords = []
message_count = 1.0
messages = ['393923']
nosy_count = 3.0
nosy_names = ['asvetlov', 'yselivanov', 'alexdelorenzo']
pr_nums = ['26228']
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44176'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

Activity

1. alexdelorenzo commented on May 19, 2021

   

   alexdelorenzomannequin

   on May 19, 2021

   MannequinAuthor

   More actions

   According to the documentation, asyncio.as_completed() takes a positional argument, aws, as an iterable of awaitables[1]:

       asyncio.as_completed(aws, *, loop=None, timeout=None)
         Run awaitable objects in the aws iterable concurrently.

   As seen in the attached as_completed_gen.py file, built-in containers like lists, and iterators over them, are accepted by as_completed() as the first parameter without raising an error.

   However, as_completed() raises TypeError if it is called with an iterable of awaitables that is also a generator. There are examples of this behavior in as_completed_gen.py, but here is a short example using a generator expression in the main() coroutine function:

       from asyncio import run, as_completed
       
       async def example(): pass
       
       async def main():
         coros = (example() for _ in range(10))
       
         for coro in as_completed(coros):  # raises TypeError
           await coro
       
       run(main())

   Running that example will raise a TypeError with this message:

   TypeError: expect an iterable of futures, not generator
   

   If we look at the first line in the body of as_completed(), we can see why this error is thrown for generators that yield awaitables:

       def as_completed(fs, *, loop=None, timeout=None):
         if futures.isfuture(fs) or coroutines.iscoroutine(fs):
           raise TypeError(f"expect an iterable of futures, not {type(fs).__name__}")
         ...

   Because generators are coroutines, and the first condition in as_completed() is True, and TypeError gets raised:

       from asyncio import coroutines

   # generators and generator expressions are coroutines
   assert coroutines.iscoroutine(example() for _ in range(10))
   

   Perhaps as_completed() can use inspect.isawaitable() instead, like so:

       from inspect import isawaitable
   
       def as_completed(fs, *, loop=None, timeout=None):
         if futures.isfuture(fs) or isawaitable(fs):
           ...

   I made a pull request with that change here[2].

   [1] https://docs.python.org/3/library/asyncio-task.html#asyncio.as_completed

   [2] #26228
2. alexdelorenzo

   added

   3.7 (EOL)end of lifeend of life

   3.8 (EOL)end of lifeend of life

   3.9 (EOL)end of lifeend of life

   3.10 (EOL)end of lifeend of life

   topic-asyncio

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   on May 19, 2021
3. ezio-melotti

   transferred this issue fromon Apr 10, 2022
4. ezio-melotti

   added this to asyncioon Jun 20, 2022
5. ezio-melotti

   moved this to Todo in asyncioon Jul 17, 2022
6. kumaraditya303

   removed

   3.10 (EOL)end of lifeend of life

   3.9 (EOL)end of lifeend of life

   3.8 (EOL)end of lifeend of life

   3.7 (EOL)end of lifeend of life

   on Oct 18, 2022
7. kumaraditya303 commented on Apr 19, 2023

   

   kumaraditya303

   on Apr 19, 2023

   Contributor

   More actions

   3.12 allows generators so this is completed.
8. kumaraditya303

   closed this as not plannedon Apr 19, 2023
9. github-project-automation

   moved this from Todo to Done in asyncioon Apr 19, 2023
10. kumaraditya303 commented on Apr 19, 2023

    

    kumaraditya303

    on Apr 19, 2023

    Contributor

    More actions

    #103626 clarifies this in doc.
11. kumaraditya303

    added a commit that references this issue on Apr 19, 2023

    GH-88342: clarify that `asyncio.as_completed` accepts generators yiel…

    da2273f
12. carljm

    added a commit that references this issue on Apr 20, 2023

    Merge branch 'main' into superopt

    5c0a21c

Issue actions