Description

assignee = None
closed_at = <Date 2020-07-06.20:18:05.903>
created_at = <Date 2020-02-05.17:28:25.689>
labels = ['type-bug', '3.8', '3.9', 'release-blocker', 'expert-asyncio']
title = "Asynchronous comprehensions don't work in asyncio REPL"
updated_at = <Date 2021-09-07.20:14:54.080>
user = 'https://github.com/jack1142'

activity = <Date 2021-09-07.20:14:54.080>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-07-06.20:18:05.903>
closer = 'pablogsal'
components = ['asyncio']
creation = <Date 2020-02-05.17:28:25.689>
creator = 'jack1142'
dependencies = []
files = []
hgrepos = []
issue_num = 39562
keywords = ['patch']
message_count = 28.0
messages = ['361443', '361445', '364600', '364601', '365311', '365312', '365313', '365347', '365881', '366838', '366841', '366843', '366849', '366851', '367015', '367018', '367734', '367847', '367855', '367890', '367892', '367897', '368334', '368835', '373131', '373159', '373168', '400234']
nosy_count = 10.0
nosy_names = ['asvetlov', 'lukasz.langa', 'serhiy.storchaka', 'mbussonn', 'Anthony Sottile', 'hroncok', 'pablogsal', 'miss-islington', 'BTaskaya', 'jack1142']
pr_nums = ['18968', '19010', '19070', '19230', '19835', '19838']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39562'
versions = ['Python 3.8', 'Python 3.9']

Activity

1. jack1142 commented on Feb 5, 2020

   

   jack1142mannequin

   on Feb 5, 2020

   MannequinAuthor

   More actions

   asyncio REPL doesn't allow using asynchronous comprehensions outside of async func. Same behavior can also be observed when using ast.PyCF_ALLOW_TOP_LEVEL_AWAIT flag in compile()

   Example with `async for`:
   >>> async def async_gen():
   ...     for x in range(5):
   ...         yield await asyncio.sleep(1, x)
   ... 
   >>> [x async for x in async_gen()]
     File "<console>", line 0
   SyntaxError: asynchronous comprehension outside of an asynchronous function
   
   
   Example with `await`:
   >>> [await asyncio.sleep(1, x) for x in range(5)]
     File "<console>", line 0
   SyntaxError: asynchronous comprehension outside of an asynchronous function
2. jack1142

   added

   3.8 (EOL)end of lifeend of life

   3.9 (EOL)end of lifeend of life

   topic-asyncio

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Feb 5, 2020
3. jack1142 commented on Feb 5, 2020

   

   jack1142mannequin

   on Feb 5, 2020

   MannequinAuthor

   More actions

   I also noticed that putting await before the async comprehension will make the code inside the comprehension run (though after it runs, it will fail on awaiting list):

   >>> await [await asyncio.sleep(1, print(x)) for x in range(5)] 
   0
   1
   2
   3
   4
   Traceback (most recent call last):
     File "C:\Python38\lib\concurrent\futures\_base.py", line 439, in result
       return self.__get_result()
     File "C:\Python38\lib\concurrent\futures\_base.py", line 388, in __get_result
       raise self._exception
     File "<console>", line 1, in <module>
   TypeError: object list can't be used in 'await' expression
4. pablogsal commented on Mar 19, 2020

   

   pablogsal

   on Mar 19, 2020

   Member

   More actions

   New changeset 9052f7a by Batuhan Taşkaya in branch 'master':
   bpo-39562: Allow executing asynchronous comprehensions in the asyncio REPL (GH-18968)
   9052f7a
5. miss-islington commented on Mar 19, 2020

   

   miss-islington

   on Mar 19, 2020

   Contributor

   More actions

   New changeset ec8a973 by Miss Islington (bot) in branch '3.8':
   bpo-39562: Allow executing asynchronous comprehensions in the asyncio REPL (GH-18968)
   ec8a973
6. isidentical

   closed this as completedon Mar 19, 2020
7. isidentical

   closed this as completedon Mar 19, 2020
8. vstinner commented on Mar 30, 2020

   

   vstinner

   on Mar 30, 2020

   Member

   More actions

   This change caused python-gmpy2 tests to fail:
   https://bugzilla.redhat.com/show_bug.cgi?id=1817710

   Reproducer:
   ---

   from __future__ import print_function, division
   import doctest
   
   filename = "doctest.txt"
   with open(filename, "w") as fp:
       print("""
   Test

   ====

       >>> all(x == 1 for x in [1, 1, 1])
       True
       """.strip(), file=fp)

   result = doctest.testfile(filename, globs=globals())
   print("result:", result)

   ---
9. 27 remaining items
10. vstinner

    closed this as completedon May 14, 2020
11. vstinner

    removed

    release-blocker

    on May 14, 2020
12. vstinner

    closed this as completedon May 14, 2020
13. vstinner

    removed

    release-blocker

    on May 14, 2020
14. Carreau commented on Jul 6, 2020

    

    Carreaumannequin

    on Jul 6, 2020

    Mannequin

    More actions

    This make non-await list comprehension coroutine-code-objects as well:

    https://bugs.python.org/issue41218

        import ast
        import inspect
        cell = '[x for x in l]'
        code = compile(cell, "<>", "exec", flags=getattr(ast,'PyCF_ALLOW_TOP_LEVEL_AWAIT', 0x0))

    inspect.CO_COROUTINE & code.co_flags == inspect.CO_COROUTINE  # this is now TRUE. 
    

    This leads to weird things in Jupyter/IPython when we try to detect wether a block of code is, or os not async.
15. pablogsal

    reopened this on Jul 6, 2020
16. pablogsal

    reopened this on Jul 6, 2020
17. pablogsal

    added

    release-blocker

    on Jul 6, 2020
18. Carreau commented on Jul 6, 2020

    

    Carreaumannequin

    on Jul 6, 2020

    Mannequin

    More actions

    Pablo, I see you reopened and marked as blocker,

    As the changes here has been released maybe you want to reclose and marked https://bugs.python.org/issue41218 as blocker it should also be fixed by
    #21357
19. pablogsal commented on Jul 6, 2020

    

    pablogsal

    on Jul 6, 2020

    Member

    More actions

    As the changes here has been released maybe you want to reclose and marked https://bugs.python.org/issue41218 as blocker it should also be fixed by
    #21357

    Thanks for the heads up, Matthias!

    Yes, I think is the best thing to do. For some reason, I thought this was not still released :(
20. pablogsal

    closed this as completedon Jul 6, 2020
21. pablogsal

    closed this as completedon Jul 6, 2020
22. asottile commented on Aug 24, 2021

    

    asottilemannequin

    on Aug 24, 2021

    Mannequin

    More actions

    this maybe shouldn't have been backported to 3.8.x as the change in compiler flags appears to break pyc files in subtle ways: https://stackoverflow.com/q/68910329/812183
23. ezio-melotti

    transferred this issue fromon Apr 10, 2022

Issue actions