Description

arhadthedev

opened on May 30, 2022

Issue body actions

Feature or enhancement

Get rid of ad-hoc implementation of async tests by using all relatively new features of standard IsolatedAsyncIoTestCase. For this we need to:

• replace all occurences of class ...(test_utils.TestCase): with class ...(unittest.IsolatedAsynIoTestCase):
• remove manually created event loops in favor of the builtin one of IsolatedAsyncIoTestCase
• make loop-dependend test cases async
• replace multithreading with asyncio wherever possible
• remove unused imports
• remove the TestCase and classes it uses (588 -> 346 lines in test_asyncio/utils.py)

Pitch

• test_asyncio.utils.TestCase provides no extra functionality over IsolatedAsyncioTestCase. Instead, it takes 200+ lines of code together with two more classes used nowhere else (TestSelector and TestLoop)
• TestCase requires manual scheduling of tasks using loop.run_until_complete while IsolatedAsyncioTestCase allows await
• TestCase was introduced on 2014 by c73701d while IsolatedAsynIoTestCase was introduced 5 years later by 4dd3e3f

Progress tracking

• find and fix a bug described in gh-97983: Revert "Lay the foundation for further work in asyncio.test_streams: port server cases to IsolatedAsyncioTestCase" #98015 (comment)
• test/test_asyncgen.py
• test/test_coroutines.py
• test/test_pdb.py
• test/test_asyncio/test_base_events.py
• test/test_asyncio/test_events.py
• test/test_asyncio/test_futures.py
• test/test_asyncio/test_pep492.py
• test/test_asyncio/test_proactor_events.py
• test/test_asyncio/test_sendfile.py
• test/test_asyncio/test_sock_lowlevel.py
• test/test_asyncio/test_ssl.py
• test/test_asyncio/test_sslproto.py
• test/test_asyncio/test_streams.py
  • gh-93357: Lay the foundation for further work in asyncio.test_streams: port server cases to IsolatedAsyncioTestCase #93369
  • gh-93357: Port test cases to IsolatedAsyncioTestCase, part 2 #97896
  • gh-97983: Revert "Lay the foundation for further work in asyncio.test_streams: port server cases to IsolatedAsyncioTestCase" #98015
• test/test_asyncio/test_subprocess.py
• test/test_asyncio/test_tasks.py
• test/test_asyncio/test_unix_events.py
• test/test_asyncio/test_windows_events.py
• replace multithreading with asyncio wherever possible
• remove the TestCase
• move asyncio.set_event_loop_policy(None) (a measure agains "test altered the execution environment" warning) into IsolatedAsyncioTestCase's shutdown method

There are 83 cases that use test_asyncio.utils.TestCase

• test/test_asyncio/test_base_events.py:class BaseEventTests(test_utils.TestCase):
• test/test_asyncio/test_base_events.py:class BaseEventLoopTests(test_utils.TestCase):
• test/test_asyncio/test_base_events.py:class BaseEventLoopWithSelectorTests(test_utils.TestCase):
• test/test_asyncio/test_base_events.py:class BaseLoopSockSendfileTests(test_utils.TestCase):
• test/test_asyncio/test_base_events.py:class TestSelectorUtils(test_utils.TestCase):
• test/test_asyncio/test_events.py: test_utils.TestCase):
• test/test_asyncio/test_events.py: test_utils.TestCase):
• test/test_asyncio/test_events.py: test_utils.TestCase):
• test/test_asyncio/test_events.py: test_utils.TestCase):
• test/test_asyncio/test_events.py: test_utils.TestCase):
• test/test_asyncio/test_events.py: test_utils.TestCase):
• test/test_asyncio/test_events.py:class HandleTests(test_utils.TestCase):
• test/test_asyncio/test_futures.py:class DuckTests(test_utils.TestCase):
• test/test_asyncio/test_futures.py:class CFutureTests(BaseFutureTests, test_utils.TestCase):
• test/test_asyncio/test_futures.py:class CSubFutureTests(BaseFutureTests, test_utils.TestCase):
• test/test_asyncio/test_futures.py:class PyFutureTests(BaseFutureTests, test_utils.TestCase):
• test/test_asyncio/test_futures.py: test_utils.TestCase):
• test/test_asyncio/test_futures.py: test_utils.TestCase):
• test/test_asyncio/test_futures.py: test_utils.TestCase):
• test/test_asyncio/test_futures.py: test_utils.TestCase):
• test/test_asyncio/test_futures.py: test_utils.TestCase):
• test/test_asyncio/test_pep492.py:class BaseTest(test_utils.TestCase):
• test/test_asyncio/test_proactor_events.py:class ProactorSocketTransportTests(test_utils.TestCase):
• test/test_asyncio/test_proactor_events.py:class ProactorDatagramTransportTests(test_utils.TestCase):
• test/test_asyncio/test_proactor_events.py:class BaseProactorEventLoopTests(test_utils.TestCase):
• test/test_asyncio/test_proactor_events.py:class ProactorEventLoopUnixSockSendfileTests(test_utils.TestCase):
• test/test_asyncio/test_selector_events.py:class BaseSelectorEventLoopTests(test_utils.TestCase):
• test/test_asyncio/test_selector_events.py:class SelectorTransportTests(test_utils.TestCase):
• test/test_asyncio/test_selector_events.py:class SelectorSocketTransportTests(test_utils.TestCase):
• test/test_asyncio/test_selector_events.py:class SelectorSocketTransportBufferedProtocolTests(test_utils.TestCase):
• test/test_asyncio/test_selector_events.py:class SelectorDatagramTransportTests(test_utils.TestCase):
• test/test_asyncio/test_sendfile.py: test_utils.TestCase):
• test/test_asyncio/test_sendfile.py: test_utils.TestCase):
• test/test_asyncio/test_sendfile.py: test_utils.TestCase):
• test/test_asyncio/test_sendfile.py: test_utils.TestCase):
• test/test_asyncio/test_sendfile.py: test_utils.TestCase):
• test/test_asyncio/test_sendfile.py: test_utils.TestCase):
• test/test_asyncio/test_sock_lowlevel.py: test_utils.TestCase):
• test/test_asyncio/test_sock_lowlevel.py: test_utils.TestCase):
• test/test_asyncio/test_sock_lowlevel.py: test_utils.TestCase):
• test/test_asyncio/test_sock_lowlevel.py: test_utils.TestCase):
• test/test_asyncio/test_sock_lowlevel.py: test_utils.TestCase):
• test/test_asyncio/test_sock_lowlevel.py: test_utils.TestCase):
• test/test_asyncio/test_ssl.py:class TestSSL(test_utils.TestCase):
• test/test_asyncio/test_sslproto.py:class SslProtoHandshakeTests(test_utils.TestCase):
• test/test_asyncio/test_streams.py:class StreamTests(test_utils.TestCase):
• test/test_asyncio/test_subprocess.py:class SubprocessTransportTests(test_utils.TestCase):
• test/test_asyncio/test_subprocess.py: test_utils.TestCase):
• test/test_asyncio/test_subprocess.py: test_utils.TestCase):
• test/test_asyncio/test_subprocess.py: test_utils.TestCase):
• test/test_asyncio/test_subprocess.py: test_utils.TestCase):
• test/test_asyncio/test_subprocess.py: test_utils.TestCase):
• test/test_asyncio/test_subprocess.py: class SubprocessProactorTests(SubprocessMixin, test_utils.TestCase):
• test/test_asyncio/test_tasks.py: test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class CTask_CFuture_SubclassTests(BaseTaskTests, test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class CTaskSubclass_PyFuture_Tests(BaseTaskTests, test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class PyTask_CFutureSubclass_Tests(BaseTaskTests, test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class CTask_PyFuture_Tests(BaseTaskTests, test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class PyTask_CFuture_Tests(BaseTaskTests, test_utils.TestCase):
• test/test_asyncio/test_tasks.py: test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class PyTask_PyFuture_SubclassTests(BaseTaskTests, test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class CTask_Future_Tests(test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class PyIntrospectionTests(test_utils.TestCase, BaseTaskIntrospectionTests):
• test/test_asyncio/test_tasks.py:class CIntrospectionTests(test_utils.TestCase, BaseTaskIntrospectionTests):
• test/test_asyncio/test_tasks.py:class PyCurrentLoopTests(BaseCurrentLoopTests, test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class CCurrentLoopTests(BaseCurrentLoopTests, test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class GenericTaskTests(test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class FutureGatherTests(GatherTestsBase, test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class CoroutineGatherTests(GatherTestsBase, test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class RunCoroutineThreadsafeTests(test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class SleepTests(test_utils.TestCase):
• test/test_asyncio/test_tasks.py:class CompatibilityTests(test_utils.TestCase):
• test/test_asyncio/test_unix_events.py:class SelectorEventLoopSignalTests(test_utils.TestCase):
• test/test_asyncio/test_unix_events.py:class SelectorEventLoopUnixSocketTests(test_utils.TestCase):
• test/test_asyncio/test_unix_events.py:class SelectorEventLoopUnixSockSendfileTests(test_utils.TestCase):
• test/test_asyncio/test_unix_events.py:class UnixReadPipeTransportTests(test_utils.TestCase):
• test/test_asyncio/test_unix_events.py:class UnixWritePipeTransportTests(test_utils.TestCase):
• test/test_asyncio/test_unix_events.py:class SafeChildWatcherTests (ChildWatcherTestsMixin, test_utils.TestCase):
• test/test_asyncio/test_unix_events.py:class FastChildWatcherTests (ChildWatcherTestsMixin, test_utils.TestCase):
• test/test_asyncio/test_windows_events.py:class ProactorLoopCtrlC(test_utils.TestCase):
• test/test_asyncio/test_windows_events.py:class ProactorMultithreading(test_utils.TestCase):
• test/test_asyncio/test_windows_events.py:class ProactorTests(test_utils.TestCase):
• test/test_asyncio/test_windows_events.py:class WinPolicyTests(test_utils.TestCase):

Linked PRs

• gh-93357: Port test_pep492 to IsolatedAsyncioTestCase #98877

Activity

1. arhadthedev

   added

   type-featureA feature request or enhancementA feature request or enhancement

   on May 30, 2022
2. AlexWaygood

   added

   testsTests in the Lib/test dirTests in the Lib/test dir

   topic-asyncio

   on May 30, 2022
3. arhadthedev

   changed the title [-]Replace use of `test.test_asyncio.utils.TestCase` with `unittest.IsolatedAsynIoTestCase`[/-] [+]Replace use of `test.test_asyncio.utils.TestCase` with `unittest.IsolatedAsyncioTestCase`[/+] on May 30, 2022
4. serhiy-storchaka

   assigned

   asvetlov

   on Jun 1, 2022
5. ezio-melotti

   added this to asyncioon Jun 20, 2022
6. ezio-melotti

   moved this to Todo in asyncioon Jul 17, 2022
7. serhiy-storchaka

   added this to Unittest & doctest issueson Aug 20, 2022
8. serhiy-storchaka

   moved this to Todo in Unittest & doctest issueson Aug 20, 2022
9. gvanrossum

   added 2 commits that reference this issue on Oct 4, 2022

   gh-93357: Start porting asyncio server test cases to IsolatedAsyncioT…

   ce8fc18

   gh-93357: Port test cases to IsolatedAsyncioTestCase, part 2 (#97896)

   09aea94
10. kumaraditya303

    unassigned

    asvetlov

    on Oct 6, 2022
11. carljm

    added a commit that references this issue on Oct 6, 2022

    Merge branch 'main' into dictwatch

    4bb5d9d
12. arhadthedev

    added 2 commits that reference this issue on Oct 7, 2022

    Revert "pythongh-93357: Port test cases to IsolatedAsyncioTestCase, p…

    3713915

    Revert "pythongh-93357: Start porting asyncio server test cases to Is…

    c17e930
13. gvanrossum

    added a commit that references this issue on Oct 7, 2022

    gh-97983: Revert "Lay the foundation for further work in asyncio.test…

    f99bb20
14. mpage

    added 3 commits that reference this issue on Oct 11, 2022

    pythongh-93357: Start porting asyncio server test cases to IsolatedAs…

    81fcd2c

    pythongh-93357: Port test cases to IsolatedAsyncioTestCase, part 2 (p…

    2d20cf5

    pythongh-97983: Revert "Lay the foundation for further work in asynci…

    e75ada1
15. kumaraditya303 commented on Oct 27, 2022

    

    kumaraditya303

    on Oct 27, 2022

    Contributor

    More actions

    If we want to continue with this, then IMO this needs to be done early in the dev cycle so we have time to fixup the CI if it breaks. We certainly don't want to deal with unstable CI at the beta stage.
16. arhadthedev commented on Oct 27, 2022

    

    arhadthedev

    on Oct 27, 2022

    MemberAuthor

    More actions

    @kumaraditya303 You're right; I'll return to this issue this weekend and start with stabilization of reverted gh-98015.
17. arhadthedev commented on Oct 30, 2022

    

    arhadthedev

    on Oct 30, 2022

    MemberAuthor

    More actions

    Put on hold until gh-98219 is resolved (it probably will fix Altered environment on Windows and Env changed in buildbot in gh-97983).
18. kumaraditya303 commented on Apr 19, 2023

    

    kumaraditya303

    on Apr 19, 2023

    Contributor

    More actions

    I'm going to just close this now, it is not the best use of our time to port these tests as it doesn't increase test coverage and chance of breaking CI is very high especially as asyncio is very flaky. When I'll remove child watcher I may port some tests to ease writing new tests but not other things.

    Reacted by Oleg Iarygin
19. kumaraditya303

    closed this as not plannedon Apr 19, 2023
20. github-project-automation

    moved this from Todo to Done in Unittest & doctest issueson Apr 19, 2023
21. github-project-automation

    moved this from Todo to Done in asyncioon Apr 19, 2023

Issue actions