Description

vstinner

opened on Aug 23, 2023

Issue body actions

The slowest tests of the Python test suite are:

• test_concurrent_futures
• test_multiprocessing_spawn
• test_peg_generator
• test_tools.test_freeze tests

They each tend take multiple minutes to run. The bulk of our others take <10 seconds. They occupy a lot of wall time as a long tail in a normal make test or regrtest run on a typical parallel run multi-core systems.

This bug originally proposed to skip them unless the "cpu" resource is enabled (it's disabled by default). That was deemed appropriate for test_peg_generator and test_tools.test_freeze which are either not platform specific or rarely needed in CI. (PR #108386) - This PR reduced the total test duration between 3 and 5 minutes.

Linked PRs

• gh-108388: test_concurrent_futures requires cpu #108389
• gh-108388: regrtest runs slowest tests first #108391
• gh-108388: regrtest splits test_asyncio package #108393
• gh-108388: Split test_multiprocessing_spawn #108396
• [3.12] gh-108388: regrtest splits test_asyncio package (GH-108393) #108397
• gh-108388: Convert test_concurrent_futures to package #108401
• [3.12] gh-108388: Split test_multiprocessing_spawn (GH-108396) #108442
• [3.12] gh-108388: Convert test_concurrent_futures to package (#108401) #108443
• [3.11] gh-108388: Split test_multiprocessing_spawn (GH-108396) #109688
• [3.11] gh-108388: Convert test_concurrent_futures to package (#108401) #109704

Activity

1. vstinner

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Aug 23, 2023
2. vstinner

   added 11 commits that reference this issue on Aug 23, 2023

   pythongh-108388: test_concurrent_futures requires cpu

   83becc5

   pythongh-108388: regrtest runs slowest tests first

   852b6a2

   pythongh-108388: regrtest splits test_asyncio package

   b9b3505

   pythongh-108388: regrtest splits test_asyncio package

   34ee8c7

   pythongh-108388: regrtest splits test_asyncio package

   5fe1f85

   pythongh-108388: regrtest splits test_asyncio package

   44ac7a7

   pythongh-108388: Split test_multiprocessing_spawn

   fef3c4d

   pythongh-108388: Split test_multiprocessing_spawn

   68290fd

   pythongh-108388: Split test_multiprocessing_spawn

   b8d93be

   pythongh-108388: Split test_multiprocessing_spawn

   bfd362d

   gh-108388: regrtest splits test_asyncio package (#108393)

   174e9da
3. miss-islington

   added a commit that references this issue on Aug 24, 2023

   pythongh-108388: regrtest splits test_asyncio package (pythonGH-108393)

   0b23a7d
4. vstinner

   added 2 commits that reference this issue on Aug 24, 2023

   gh-108388: Split test_multiprocessing_spawn (#108396)

   aa9a359

   pythongh-108388: Convert test_concurrent_futures to package

   fd89c1a
5. 12 remaining items
6. gpshead

   added

   3.12only security fixesonly security fixes

   buildThe build process and cross-buildThe build process and cross-build

   and removed

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Aug 24, 2023
7. sobolevn commented on Aug 25, 2023

   

   sobolevn

   on Aug 25, 2023

   Member

   More actions

   I experience new failures of test_concurrent_futures on Windows on my unrelated PR: https://github.com/python/cpython/actions/runs/5972558288/job/16203298171?pr=108456

   Failing tests:

        test.test_concurrent_futures.test_deadlock
        test.test_concurrent_futures.test_shutdown
   
8. vstinner commented on Aug 25, 2023

   

   vstinner

   on Aug 25, 2023

   MemberAuthor

   More actions

   I experience new failures of test_concurrent_futures on Windows on my unrelated PR

   Sadly, the issue is known for at least one month: see issue #107219.

   Logs:

     File "D:\a\cpython\cpython\Lib\test\test_concurrent_futures\test_deadlock.py", line 236 in test_crash_big_data
   

   It's this test which hangs sometimes on Windows.

   Reacted by sobolevn
9. Yhg1s

   added 2 commits that reference this issue on Aug 25, 2023

   [3.12] gh-108388: Split test_multiprocessing_spawn (GH-108396) (#108442)

   2262190

   [3.12] gh-108388: Convert test_concurrent_futures to package (#108401) (

   bba9aa6
10. vstinner

    added 2 commits that reference this issue on Sep 2, 2023

    pythongh-108388: regrtest splits test_asyncio package (python#108393)

    30120ff

    [3.11] gh-108822: Backport libregrtest changes from the main branch (#…

    79f7a4c
11. vstinner commented on Sep 13, 2023

    

    vstinner

    on Sep 13, 2023

    MemberAuthor

    More actions

    This feature has been implemented in the main branch, and backported to the 3.12 branch. I don't think that it's worth it to backport it to the 3.11 branch.

    I splitted these test packages:

    • test_concurrent_futures
    • test_multiprocessing_fork
    • test_multiprocessing_forkserver
    • test.test_multiprocessing_spawn

    Example:

    $ ./python -m test test_concurrent_futures --list-tests
    test_concurrent_futures.test_as_completed
    test_concurrent_futures.test_deadlock
    test_concurrent_futures.test_future
    test_concurrent_futures.test_init
    test_concurrent_futures.test_process_pool
    test_concurrent_futures.test_shutdown
    test_concurrent_futures.test_thread_pool
    test_concurrent_futures.test_wait
    

    I close my issue.

    Reacted by Erlend E. Aasland
12. vstinner

    closed this as completedon Sep 13, 2023
13. miss-islington

    added a commit that references this issue on Sep 21, 2023

    pythongh-108388: Split test_multiprocessing_spawn (pythonGH-108396)

    b2cea8f
14. vstinner

    added 3 commits that reference this issue on Sep 21, 2023

    pythongh-108388: Split test_multiprocessing_spawn (pythonGH-108396)

    9ae7650

    [3.11] gh-108388: Split test_multiprocessing_spawn (GH-108396) (#109688)

    b3af888

    [3.11] gh-108388: Convert test_concurrent_futures to package (#108401) (

    66a973a

Issue actions