Description

serhiy-storchaka

opened on Sep 17, 2021

Issue body actions

BPO	45229
Nosy	@terryjreedy, @ambv, @ambv, @zware, @serhiy-storchaka, @pablogsal, @miss-islington
PRs	bpo-45229: Remove test_main in many tests #28405 bpo-45229: Fix setUpModule in test_ssl #28454 [3.10] bpo-45229: Remove test_main in many tests (GH-28405) #28455 [3.9] bpo-45229: Remove test_main in many tests (GH-28405) #28456 bpo-45229: Make pickle tests discoverable #28467 bpo-45229: Use doctest.DocTestSuite instead of run_doctest #28468 [3.10] bpo-45229: Make pickle tests discoverable (GH-28467) #28478 [3.9] bpo-45229: Make pickle tests discoverable (GH-28467) #28479 bpo-45229: Make datetime tests discoverable #28615 bpo-45229: Make tkinter tests discoverable #28637 [3.10] bpo-45229: Make datetime tests discoverable (GH-28615) #28639 [3.9] bpo-45229: Make datetime tests discoverable (GH-28615). #28645 bpo-45229: Revert "Make datetime tests discoverable" #28650 [3.9] Revert "[3.9] bpo-45229: Make datetime tests discoverable (GH-28615) (GH-28645)" #28660 [3.9] Revert "[3.9] Revert "[3.9] bpo-45229: Make datetime tests discoverable (GH-28615) (GH-28645)"" #28666 Revert "bpo-45229: Revert "Make datetime tests discoverable"" #28667 [3.10] bpo-45229: Make datetime tests discoverable (GH-28615) #28668 [3.10] bpo-45229: Make tkinter tests discoverable (GH-28637) #28927 [3.9] bpo-45229: Make tkinter tests discoverable (GH-28637) #28929 bpo-45229: Make doctest tests discoverable #28986 bpo-45229: Make test_http_cookiejar discoverable #29004 [3.10] bpo-45229: Make test_http_cookiejar discoverable (GH-29004) #29006 [3.9] bpo-45229: Make test_http_cookiejar discoverable (GH-29004) #29007 [3.10] bpo-45229: Make doctest tests discoverable (GH-28986) #29095 [3.9] bpo-45229: Make doctest tests discoverable (GH-28986) #29096
Dependencies	bpo-40058: Running test_datetime twice fails with: module 'datetime' has no attribute '_divide_and_round' bpo-40173: test.support.import_fresh_module fails to correctly block submodules when fresh is specified bpo-45185: test.test_ssl.TestEnumerations is not run bpo-45187: Some tests in test_socket are not run bpo-45500: Rewrite test_dbm bpo-45502: Fix test_shelve and make it discoverable

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = None
created_at = <Date 2021-09-17.07:31:12.126>
labels = ['type-feature', 'tests', '3.9', '3.10', '3.11']
title = 'Always use unittest for collecting tests in regrtests'
updated_at = <Date 2021-10-20.16:52:45.847>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2021-10-20.16:52:45.847>
actor = 'lukasz.langa'
assignee = 'serhiy.storchaka'
closed = False
closed_date = None
closer = None
components = ['Tests']
creation = <Date 2021-09-17.07:31:12.126>
creator = 'serhiy.storchaka'
dependencies = ['40058', '40173', '45185', '45187', '45500', '45502']
files = []
hgrepos = []
issue_num = 45229
keywords = ['patch']
message_count = 33.0
messages = ['402006', '402009', '402053', '402076', '402154', '402156', '402168', '402195', '402214', '402228', '402245', '402254', '402255', '402903', '402945', '402952', '402953', '402956', '402959', '402974', '402982', '402996', '402999', '403000', '403846', '403853', '403855', '404137', '404142', '404143', '404466', '404495', '404496']
nosy_count = 7.0
nosy_names = ['terry.reedy', 'lukasz.langa', 'lukasz.langa', 'zach.ware', 'serhiy.storchaka', 'pablogsal', 'miss-islington', 'jerrylikerice1s']
pr_nums = ['28405', '28454', '28455', '28456', '28467', '28468', '28478', '28479', '28615', '28637', '28639', '28645', '28650', '28660', '28666', '28667', '28668', '28927', '28929', '28986', '29004', '29006', '29007', '29095', '29096']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue45229'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

Linked PRs

• gh-89392: Make test_decimal discoverable #106209
• [3.11] gh-89392: Make test_decimal discoverable (GH-106209) #106229
• [3.12] gh-89392: Make test_decimal discoverable (GH-106209) #106230
• bpo-45229: Make ElementTree tests discoverable #108859
• gh-89392: Remove test_main() in test_netrc #108860
• gh-89392: Make test_pep646_syntax discoverable #108861
• [3.11] gh-89392: Remove test_main() in test_netrc (GH-108860) #108868
• [3.11] gh-89392: Make test_pep646_syntax discoverable (GH-108861) #108869
• [3.12] gh-89392: Make test_pep646_syntax discoverable (GH-108861) #108871
• [3.12] gh-89392: Remove test_main() in test_netrc (GH-108860) #108872
• [3.12] bpo-45229: Make ElementTree tests discoverable (GH-108859) #108873
• [3.11] bpo-45229: Make ElementTree tests discoverable (GH-108859) #108874
• gh-89392: Fix running test_pep646_syntax as script #108875
• gh-89392: Remove support of test_main() in libregrtest #108876
• [3.11] gh-89392: Fix running test_pep646_syntax as script (GH-108875) #108877
• [3.12] gh-89392: Remove support of test_main() in libregrtest (GH-108876) #108897
• [3.11] gh-89392: Remove support of test_main() in libregrtest (GH-108876) #108898
• gh-89392: Use normal unittest runner in test_type_cache #108911
• [3.12] gh-89392: Use normal unittest runner in test_type_cache (GH-108911) #108913
• [3.11] gh-89392: Use normal unittest runner in test_type_cache (GH-108911) #108914
• gh-89392: Use unittest test runner for doctests in test_getopt #108916
• [3.12] gh-89392: Use unittest test runner for doctests in test_getopt (GH-108916) #108919
• [3.11] gh-89392: Use unittest test runner for doctests in test_getopt (GH-108916) #108920

Activity

1. serhiy-storchaka commented on Sep 17, 2021

   

   serhiy-storchaka

   on Sep 17, 2021

   MemberAuthor

   More actions

   Currently regrtest supports two ways of collecting and running tests in module.

   1. If the module contains the "test_main" function, regrtest just calls it. This function usually calls run_unittest() with a list of test classes, composed manually, it can also run doctests with run_doctest(), generate list of test classes dynamically, and run some code before and after running tests.

   2. Otherwise regrtest uses unittest for for collecting tests.

   The disadvantage of the former way is that new test classes should be added manually to the list. If you forget to do this, new tests will not be run. See for example bpo-45185 and bpo-45187. Not runned tests can hide bugs.

   So it would be better to eliminate human factor and detect tests automatically.
2. serhiy-storchaka

   added

   3.9 (EOL)end of lifeend of life

   3.10 (EOL)end of lifeend of life

   3.11only security fixesonly security fixes

   on Sep 17, 2021
3. serhiy-storchaka

   self-assigned this

   on Sep 17, 2021
4. serhiy-storchaka

   added

   3.10 (EOL)end of lifeend of life

   testsTests in the Lib/test dirTests in the Lib/test dir

   type-featureA feature request or enhancementA feature request or enhancement

   3.11only security fixesonly security fixes

   on Sep 17, 2021
5. serhiy-storchaka

   self-assigned this

   on Sep 17, 2021
6. serhiy-storchaka

   added

   testsTests in the Lib/test dirTests in the Lib/test dir

   type-featureA feature request or enhancementA feature request or enhancement

   on Sep 17, 2021
7. serhiy-storchaka commented on Sep 17, 2021

   

   serhiy-storchaka

   on Sep 17, 2021

   MemberAuthor

   More actions

   PR 28405 contains simple changes for 60 test files. It does not cover files which use complex code for generating list of test classes and running doctests (because there are subtle differences between running doctests with run_doctest() and via DocTestSuite). They are left for the following PRs.
8. zware commented on Sep 17, 2021

   

   zware

   on Sep 17, 2021

   Member

   More actions

   See also bpo-16748
9. 67 remaining items
10. serhiy-storchaka

    added

    3.13only security fixesonly security fixes

    on Sep 5, 2023
11. serhiy-storchaka

    added 2 commits that reference this issue on Sep 5, 2023

    pythongh-89392: Use normal unittest runner in test_type_cache

    32680fa

    gh-89392: Use normal unittest runner in test_type_cache (GH-108911)

    eaabaac
12. miss-islington

    added 2 commits that reference this issue on Sep 5, 2023

    pythongh-89392: Use normal unittest runner in test_type_cache (python…

    5a720ae

    pythongh-89392: Use normal unittest runner in test_type_cache (python…

    f8cc844
13. serhiy-storchaka

    added 3 commits that reference this issue on Sep 5, 2023

    pythongh-89392: Use unittest test runner for doctests in test_getopt

    1eea982

    [3.11] gh-89392: Use normal unittest runner in test_type_cache (GH-10…

    e2404f5

    gh-89392: Use unittest test runner for doctests in test_getopt (GH-10…

    f980cc1
14. miss-islington

    added 2 commits that reference this issue on Sep 5, 2023

    pythongh-89392: Use unittest test runner for doctests in test_getopt (p…

    df6846b

    pythongh-89392: Use unittest test runner for doctests in test_getopt (p…

    af4e067
15. serhiy-storchaka

    added a commit that references this issue on Sep 5, 2023

    [3.11] gh-89392: Use unittest test runner for doctests in test_getopt (…

    4e5fd6d
16. Yhg1s

    added 2 commits that reference this issue on Sep 8, 2023

    [3.12] gh-89392: Use normal unittest runner in test_type_cache (GH-10…

    da02508

    [3.12] gh-89392: Use unittest test runner for doctests in test_getopt (…

    038b0a9
17. serhiy-storchaka commented on Sep 9, 2023

    

    serhiy-storchaka

    on Sep 9, 2023

    MemberAuthor

    More actions

    Finally it has been finished.

    Reacted by Alex Waygood
18. serhiy-storchaka

    closed this as completedon Sep 9, 2023

Issue actions