Description

erlend-aasland

opened on Jul 3, 2023

Issue body actions

The Argument Clinic test suite consists of three parts:

• Lib/test/test_clinic.py: mostly parser tests
• Lib/test/clinic.test: generated code, checked by test_clinic.py (ClinicExternalTest)
• Modules/_testclinic.c: mostly converter tests (thanks, @colorfulappl!)

Test coverage is currently just above 80%, which is too low. I'd say we try to reach 90-something percent coverage.

Reminder: PRs for increased test coverage should be backported to all bugfix branches.

Linked PRs

• gh-106368: Increase Argument Clinic test coverage #106369
• [3.12] gh-106368: Increase Argument Clinic test coverage (GH-106369) #106370
• gh-106368: Clean up Argument Clinic tests #106373
• [3.11] gh-106368: Increase Argument Clinic test coverage (#106369) #106374
• [3.12] gh-106368: Clean up Argument Clinic tests (#106373) #106379
• [3.11] gh-106368: Clean up Argument Clinic tests (#106373) #106381
• gh-106368: Harden Argument Clinic parser tests #106384
• [3.12] gh-106368: Harden Argument Clinic parser tests (GH-106384) #106387
• [3.11] gh-106368: Harden Argument Clinic parser tests (GH-106384) #106388
• gh-106368: Increase Argument Clinic test coverage #106389
• [3.12] gh-106368: Increase Argument Clinic test coverage (GH-106389) #106390
• [3.11] gh-106368: Increase Argument Clinic test coverage (GH-106389) #106391
• gh-106368: Add tests for permutation helpers in Argument Clinic #106407
• [3.12] gh-106368: Add tests for permutation helpers in Argument Clinic (GH-106407) #106409
• [3.11] gh-106368: Add tests for permutation helpers in Argument Clinic (GH-106407) #106410
• gh-106368: Add tests for formatting helpers in Argument Clinic #106415
• [3.12] gh-106368: Add tests for formatting helpers in Argument Clinic (GH-106415) #106438
• [3.11] gh-106368: Add tests for formatting helpers in Argument Clinic (GH-106415) #106439
• gh-106368: Increase Argument Clinic test coverage #106728
• [3.12] gh-106368: Increase Argument Clinic test coverage (GH-106728) #106730
• [3.11] gh-106368: Increase Argument Clinic test coverage (#106728) #106731
• gh-106368: Increase Argument Clinic BlockParser test coverage #106759
• [3.12] gh-106368: Increase Argument Clinic BlockParser test coverage (GH-106759) #106769
• [3.11] gh-106368: Increase Argument Clinic BlockParser test coverage (GH-106759) #106770
• gh-106368: Add functional tests for Tools/clinic/cpp.py #106813
• gh-106368: Increase Argument Clinic test coverage for cpp.Monitor #106833
• [3.12] gh-106368: Increase Argument Clinic test coverage for cpp.Monitor (GH-106833) #106838
• [3.11] gh-106368: Increase Argument Clinic test coverage for cpp.Monitor (GH-106833) #106839
• gh-106368: Increase Argument Clinic test coverage for IndentStack #106933
• [3.12] gh-106368: Increase Argument Clinic test coverage for IndentStack (GH-106933) #106943
• [3.11] gh-106368: Increase Argument Clinic test coverage for IndentStack (GH-106933) #106944
• gh-106368: Increase coverage for Argument Clinic output directive #106979
• [3.12] gh-106368: Increase coverage for Argument Clinic output directive (GH-106979) #106994
• [3.11] gh-106368: Increase coverage for Argument Clinic output directive (GH-106979) #107002
• gh-106368: Increase Argument Clinic CLI test coverage #107156
• [3.12] gh-106368: Increase Argument Clinic CLI test coverage (GH-107156) #107189
• [3.11] gh-106368: Increase Argument Clinic CLI test coverage (GH-107156) #107190
• gh-106368: Increase Argument Clinic CLI test coverage #107277
• [3.12] gh-106368: Increase Argument Clinic CLI test coverage (GH-107277) #107282
• [3.11] gh-106368: Increase Argument Clinic CLI test coverage (#107277) #107326
• gh-106368: Argument clinic tests: improve failure message when tests in ClinicExternalTests fail #107364
• [3.12] gh-106368: Argument clinic tests: improve failure message when tests in ClinicExternalTests fail (GH-107364) #107365
• [3.11] gh-106368: Argument clinic tests: improve failure message when tests in ClinicExternalTests fail (GH-107364) #107366
• gh-106368: Argument clinic: Fix minor bug in state_modulename_name #107387
• gh-106368: Add test for Argument Clinic misbehaving custom converter_init() #107496
• [3.12] gh-106368: Add test for Argument Clinic misbehaving custom converter_init() (GH-107496) #107499
• [3.11] gh-106368: Add test for Argument Clinic misbehaving custom converter_init() (GH-107496) #107500
• gh-106368: Increase test coverage for Argument Clinic #107514
• gh-106368: Increase Argument Clinic test coverage #107582
• gh-106368: Argument clinic tests: improve error message when expect_success() fails #107606
• gh-106368: Increase Argument Clinic test coverage #107611
• gh-106368: Argument clinic: improve coverage for self.valid_line() calls #107641
• gh-106368: Improve coverage reports for argument clinic #107693
• gh-106368: Argument clinic: add tests for more failure paths #107731
• gh-106368: Argument Clinic: Add tests for cloned functions with custom C base names #107977
• gh-106368: Argument Clinic: Test that keyword params are disallowed in groups #107985

Reacted by Alex Waygood

Activity

1. erlend-aasland

   added

   type-featureA feature request or enhancementA feature request or enhancement

   topic-argument-clinic

   on Jul 3, 2023
2. erlend-aasland commented on Jul 3, 2023

   

   erlend-aasland

   on Jul 3, 2023

   ContributorAuthor

   More actions

   Also: a side effect of good coverage, is that it can help identify unreachable paths in the code.
3. erlend-aasland commented on Jul 3, 2023

   

   erlend-aasland

   on Jul 3, 2023

   ContributorAuthor

   More actions

   cc. @sobolevn, who I know enjoys increasing test coverage 📈
4. erlend-aasland

   added 2 commits that reference this issue on Jul 3, 2023

   pythongh-106368: Increase Argument Clinic test coverage

   afe91ca

   gh-106368: Increase Argument Clinic test coverage (#106369)

   7f4c812
5. miss-islington

   added a commit that references this issue on Jul 3, 2023

   pythongh-106368: Increase Argument Clinic test coverage (pythonGH-106369

   e0f4ef0
6. erlend-aasland

   added 8 commits that reference this issue on Jul 3, 2023

   pythongh-106368: Clean up Argument Clinic tests

   2197527

   [3.11] pythongh-106368: Increase Argument Clinic test coverage (pytho…

   2899599

   [3.12] gh-106368: Increase Argument Clinic test coverage (GH-106369) (#…

   ddff473

   gh-106368: Clean up Argument Clinic tests (#106373)

   3ee8dac

   [3.12] pythongh-106368: Clean up Argument Clinic tests (python#106373)

   e3a1cbd

   [3.11] gh-106368: Increase Argument Clinic test coverage (#106369) (#…

   52d5049

   [3.11] pythongh-106368: Clean up Argument Clinic tests (python#106373)

   098f500

   [3.12] gh-106368: Clean up Argument Clinic tests (#106373) (#106379)

   38fe0e7
7. 99 remaining items
8. erlend-aasland

   reopened this on Aug 3, 2023
9. erlend-aasland

   added 2 commits that reference this issue on Aug 3, 2023

   pythongh-106368: Increase Argument Clinic test coverage

   eea4503

   gh-106368: Increase Argument Clinic test coverage (#107611)

   0bd784b
10. erlend-aasland

    closed this as completedon Aug 4, 2023
11. erlend-aasland commented on Aug 4, 2023

    

    erlend-aasland

    on Aug 4, 2023

    ContributorAuthor

    More actions

    We're now at 93% coverage.

    Reacted by Alex Waygood
12. AlexWaygood

    added 2 commits that reference this issue on Aug 4, 2023

    gh-106368: Argument clinic: improve coverage for `self.valid_line()` …

    2c25bd8

    gh-106368: Improve coverage reports for argument clinic (#107693)

    ee3bf45
13. kostyafarber

    added a commit that references this issue on Aug 7, 2023

    pythongh-106368: Improve coverage reports for argument clinic (python…

    0c14eef
14. AlexWaygood

    added a commit that references this issue on Aug 8, 2023

    gh-106368: Argument clinic: add tests for more failure paths (#107731)

    7c5153d
15. erlend-aasland

    added 2 commits that reference this issue on Aug 15, 2023

    pythongh-106368: Argument Clinic: Add test for cloned function with c…

    b7851c3

    gh-106368: Argument Clinic: Add tests for cloned functions with custo…

    bb456a0
16. erlend-aasland

    reopened this on Aug 15, 2023
17. erlend-aasland

    added 2 commits that reference this issue on Aug 15, 2023

    pythongh-106368: Argument Clinic: Test that keyword params are disall…

    db82762

    gh-106368: Argument Clinic: Test that keyword params are disallowed i…

    57a20b0
18. erlend-aasland

    closed this as completedon Oct 12, 2023

Issue actions