Description

donbarbos

opened on Mar 13, 2025

Issue body actions

While working on #130160 and #93096 it was decided that it would be worth paying attention to the problem of lack of tests for command line interface.

List of all cli modules is in the documentation: https://docs.python.org/dev/library/cmdline.html

Many modules are missing command line interface tests, so I suggest using this issue to submit PRs for now only for well-documented interfaces, for example:

• tokenize (gh-131178: Add tests for tokenize command-line interface #131274)
• platform (gh-131524: Update platform CLI to use argparse #131542)
• pickle (gh-131178: Add tests for pickle command-line interface #131275)
• ast (gh-131178: Add tests for ast command-line interface #133329)
• pickletools (gh-131178: Add tests for pickletools command-line interface #131287)
• http.server (gh-131178: Add tests for http.server command-line interface #132540)
• profile (gh-131178: Add CLI tests for profile module #134103)
• cProfile (gh-131178: Add CLI tests for cProfile #134269)
• site ( gh-131178: Add tests for site command-line interface  #133582)

TODO: Make sure all documented have tests for CLI (others are not so important as it has not been decided what to do with them yet)

You can use PR #127759 for dis module or #118132 for random module as a reference.
And I also suggest using subprocess.Popen to allocate a separate resource since some modules can block a lot of time.

Important

For those who want to work on the issue, please:

• Select one module for which tests will be improved. It's easier to review and backport.
• Open a pull request with the following title: gh-131178: Add tests for `MODULE_NAME` command-line interface

Linked PRs

• gh-131178: Add tests for tokenize command-line interface #131274
• gh-131178: Add tests for pickle command-line interface #131275
• gh-131178: Add tests for pickletools command-line interface #131287
• gh-131178: Update help message for timeit CLI #131326
• gh-131178: Add tests for http.server command-line interface #132540
• gh-131178: Add tests for ast command-line interface #133329
• gh-131178: Add tests for site command-line interface  #133582
• gh-131178: Add CLI tests for profile module #134103
• gh-131178: fix SSL tests for http.server command-line interface #134224
• gh-131178: Add CLI tests for cProfile #134269
• gh-131178: add E2E mockless tests for http.server command-line interface #134279
• gh-131178: remove runtime tests for http.server CLI #134287
• gh-131178: Use support.captured_stdout and support.captured_stderr #137748
• gh-131178: Add tests for site command-line interface #137807
• gh-131178: Fix test_unknown_flag for platform CLI #137816
• [3.14] gh-131178: Add tests for site command-line interface (GH-133582) #137832
• [3.13] gh-131178: Add tests for site command-line interface (GH-133582) #137833
• gh-131178: Add tests for sysconfig command-line interface #139934
• gh-131178: add tests for filecmp command-line interface #139947
• gh-131178: add unittest for turtledemo #139983
• gh-131178: add tests for filecmp command-line interface #140199
• gh-131178: add tests for filecmp command-line interface #140200
• gh-131178: Add CLI tests for the profile module #143835
• gh-131178: Add tests for cProfile command-line interface #144918
• gh-131178: Add tests for pyclbr command-line interface #147995
• gh-131178: Add tests for mimetypes command-line interface #149681
• gh-131178: Fix mimetypes CLI docs to mention that errors go to stdout #149683
• gh-131178: Add tests for quopri command-line interface #149825
• gh-131178: Add more tests for json command-line interface #149868
• gh-131178: Add tests for zipapp command-line interface #150203
• [3.15] gh-131178: Fix mimetypes CLI docs, mention that errors go to stdout (GH-149683) #150655
• [3.14] gh-131178: Fix mimetypes CLI docs, mention that errors go to stdout (GH-149683) #150656
• gh-131178: Add tests for this command-line interface #151855
• gh-131178: Add CLI tests for pydoc module #151971
• gh-131178: Add tests for pstats command-line interface #157133
• gh-131178: Add tests for ensurepip command-line interface #157154
• gh-131178: Add tests for profile module command-line interface #158542
• gh-131178: Add tests for code command-line interface #158588
• gh-131178: Add tests for calendar command-line interface #159045
• gh-131178: Add tests for \encodings.rot_13\ command-line interface #159299
• gh-131178: Add tests for ftplib command-line interface #161080
• gh-131178: Fix test_cli_retrieve_file when stdout is StringIO in test_ftplib #161135
• [3.14] gh-131178: Add tests for ftplib command-line interface (GH-161080) #161153
• [3.15] gh-131178: Add tests for ftplib command-line interface (GH-161080) #161154
• gh-131178: Add tests for symtable command-line interface #161374
• gh-131178: Add tests for filecmp command-line interface #161606

Reacted by Sergey Miryanov and Weilin Du

Activity

1. picnixz

   added

   type-featureA feature request or enhancementA feature request or enhancement

   testsTests in the Lib/test dirTests in the Lib/test dir

   on Mar 13, 2025
2. hugovk commented on Mar 13, 2025

   

   hugovk

   on Mar 13, 2025

   Member

   More actions

   See also the tests added in #118132, based on https://pythontest.com/testing-argparse-apps/

   Reacted by Semyon Moroz and QuinzyW
3. hugovk

   added

   easy

   on Mar 13, 2025
4. sobolevn

   assigned

   donbarbos

   on Mar 13, 2025
5. sobolevn commented on Mar 13, 2025

   

   sobolevn

   on Mar 13, 2025

   Member

   More actions

   @donbarbos I assigned this issue to you, since you mentioned that you are working on it :)

   Reacted by Semyon Moroz
6. hugovk

   added 2 commits that reference this issue on Mar 27, 2025

   gh-131178: Add tests for `tokenize` command-line interface (#131274)

   9b83670

   gh-131178: Update help message for `timeit` CLI (#131326)

   55150a7
7. diegorusso

   added a commit that references this issue on Apr 1, 2025

   pythongh-131178: Add tests for `tokenize` command-line interface (pyt…

   7a2dc09
8. picnixz

   added a commit that references this issue on Apr 6, 2025

   gh-131178: Add tests for `pickle` command-line interface (#131275)

   f247e1d
9. sobolevn

   added a commit that references this issue on May 4, 2025

   gh-131178: Add tests for `ast` command-line interface (#133329)

   6ce60f1
10. diegorusso

    added a commit that references this issue on May 4, 2025

    Merge remote-tracking branch 'origin/main' into call-len

    c03babc
11. ggqlq

    added 3 commits that reference this issue on May 7, 2025

    pythongh-131178: Add tests for command-line interface

    2cfed33

    pythongh-131178: Add tests for command-line interface

    fc36305

    pythongh-131178: Add tests for site command-line interface

    6497282
12. 78 remaining items
13. Bhumika-1432006

    added 2 commits that reference this issue on Oct 13, 2026

    pythongh-131178: Add NEWS entry for encodings.rot_13 CLI tests

    ff77d95

    pythongh-131178: Fix NEWS entry to use inline code instead of mod role

    7e84658
14. kevinprimeraquecreo-stack

    added this to Project_VKon Oct 18, 2026
15. github-project-automation

    moved this to In progress in Project_VKon Oct 18, 2026
16. kevinprimeraquecreo-stack

    added this to Project_VKon Oct 18, 2026
17. Bhumika-1432006

    added 3 commits that reference this issue on Oct 21, 2026

    pythongh-131178: Add tests for `encodings.rot_13` command-line interface

    7658eac

    pythongh-131178: Add NEWS entry for encodings.rot_13 CLI tests

    aa15f12

    pythongh-131178: Fix NEWS entry to use inline code instead of mod role

    ae2ca28
18. orsenthil

    added 4 commits that reference this issue on Nov 1, 2026

    gh-131178: Add tests for `ftplib` command-line interface (#161080)

    c29f95a

    gh-131178: Fix test_cli_retrieve_file when stdout is StringIO in test…

    3d81c05

    [3.14] gh-131178: Add tests for `ftplib` command-line interface (GH-1…

    86b2b3e

    [3.15] gh-131178: Add tests for `ftplib` command-line interface (GH-1…

    0d43164
19. Martin-Aziz

    added 3 commits that reference this issue on Nov 8, 2026

    pythongh-131178: Add tests for filecmp command-line interface

    3a5f66d

    pythongh-131178: Simplify CLI test fixtures

    e274d0e

    pythongh-131178: Handle colored tracebacks in CLI tests

    fe164cd

Issue actions