Description

EliseevEgor

opened on Jul 10, 2023

Issue body actions

Bug report

In Python 3.12, a new exit code (5, instead of 0) was added if all tests were skipped in unittest. But in some cases it still returns 0. See code examples.

Code example 1:

import unittest


class TestSimple(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        raise unittest.SkipTest("Skip whole Case")

    def test_true(self):
        self.assertTrue(True)

    def test_false(self):
        self.assertTrue(False, msg="Is not True")

Output:

Ran 0 tests in 0.000s

NO TESTS RAN (skipped=1)

Ok, we get exit code 5 here.

Code example 2:

import unittest


class TestCase(unittest.TestCase):
    @unittest.skip("something")
    def test(self):
        self.assertTrue(False)

Output:

Ran 1 test in 0.000s

OK (skipped=1)

Here we can see output 0. Ran one test and skipped. But I think it is incorrect, because all tests were skipped.

Your environment

• CPython versions tested on: 3.12
• Operating system and architecture: I think it does not matter.

Linked PRs

• gh-106584: Fix exit code for unittest in Python 3.12 #106588
• [3.12] gh-106584: Fix exit code for unittest in Python 3.12 (GH-106588) #109725
• gh-113267: Revert "gh-106584: Fix exit code for unittest in Python 3.12 (#106588)" #114470

Activity

1. EliseevEgor

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Jul 10, 2023
2. EliseevEgor

   added a commit that references this issue on Jul 10, 2023

   pythonGH-106584: Fix exit code for unittest in Python 3.12

   6e1be71
3. AlexWaygood

   added

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   3.12only security fixesonly security fixes

   3.13only security fixesonly security fixes

   on Jul 10, 2023
4. terryjreedy

   added this to Unittest & doctest issueson Jul 10, 2023
5. github-project-automation

   moved this to Todo in Unittest & doctest issueson Jul 10, 2023
6. EliseevEgor

   added 4 commits that reference this issue on Jul 11, 2023

   pythonGH-106584: Apply comment suggestion

   43248a9

   pythonGH-106584: Remove the version info in the NEWS

   901cf01

   pythonGH-106584: Remove empty line in the NEWS

   1ab5cfa

   pythonGH-106584: Check buffers before seek

   6313c98
7. ambv

   added a commit that references this issue on Sep 22, 2023

   gh-106584: Fix exit code for unittest in Python 3.12 (#106588)

   8fc0713
8. miss-islington

   added a commit that references this issue on Sep 22, 2023

   pythongh-106584: Fix exit code for unittest in Python 3.12 (pythonGH-…

   95fcc0d
9. csm10495

   added a commit that references this issue on Sep 28, 2023

   pythongh-106584: Fix exit code for unittest in Python 3.12 (python#10…

   53fda70
10. Yhg1s

    added a commit that references this issue on Oct 2, 2023

    [3.12] gh-106584: Fix exit code for unittest in Python 3.12 (GH-106588)…

    551aa6a
11. 1 remaining item
12. hugovk

    closed this as completedon Nov 10, 2023
13. github-project-automation

    moved this from Todo to Done in Unittest & doctest issueson Nov 10, 2023
14. serhiy-storchaka

    added a commit that references this issue on Jan 23, 2024

    Revert "pythongh-106584: Fix exit code for unittest in Python 3.12 (p…

    15d1968
15. serhiy-storchaka commented on Jan 23, 2024

    

    serhiy-storchaka

    on Jan 23, 2024

    Member

    More actions

    I think that it was a wrong change, and not only because it makes startTest() and stopTest() calls unbalanced (#113267), but because it does not solve the original issue. The third example should be equivalent to the second example:

    import unittest
    
    class TestCase(unittest.TestCase):
        def test(self):
            self.skipTest("something")

    But it gets error code 0 while the second example gets error code 5.

    It also gives a wrong answer to the problem. #113661 fixes it in the opposite way.

    Reacted by Erlend E. Aasland
16. serhiy-storchaka

    reopened this on Jan 23, 2024
17. github-project-automation

    moved this from Done to In Progress in Unittest & doctest issueson Jan 23, 2024
18. serhiy-storchaka

    added a commit that references this issue on Feb 4, 2024

    gh-113267: Revert "gh-106584: Fix exit code for unittest in Python 3.…

    ecabff9
19. miss-islington

    added a commit that references this issue on Feb 4, 2024

    pythongh-113267: Revert "pythongh-106584: Fix exit code for unittest …

    b79c37f
20. serhiy-storchaka

    added a commit that references this issue on Feb 4, 2024

    [3.12] gh-113267: Revert "gh-106584: Fix exit code for unittest in Py…

    0a73763
21. aisk

    added a commit that references this issue on Feb 11, 2024

    pythongh-113267: Revert "pythongh-106584: Fix exit code for unittest …

    9d44784
22. fsc-eriker

    added a commit that references this issue on Feb 14, 2024

    pythongh-113267: Revert "pythongh-106584: Fix exit code for unittest …

    39378cc
23. carljm commented on Feb 29, 2024

    

    carljm

    on Feb 29, 2024

    Member

    More actions

    Looks like this change has been reverted, and #113661 PRs are merged also, and that issue is closed. Seems there is nothing left to do here?
24. carljm

    closed this as completedon Feb 29, 2024
25. github-project-automation

    moved this from In Progress to Done in Unittest & doctest issueson Feb 29, 2024
26. Glyphack

    added a commit that references this issue on Sep 2, 2024

    pythongh-106584: Fix exit code for unittest in Python 3.12 (python#10…

    4eaba4a

Issue actions