Description

sobolevn

opened on Mar 29, 2024

Issue body actions

Feature or enhancement

Right now there's no way to skip the whole test / class doctest. You can only skip individual statements with # doctest: +SKIP

This causes problems in several use-cases:

• You have a doctest that should be skipped based on the platform
• We have doctests that should be skipped based on a resource availability
• Libraries might want to skip some doctests based on the Python version / any other lib version

Right now we have this hack:

cpython/Lib/test/test_doctest/test_doctest.py

Lines 727 to 730 in 2e9be80

	if int.__doc__: # simple check for --without-doc-strings, skip if lacking
	def non_Python_modules(): r"""
	Finding Doctests in Modules Not Written in Python

cpython/Lib/test/test_doctest/test_doctest.py

Lines 1999 to 2003 in 2e9be80

	if not hasattr(sys, 'gettrace') or not sys.gettrace():
	def test_pdb_set_trace():
	"""Using pdb.set_trace from a doctest.
	You can use pdb.set_trace from a doctest. To do so, you must

cpython/Lib/test/test_doctest/test_doctest.py

Lines 2938 to 2943 in 2e9be80

	if supports_unicode:
	def test_unicode(): """
	Check doctest with a non-ascii filename:
	>>> doc = '''
	... >>> raise Exception('clé')

However, this is just test functions. For library functions it would be much harder to do.
You have to jiggle __doc__ attribute around.

Since #117297 we now have correct skipped count for doctest + unittest integration.

So, I propose adding this decorator to doctest.py

def skip_if(condition):
    def decorator(func):
        if condition and HAVE_DOCSTRINGS:
            func.__doc__ = ">>> pass  # doctest: +SKIP"
        return func
    return decorator

It will allow us skipping some tests conditionally with the proper reported results.

Refs #116758
CC @serhiy-storchaka and @furkanonder

Linked PRs

• gh-117364: Add doctest.SkipTest to skip all or some doctest examples #122935

Reacted by Aleksei Stepanov

Activity

1. sobolevn

   added

   type-featureA feature request or enhancementA feature request or enhancement

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   on Mar 29, 2024
2. sobolevn

   self-assigned this

   on Mar 29, 2024
3. serhiy-storchaka commented on Mar 29, 2024

   

   serhiy-storchaka

   on Mar 29, 2024

   Member

   More actions

   There is a problem with applying this to non-test functions. We do not want to replace docstrings of library functions if doctests are skipped.

   Also, what is the solution for conditional skipping of module-level doctests?

   Reacted by Furkan Onder
4. serhiy-storchaka

   added

   3.13only security fixesonly security fixes

   on Mar 29, 2024
5. sobolevn commented on Mar 29, 2024

   

   sobolevn

   on Mar 29, 2024

   MemberAuthor

   More actions

   There is a problem with applying this to non-test functions. We do not want to replace docstrings of library functions if doctests are skipped.

   Hm, indeed. This might be a bit more complex: we can add # doctest: +SKIP to all >>> lines dynamically.

   Does not seem too hard, but it will be harder for sure.

   Also, what is the solution for conditional skipping of module-level doctests?

   This can be adjusted to include doctest.skip_if(condition, module=True).
6. sobolevn commented on Mar 31, 2024

   

   sobolevn

   on Mar 31, 2024

   MemberAuthor

   More actions

   Or instead of patching the string, we can go with .__doctest_skip__ attribute for a function.

   Reacted by Serhiy Storchaka
7. serhiy-storchaka commented on Mar 31, 2024

   

   serhiy-storchaka

   on Mar 31, 2024

   Member

   More actions

   This was my idea too. I thought about making it more general and also setting other options for the whole test, but I don't see a use case for other options for now.

   Alternative ideas:

   • Introduce a special exception doctest.SkipTest, so that raising it will skip all following examples.
   • Introduce a syntax for conditional skipping, e.g. +SKIP if varname. It is enough to support only simple variable names, not complex expressions. It would be nice to also add syntax to apply options to all subsequent examples instead of just one example.

   They are more complex changes of doctest machinery.
8. sobolevn

   added a commit that references this issue on Aug 12, 2024

   pythongh-117364: Add `doctest.SkipTest` to skip all or some doctest e…

   ea3e74c

Issue actions