GeezerVison

I think it would be useful to either configure warnings as errors in the test suite (and then capture unraisable/threadexcept errors using a mechanism like pytest has) or tee any warnings produced, and fail CI if any have been produced that were not captured with catch_warnings or assertWarns etc like https://pypi.org/project/pytest-max-warnings/

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

https://discuss.python.org/t/its-too-easy-to-introduce-noise-in-cpythons-test-suite-output-resourcewarnings-and-deprecationwarnings/75234

Linked PRs

Activity

graingert
added
type-featureA feature request or enhancementA feature request or enhancement
on Jan 13, 2025
erlend-aasland
added
testsTests in the Lib/test dirTests in the Lib/test dir
on Jan 13, 2025
sobolevn commented on Jan 13, 2025
sobolevn
on Jan 13, 2025
Member
More actions
See #110012 test_socket raises warnings right now.
graingert commented on Jan 14, 2025
graingert
on Jan 14, 2025
ContributorAuthor
More actions
See #128816 test_doctest also has a warning
graingert commented on Jan 14, 2025
graingert
on Jan 14, 2025
ContributorAuthor
More actions

See #110012 test_socket raises warnings right now.

I'm thinking we mark these as default with catch_warnings in the test for now?
hugovk
added a commit that references this issue on Jan 17, 2025
gh-128770: raise warnings as errors in test suite - except for test_s…
7807b40
miss-islington
added a commit that references this issue on Jan 17, 2025
pythongh-128770: raise warnings as errors in test suite - except for …
128244d
hugovk
added 2 commits that reference this issue on Jan 17, 2025
Revert "pythongh-128770: raise warnings as errors in test suite - exc…
0342073
Revert "gh-128770: raise warnings as errors in test suite - except fo…
76856ae
graingert
added a commit that references this issue on Jan 18, 2025
Reapply "pythongh-128770: raise warnings as errors in test suite - ex…
a6bb15e
srinivasreddy
added 2 commits that reference this issue on Jan 21, 2025
pythongh-128770: raise warnings as errors in test suite - except for …
f12e00e
Revert "pythongh-128770: raise warnings as errors in test suite - exc…
076d0eb
pablogsal
added a commit that references this issue on Jan 22, 2025
gh-128770: fix ResourceWarning in test_pyrepl (#128906)
24c84d8
miss-islington
added a commit that references this issue on Jan 22, 2025
pythongh-128770: fix ResourceWarning in test_pyrepl (pythonGH-128906)
79bba48
pablogsal
added a commit that references this issue on Jan 22, 2025
[3.13] gh-128770: fix ResourceWarning in test_pyrepl (GH-128906) (#12…
67971cd
hugovk
added a commit that references this issue on Mar 27, 2025
gh-128770: raise warnings as errors in test suite - except for test_s…
8a00c9a
graingert
added a commit that references this issue on Mar 27, 2025
pythongh-128770: raise warnings as errors in test suite - except for …
18dd42a
hugovk
added a commit that references this issue on Mar 29, 2025
[3.13] gh-128770: raise warnings as errors in test suite - except for…
07d4c7e
diegorusso
added a commit that references this issue on Apr 1, 2025
pythongh-128770: raise warnings as errors in test suite - except for …
b1996a3

kulikjak commented on Apr 9, 2025

kulikjak

on Apr 9, 2025

Contributor

More actions

Hi, after this change (specifically #131802), I am seeing the following errors when running test_peg_generator:

======================================================================
ERROR: test_with_stmt_with_paren (test.test_peg_generator.test_c_parser.TestCParser.test_with_stmt_with_paren)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/Python-3.13.3/Lib/test/test_peg_generator/test_c_parser.py", line 113, in setUp
    cmd = support.missing_compiler_executable()
  File "/build/Python-3.13.3/Lib/test/support/__init__.py", line 1916, in missing_compiler_executable
    if spawn.find_executable(cmd[0]) is None:
       ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/usr/lib/python3.13/vendor-packages/setuptools/_distutils/spawn.py", line 87, in find_executable
    warnings.warn(
    ~~~~~~~~~~~~~^
        'Use shutil.which instead of find_executable', DeprecationWarning, stacklevel=2
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
DeprecationWarning: Use shutil.which instead of find_executable

I tried replacing spawn.find_executable with shutil.which as the warning suggests, but then I get a different distutils warning and test still fails:

======================================================================
ERROR: test_with_stmt_with_paren (test.test_peg_generator.test_c_parser.TestCParser.test_with_stmt_with_paren)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/Python-3.13.3/Lib/test/test_peg_generator/test_c_parser.py", line 395, in test_with_stmt_with_paren
    self.run_test(grammar_source, test_source)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/build/Python-3.13.3/Lib/test/test_peg_generator/test_c_parser.py", line 134, in run_test
    self.build_extension(grammar_source)
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/build/Python-3.13.3/Lib/test/test_peg_generator/test_c_parser.py", line 131, in build_extension
    generate_parser_c_extension(grammar, Path('.'), library_dir=self.library_dir)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/build/Python-3.13.3/Tools/peg_generator/pegen/testutil.py", line 107, in generate_parser_c_extension
    compile_c_extension(
    ~~~~~~~~~~~~~~~~~~~^
        str(source),
        ^^^^^^^^^^^^
    ...<3 lines>...
        library_dir=library_dir,
        ^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/build/Python-3.13.3/Tools/peg_generator/pegen/build.py", line 98, in compile_c_extension
    from setuptools._distutils.dep_util import newer_group
  File "/usr/lib/python3.13/vendor-packages/setuptools/_distutils/dep_util.py", line 9, in __getattr__
    warnings.warn(
    ~~~~~~~~~~~~~^
        "dep_util is Deprecated. Use functions from setuptools instead.",
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        DeprecationWarning,
        ^^^^^^^^^^^^^^^^^^^
        stacklevel=2,
        ^^^^^^^^^^^^^
    )
    ^
DeprecationWarning: dep_util is Deprecated. Use functions from setuptools instead.

This is with 3.13 only, which makes sense as that is the only one which we have setuptools for also installed.

I wonder what is different about my setup considering that nobody else reported this?

nascheme
added a commit that references this issue on Apr 9, 2025
Call resetwarnings() in a few places.
42d157b
kulikjak commented on May 6, 2025
kulikjak
on May 6, 2025
Contributor
More actions
FYI: both these issues were fixed recently in #132906 and #132921.

I also encountered another issue uncovered by this change and created an issue for it here: #133509
graingert commented on May 19, 2025
graingert
on May 19, 2025
ContributorAuthor
More actions
this is in place now
Reacted by Hugo van Kemenade
graingert
closed this as completed on May 19, 2025
AmazerThaDik commented on May 19, 2025
AmazerThaDik
on May 19, 2025 via email · Hidden as off-topic
show commentMore actions

Metadata

Assignees

No one assigned

Labels

testsTests in the Lib/test dirTests in the Lib/test dirtype-featureA feature request or enhancementA feature request or enhancement

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

configure warnings as errors in the test suite #128770

Description

Feature or enhancement

Proposal:

Has this already been discussed elsewhere?

Links to previous discussion of this feature:

Linked PRs

Activity

sobolevn commented on Jan 13, 2025

graingert commented on Jan 14, 2025

graingert commented on Jan 14, 2025

kulikjak commented on Apr 9, 2025

kulikjak commented on May 6, 2025

graingert commented on May 19, 2025

AmazerThaDik commented on May 19, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions