GeezerVison is showing github.com — gh-108297: Update test_crashers by vstinner · Pull Request #108299 · python/cpython
Conversation
vstinner
commented
Aug 22, 2023
•
edited by AlexWaygood
Loading
Uh oh!
There was an error while loading. Please reload this page.
-
Rename Lib/test/crashers/ to Lib/test/test_crashers/.
-
Move Lib/test/test_crashers.py to Lib/test/test_crashers/__init__.py.
-
test_crashers is no longer skipped and makes sure that scripts do crash, and no simply fail with a non-zero exit code.
-
Update bogus_code_obj.py to use CodeType.replace().
-
Scripts crashing Python now uses SuppressCrashReport of test.support to not create coredump files.
-
Remove Lib/test/crashers/ scripts which no longer crash:
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
I don't understand why they should be moved.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
I created #108303 to move test files to sub-directories: see the rationale there.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
* Rename Lib/test/crashers/ to Lib/test/test_crashers/.
* Move Lib/test/test_crashers.py to
Lib/test/test_crashers/__init__.py.
* test_crashers is no longer skipped and makes sure that scripts do
crash, and no simply fail with a non-zero exit code.
* Update bogus_code_obj.py to use CodeType.replace().
* Scripts crashing Python now uses SuppressCrashReport of
test.support to not create coredump files.
* Remove Lib/test/crashers/ scripts which no longer crash:
* recursive_call.py: fixed by pythongh-89419
* mutation_inside_cyclegc.py: fixed by pythongh-97922
* trace_at_recursion_limit.py: fixed by Python 3.7
|
test_crashers was skipped in 2011 by Antoine Pitrou: commit 89ba56d.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
🤖 New build scheduled with the buildbot fleet by @vstinner for commit ecb3fd6 🤖
If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
vstinner
commented
Aug 23, 2023
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
|
Buildbot failures.
test_crashers failed on macOS: underlying_dict.py didn't crash.
Checking crasher: underlying_dict.py
AssertionError: Process return code is 0
(...)
stdout:
---
2
---
Unrelated: 1 test altered the execution environment: test_ssl. I created #108342 for this issue.
- AMD64 Windows10 PR
- AMD64 Windows11 Bigmem PR
- ARM64 Windows Non-Debug PR
- ARM64 Windows PR
Unrelated: aarch64 Fedora Stable Clang Installed PR: 1 test failed: test_venv
======================================================================
ERROR: test_zippath_from_non_installed_posix (test.test_venv.BasicTest.test_zippath_from_non_installed_posix)
Test that when create venv from non-installed python, the zip path
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/buildbot/buildarea/pull_request.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.13/test/test_venv.py", line 612, in test_zippath_from_non_installed_posix
subprocess.check_call(cmd, env=child_env)
(...)
File "/home/buildbot/buildarea/pull_request.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.13/subprocess.py", line 1950, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpqzy4slzu/bin/python3'
======================================================================
FAIL: test_upgrade_dependencies (test.test_venv.BasicTest.test_upgrade_dependencies)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/buildbot/buildarea/pull_request.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.13/test/test_venv.py", line 236, in test_upgrade_dependencies
builder.upgrade_dependencies(fake_context)
(...)
File "/home/buildbot/buildarea/pull_request.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.13/test/test_venv.py", line 222, in pip_cmd_checker
self.assertEqual(
AssertionError: Lists differ: ['/tmp/tmpt86asn2c/bin/python3', '-m', 'pip', 'install', '--upgrade', 'pip'] != ['/tmp/tmpt86asn2c/bin/python3.13', '-m', 'pip', 'install', '--upgrade', 'pip']
First differing element 0:
'/tmp/tmpt86asn2c/bin/python3'
'/tmp/tmpt86asn2c/bin/python3.13'
- ['/tmp/tmpt86asn2c/bin/python3', '-m', 'pip', 'install', '--upgrade', 'pip']
+ ['/tmp/tmpt86asn2c/bin/python3.13', '-m', 'pip', 'install', '--upgrade', 'pip']
? +++
x86 Gentoo Installed with X PR: same error.
Unrelated: AMD64 RHEL8 FIPS Only Blake2 Builtin Hash PR
ERROR: test_hmac_sha1 (test.test_socket.LinuxKernelCryptoAPI.test_hmac_sha1)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/buildbot/buildarea/pull_request.cstratak-RHEL8-fips-x86_64.no-builtin-hashes-except-blake2/build/Lib/test/test_socket.py", line 6477, in test_hmac_sha1
algo.setsockopt(socket.SOL_ALG, socket.ALG_SET_KEY, b"Jefe")
OSError: [Errno 22] Invalid argument
(...)
1 test failed: test_socket
Unrelated: PPC64 Fedora PR: https://buildbot.python.org/all/#/builders/256/builds/1206
checking for stdlib extension module _ssl... missing
(...)
Could not build the ssl module!
Python requires a OpenSSL 1.1.1 or newer
(...)
0:38:47 load avg: 2.92 [186/447/2] test_math crashed (Exit code 1) -- running: test_io (1 min 2 sec)
Timeout (0:15:00)!
Thread 0x00003fff7d985450 (most recent call first):
File "/home/shager/cpython-buildarea/pull_request.edelsohn-fedora-ppc64/build/Lib/test/test_math.py", line 1367 in test_sumprod_stress
(...)
1:13:23 load avg: 1.21 Re-running test_math in verbose mode
Ran 0 tests in 0.000s
NO TESTS RAN
(...)
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
test_build_cpp03 (test.test_cppext.TestCPPExt.test_build_cpp03) ... ERROR
(...)
1 test failed: test_cppext
2 re-run tests: test_cppext test_math
1 test run no tests: test_math
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Buildbot failures.
Unrelated. aarch64 Fedora Stable Clang Installed PR: https://buildbot.python.org/all/#/builders/819/builds/750
1 test failed: test_gdb
Same error on aarch64 Fedora Stable Clang PR.
Unrelated. buildbot/aarch64 RHEL8 Refleaks PR: https://buildbot.python.org/all/#/builders/802/builds/975
0:02:32 load avg: 27.87 [420/447/1] test_ssl failed (env changed) (1 min 20 sec) -- running: (...)
beginning 6 repetitions
123456
Resource 'ipv6.google.com' is not available
.Resource 'ipv6.google.com' is not available
Warning -- threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 2)
Warning -- Dangling thread: <SingleConnectionTestServerThread(preauth_data_to_tls_server, stopped 281473271591360)>
Warning -- Dangling thread: <_MainThread(MainThread, started 281473314145680)>
.Resource 'ipv6.google.com' is not available
Warning -- threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 2)
Warning -- Dangling thread: <SingleConnectionTestServerThread(preauth_data_to_tls_server, stopped 281473271591360)>
Warning -- Dangling thread: <_MainThread(MainThread, started 281473314145680)>
.Resource 'ipv6.google.com' is not available
Warning -- threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 2)
Warning -- Dangling thread: <SingleConnectionTestServerThread(preauth_data_to_tls_server, stopped 281473271591360)>
Warning -- Dangling thread: <_MainThread(MainThread, started 281473314145680)>
.Resource 'ipv6.google.com' is not available
Warning -- threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 2)
Warning -- Dangling thread: <SingleConnectionTestServerThread(preauth_data_to_tls_server, stopped 281473271591360)>
Warning -- Dangling thread: <_MainThread(MainThread, started 281473314145680)>
.Resource 'ipv6.google.com' is not available
Warning -- threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 2)
Warning -- Dangling thread: <SingleConnectionTestServerThread(preauth_data_to_tls_server, stopped 281473271591360)>
Warning -- Dangling thread: <_MainThread(MainThread, started 281473314145680)>
.
(...)
1 test altered the execution environment:
test_ssl
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Unrelated: s390x RHEL7 LTO + PGO PR: https://buildbot.python.org/all/#/builders/418/builds/1174
Could not build the ssl module!
Python requires a OpenSSL 1.1.1 or newer
(...)
cc1plus: error: command line option ‘-std=gnu11’ is valid for C/ObjC but not for C++ [-Werror]
That's the issue #105776: test_cpp and -std=c11.
- s390x RHEL7 LTO PR: Same error.
- s390x RHEL7 PR: Same error
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
test_crashers failed on macOS: underlying_dict.py didn't crash.
That's a major issue :-( The test was skipped in 2011 because it was not reliable. Is it important that crashers scripts crash "in a reliable way"? Many trigger undefined behaviors which have... undefined behaviors :-( Building Python with UBSan would make these undefined behaviors to have a deterministic behavior, but UBSan is rarely used. Today, we only have ASAN GHA job and buildbots.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
If the test is crashed in 50% of cases, repeating it twice will make it crashing in 75%, and repeating it 10 times will make it crashing in 99.9%. If attempts are independent.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
I created this PR to reenable test_crashers, but now I dislike this idea since crashes are not deterministic: sometimes scripts crash Python, sometimes not. I would prefer to remove test_crashers and keep Lib/crashers/ instead.
I close this PR.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|