GeezerVison is showing github.com — gh-109276: libregrtest: add RunTests.work_dir by vstinner · Pull Request #109290 · python/cpython
Conversation
vstinner
commented
Sep 11, 2023
•
edited by bedevere-app
Bot
Loading
Uh oh!
There was an error while loading. Please reload this page.
- WorkerThread now always creates a temporary directory, even on Emscripten and WASI: it's used as the working directory of the test worker process.
- Fix Emscripten and WASI: start the test worker process in the Python source code directory, where 'python.js' and 'python.wasm' can be found. Then worker_process() goes to the temporary directory created to run tests.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
This PR fix how libregrtest creates temporary directories for worker processes and which directory is used as the working directory of worker processes. It seems like two tests fail with this change since they leak temporary files/directories!
- test.test_asyncio.test_events
- test_compileall
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
It seems like two tests fail with this change since they leak temporary files/directories!
test.test_asyncio.test_events
test_compileall
I created issue #109295 to track this issue and PR #109298 to fix it.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Oh, and Windows two tests also failed, but other tests: test_generated_cases and test_os.
ERROR: test_access_denied (test.test_os.StatAttributeTests.test_access_denied)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\a\cpython\cpython\Lib\test\test_os.py", line 742, in test_access_denied
create_file(fname, b'ABC')
File "D:\a\cpython\cpython\Lib\test\test_os.py", line 94, in create_file
with open(filename, "xb", 0) as fp:
^^^^^^^^^^^^^^^^^^^^^^^
FileExistsError: [Errno 17] File exists: 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\test_python_worker_a5djsgg5\\@test_5340_tmp�'
and
0:19:14 load avg: 6.45 [2/3/2] test_generated_cases failed (uncaught exception)
Re-running test_generated_cases in verbose mode
test test_generated_cases crashed -- Traceback (most recent call last):
File "D:\a\cpython\cpython\Lib\test\libregrtest\single.py", line 139, in _runtest_env_changed_exc
_load_run_test(result, runtests)
File "D:\a\cpython\cpython\Lib\test\libregrtest\single.py", line 87, in _load_run_test
test_mod = importlib.import_module(module_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\a\cpython\cpython\Lib\importlib\__init__.py", line 88, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1381, in _gcd_import
File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
File "<frozen importlib._bootstrap>", line 1325, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 929, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 1006, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "D:\a\cpython\cpython\Lib\test\test_generated_cases.py", line 11, in <module>
import generate_cases
File "D:\a\cpython\cpython\Tools\cases_generator\generate_cases.py", line 41, in <module>
DEFAULT_INPUT = os.path.relpath(os.path.join(ROOT, "Python/bytecodes.c"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\a\cpython\cpython\Lib\ntpath.py", line 791, in relpath
raise ValueError("path is on mount %r, start on mount %r" % (
ValueError: path is on mount 'D:', start on mount 'C:'
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
* Enable collect_sysconfig() on Windows.
* Add sysconfig 'abs_builddir' and 'srcdir'
* Add sysconfig.is_python_build()
* Add tempfile.gettempdir()
* Remove compatiblity with Python 2.7 (print_function).
* WorkerThread now always creates a temporary directory, even on
Emscripten and WASI: it's used as the working directory of the test
worker process.
* Fix Emscripten and WASI: start the test worker process in the
Python source code directory, where 'python.js' and 'python.wasm'
can be found. Then worker_process() goes to the temporary directory
created to run tests.
* --cleanup now also removes "temporary" directories of test worker
processes (in /tmp).
|
Many tests fail on the macOS CI because the AF_UNIX domain name is too longer (more than ~100 bytes):
15 tests failed:
test.test_asyncio.test_unix_events
test.test_concurrent_futures.test_as_completed
test.test_concurrent_futures.test_deadlock
test.test_concurrent_futures.test_init
test.test_concurrent_futures.test_process_pool
test.test_concurrent_futures.test_shutdown
test.test_concurrent_futures.test_wait
test.test_multiprocessing_forkserver.test_manager
test.test_multiprocessing_forkserver.test_misc
test.test_multiprocessing_forkserver.test_processes
test.test_multiprocessing_forkserver.test_threads
test.test_multiprocessing_spawn.test_manager
test.test_multiprocessing_spawn.test_misc
test.test_multiprocessing_spawn.test_processes
test_multiprocessing_main_handling
The main test process:
tempfile.gettempdir: /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T
self.tmp_dir: /Users/runner/work/cpython/cpython/build
runtests.work_dir: /Users/runner/work/cpython/cpython/build/test_python_4592æ
cwd: /Users/runner/work/cpython/cpython/build/test_python_4592æ
A worker test process:
worker test_turtle (pid 4597): /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/test_python_worker_2eq4te_m
/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/test_python_worker_2eq4te_m is 84 (ASCII) characters long.
multiprocessing.util.get_temp_dir() creates a temporary directory with pymp-XXXXXXXX (13 characters) name.
multiprocessing.connection.arbitrary_connection(AF_UNIX) creates a listener-XXXXXXXX (17 characters) temporary file in pymp-XXXXXXXX directory.
A UNIX domain name liike /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/test_python_worker_2eq4te_m/pymp-XXXXXXXX/listener-XXXXXXXX is already 117 characters long.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Windows x64:
- Python source code:
D:\a\cpython\cpython (drive D:)
- Temporary directory:
C:\Users\RUNNER~1\AppData\Local\Temp (drive C:)
test.pythoninfo:
sysconfig.is_python_build: True
sysconfig[prefix]: D:\a\cpython\cpython
sysconfig[srcdir]: D:\a\cpython\cpython
tempfile.gettempdir: C:\Users\RUNNER~1\AppData\Local\Temp
Main test process:
tempfile.gettempdir: C:\Users\RUNNER~1\AppData\Local\Temp
self.tmp_dir: D:\a\cpython\cpython\build
runtests.work_dir: D:\a\cpython\cpython\build\test_python_3212�
cwd: D:\a\cpython\cpython\build\test_python_3212�
A test worker process:
C:\Users\RUNNER~1\AppData\Local\Temp\test_python_worker_xa_vmpox => C:\Users\runneradmin\AppData\Local\Temp\test_python_worker_xa_vmpox
test_generated_cases was skipped:
0:16:54 load avg: 11.61 [393/463/2] test_generated_cases skipped -- running (1): test_subprocess (1 min 1 sec)
test_generated_cases skipped -- the current working directory and the Python source code directory have different mount drives (C: and D:)
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
PR #106219 is awaiting for an Emscripten/WASI fix for libregrtest.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
This change seems to be too disruptive, so I wrote a more conservative change (come back to old behavior): PR #109313.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
This change seems to be too disruptive, so I wrote a more conservative change (come back to old behavior): PR #109313.
Merged. I close this PR.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|