GeezerVison is showing github.com — gh-109276: libregrtest: WASM use filename for JSON by vstinner · Pull Request #109326 · python/cpython
Conversation
vstinner
commented
Sep 12, 2023
•
edited by bedevere-app
Bot
Loading
Uh oh!
There was an error while loading. Please reload this page.
On Emscripten and WASI platforms, libregrtest now uses a filename for the JSON file. Passing a file descriptor to a child process doesn't work on these platforms.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
This was referenced Sep 12, 2023
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
🤖 New build scheduled with the buildbot fleet by @vstinner for commit c45529b 🤖
The command will test the builders whose names match following regular expression: wasm32
The builders matched are:
wasm32-emscripten node (dynamic linking) PRwasm32-wasi PRwasm32-emscripten node (pthreads) PRwasm32-emscripten browser (dynamic linking, no tests) PR
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Emscripten/WASI still use a file descriptor. It makes no sense to me :-( I'm not sure if buildbots got the right commit.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
On Emscripten and WASI platforms, libregrtest now uses a filename for
the JSON file. Passing a file descriptor to a child process doesn't
work on these platforms.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
🤖 New build scheduled with the buildbot fleet by @vstinner for commit 9c052ba 🤖
The command will test the builders whose names match following regular expression: wasm32
The builders matched are:
wasm32-emscripten node (dynamic linking) PRwasm32-wasi PRwasm32-emscripten node (pthreads) PRwasm32-emscripten browser (dynamic linking, no tests) PR
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
🤖 New build scheduled with the buildbot fleet by @vstinner for commit 9c052ba 🤖
The command will test the builders whose names match following regular expression: wasm32
The builders matched are:
wasm32-emscripten node (dynamic linking) PRwasm32-wasi PRwasm32-emscripten node (pthreads) PRwasm32-emscripten browser (dynamic linking, no tests) PR
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
wasm32-emscripten node (dynamic linking) PR failed with:
_PYTHON_HOSTRUNNER='/opt/emsdk/node/current/bin/node --experimental-wasm-bigint' _PYTHON_PROJECT_BASE=/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.emscripten-node-dl/build/build_oot/host _PYTHON_HOST_PLATFORM=emscripten-wasm32 PYTHONPATH=/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.emscripten-node-dl/build/build_oot/host/build/lib.emscripten-wasm32-3.13:../../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__emscripten_wasm32-emscripten ../build/python ../../Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=900 -j2 --junit-xml test-results.xml -j2
== CPython 3.13.0a0 (heads/refs/pull/109326/merge-dirty:a70b894cdf, Sep 12 2023, 15:11:12) [GCC 9.4.0]
== Linux-5.15.0-1019-azure-x86_64-with-glibc2.31 little-endian
== Python build: release
== cwd: /opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.emscripten-node-dl/build/build_oot/host/build/test_python_worker_432395æ
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
Using random seed 96764921
0:00:00 load avg: 2.54 Run 463 tests in parallel using 2 worker processes (timeout: 15 min, worker timeout: 20 min)
main process is_emscripten: False
main process is_wasi: False
main process JSON_FILE_USE_FILENAME: False
main process is_emscripten: False
main process is_wasi: False
main process JSON_FILE_USE_FILENAME: False
main process: create TemporaryFile
main process json_type file: <class 'int'>
main process json_type: 5
main process: create TemporaryFile
main process json_type file: <class 'int'>
main process json_type: 6
0:00:03 load avg: 2.54 [ 1/463/1] test_optparse process crashed (Exit code 1)
worker: json_file type: <class 'int'>
worker: json_file: 5
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.emscripten-node-dl/build/Lib/test/libregrtest/worker.py", line 131, in <module>
main()
File "/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.emscripten-node-dl/build/Lib/test/libregrtest/worker.py", line 127, in main
worker_process(worker_json)
File "/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.emscripten-node-dl/build/Lib/test/libregrtest/worker.py", line 110, in worker_process
with open(json_file, 'w', encoding='utf-8') as fp:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 8] Bad file descriptor
wasm32-wasi PR failed with:
_PYTHON_HOSTRUNNER='wasmtime run --env PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib --mapdir /::../.. --' _PYTHON_PROJECT_BASE=/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host _PYTHON_HOST_PLATFORM=wasi-wasm32 PYTHONPATH=/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/lib.wasi-wasm32-3.13:../../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__wasi_wasm32-wasi ../build/python ../../Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=900 -j2 --junit-xml test-results.xml -j2
== CPython 3.13.0a0 (heads/refs/pull/109326/merge-dirty:a70b894cdf, Sep 12 2023, 15:14:49) [GCC 9.4.0]
== Linux-5.15.0-1019-azure-x86_64-with-glibc2.31 little-endian
== Python build: release
== cwd: /opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_worker_458151æ
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
Using random seed 76879218
0:00:00 load avg: 2.51 Run 463 tests in parallel using 2 worker processes (timeout: 15 min, worker timeout: 20 min)
main process is_emscripten: False
main process is_wasi: False
main process JSON_FILE_USE_FILENAME: False
main process is_emscripten: False
main process is_wasi: False
main process JSON_FILE_USE_FILENAME: False
main process: create TemporaryFile
main process json_type file: <class 'int'>
main process json_type: 5
main process: create TemporaryFile
main process json_type file: <class 'int'>
main process json_type: 6
0:00:00 load avg: 2.47 [ 1/463/1] test_embed process crashed (Exit code 1)
/Lib/test/support/os_helper.py:531: RuntimeWarning: tests may fail, unable to create temporary directory '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_48104æ': [Errno 44] No such file or directory: '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_48104æ'
with temp_dir(path=name, quiet=quiet) as temp_path:
/Lib/test/support/os_helper.py:532: RuntimeWarning: tests may fail, unable to change the current working directory to '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_48104æ': [Errno 44] No such file or directory: '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_48104æ'
with change_cwd(temp_path, quiet=quiet) as cwd_dir:
worker: json_file type: <class 'int'>
worker: json_file: 6
test_embed skipped -- test module requires subprocess
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/Lib/test/libregrtest/worker.py", line 131, in <module>
main()
File "/Lib/test/libregrtest/worker.py", line 127, in main
worker_process(worker_json)
File "/Lib/test/libregrtest/worker.py", line 110, in worker_process
with open(json_file, 'w', encoding='utf-8') as fp:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 8] Bad file descriptor
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
I just merged PR #109337 which collect more data in test.support and test.support sub-modules.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
I just merged PR #109337 which collect more data in test.support and test.support sub-modules.
wasm32-emscripten node (dynamic linking) 3.x: test.pythoninfo, support.is_emscripten: True and sys.platform: emscripten as expected.
support.has_fork_support: False
support.has_socket_support: False
support.has_strftime_extensions: False
support.has_subprocess_support: False
support.is_android: False
support.is_emscripten: True
support.is_jython: False
support.is_wasi: False
support_threading_helper.can_start_thread: False
sys.platform: emscripten
wasm32-emscripten node (pthreads) 3.x: test.pythoninfo, support.is_emscripten: True and sys.platform: emscripten as expected
support.has_fork_support: False
support.has_socket_support: False
support.has_strftime_extensions: False
support.has_subprocess_support: False
support.is_android: False
support.is_emscripten: True
support.is_jython: False
support.is_wasi: False
support_threading_helper.can_start_thread: True
sys.platform: emscripten
wasm32-wasi 3.x: test.pythoninfo, support.is_wasi: True and sys.platform: wasi as expected
support.has_fork_support: False
support.has_socket_support: False
support.has_strftime_extensions: True
support.has_subprocess_support: False
support.is_android: False
support.is_emscripten: False
support.is_jython: False
support.is_wasi: True
support_threading_helper.can_start_thread: False
sys.platform: wasi
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
🤖 New build scheduled with the buildbot fleet by @vstinner for commit c436370 🤖
The command will test the builders whose names match following regular expression: wasm32
The builders matched are:
wasm32-emscripten node (dynamic linking) PRwasm32-wasi PRwasm32-emscripten node (pthreads) PRwasm32-emscripten browser (dynamic linking, no tests) PR
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
🤖 New build scheduled with the buildbot fleet by @vstinner for commit 4c8d5c8 🤖
The command will test the builders whose names match following regular expression: wasm32
The builders matched are:
wasm32-emscripten node (dynamic linking) PRwasm32-wasi PRwasm32-emscripten node (pthreads) PRwasm32-emscripten browser (dynamic linking, no tests) PR
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
🤖 New build scheduled with the buildbot fleet by @vstinner for commit 4e2cfab 🤖
The command will test the builders whose names match following regular expression: wasm32
The builders matched are:
wasm32-emscripten node (dynamic linking) PRwasm32-wasi PRwasm32-emscripten node (pthreads) PRwasm32-emscripten browser (dynamic linking, no tests) PR
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Ok, I wrote a clean fix from this debug PR: PR #109340.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
wasm32-wasi PR failed: the WASI test worker process doesn't seem to have access to /tmp/test_python_json_7sbyggem of the Linuxparent process :-(
_PYTHON_HOSTRUNNER='wasmtime run --env PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib --mapdir /::../.. --' _PYTHON_PROJECT_BASE=/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host _PYTHON_HOST_PLATFORM=wasi-wasm32 PYTHONPATH=/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/lib.wasi-wasm32-3.13:../../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__wasi_wasm32-wasi ../build/python ../../Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=900 -j2 --junit-xml test-results.xml -j2
1118966 JsonFileType: int | str
main process start sys.platform: linux
main process start is_emscripten: False
main process start is_wasi: False
main process python_cmd: ('wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm')
== CPython 3.13.0a0 (heads/refs/pull/109326/merge-dirty:0540fe293c, Sep 12 2023, 17:39:37) [GCC 9.4.0]
== Linux-5.15.0-1019-azure-x86_64-with-glibc2.31 little-endian
== Python build: release
== cwd: /opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_worker_1118966æ
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
Using random seed 30180538
0:00:00 load avg: 1.66 Run 463 tests in parallel using 2 worker processes (timeout: 15 min, worker timeout: 20 min)
main process: json_file_use_filename? ('wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm')
main process: json_file_use_filename? ('wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm')
main process: create NamedTemporaryFile /tmp/test_python_json_7sbyggem
main process json_file type: <class 'str'>
main process json_file: /tmp/test_python_json_7sbyggem
main process: create NamedTemporaryFile /tmp/test_python_json_yq7jvz76
main process json_file type: <class 'str'>
main process json_file: /tmp/test_python_json_yq7jvz76
main process executable: ('wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm')
main process worker cmd: ['wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm', '-bb', '-Wdefault', '-u', '-m', 'test.libregrtest.worker', '{"tests": ["test_bigaddrspace"], "fail_fast": false, "fail_env_changed": true, "match_tests": null, "ignore_tests": null, "match_tests_dict": null, "rerun": null, "forever": false, "pgo": false, "pgo_extended": false, "output_on_failure": true, "timeout": 900.0, "verbose": 0, "quiet": false, "hunt_refleak": null, "test_dir": null, "use_junit": true, "memory_limit": null, "gc_threshold": null, "use_resources": ["audio", "curses", "largefile", "network", "decimal", "cpu", "subprocess", "urlfetch", "gui", "walltime"], "python_cmd": ["wasmtime", "run", "--env", "PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib", "--mapdir", "/::../..", "--", "python.wasm"], "randomize": true, "random_seed": 30180538, "json_file": "/tmp/test_python_json_7sbyggem", "__runtests__": true}']
create_worker_process() json_file: filename
main process executable: ('wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm')
main process worker cmd: ['wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm', '-bb', '-Wdefault', '-u', '-m', 'test.libregrtest.worker', '{"tests": ["test_venv"], "fail_fast": false, "fail_env_changed": true, "match_tests": null, "ignore_tests": null, "match_tests_dict": null, "rerun": null, "forever": false, "pgo": false, "pgo_extended": false, "output_on_failure": true, "timeout": 900.0, "verbose": 0, "quiet": false, "hunt_refleak": null, "test_dir": null, "use_junit": true, "memory_limit": null, "gc_threshold": null, "use_resources": ["audio", "curses", "largefile", "network", "decimal", "cpu", "subprocess", "urlfetch", "gui", "walltime"], "python_cmd": ["wasmtime", "run", "--env", "PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib", "--mapdir", "/::../..", "--", "python.wasm"], "randomize": true, "random_seed": 30180538, "json_file": "/tmp/test_python_json_yq7jvz76", "__runtests__": true}']
create_worker_process() json_file: filename
0:00:00 load avg: 1.66 [ 1/463/1] test_bigaddrspace process crashed (Exit code 1)
42 JsonFileType: int | str
worker process sys.platform: wasi
worker process is_emscripten: False
worker process is_wasi: True
/Lib/test/support/os_helper.py:531: RuntimeWarning: tests may fail, unable to create temporary directory '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_230432æ': [Errno 44] No such file or directory: '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_230432æ'
with temp_dir(path=name, quiet=quiet) as temp_path:
/Lib/test/support/os_helper.py:532: RuntimeWarning: tests may fail, unable to change the current working directory to '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_230432æ': [Errno 44] No such file or directory: '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_230432æ'
with change_cwd(temp_path, quiet=quiet) as cwd_dir:
worker: json_file type: <class 'str'>
worker: json_file: /tmp/test_python_json_7sbyggem
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/Lib/test/libregrtest/worker.py", line 143, in <module>
main()
File "/Lib/test/libregrtest/worker.py", line 139, in main
worker_process(worker_json)
File "/Lib/test/libregrtest/worker.py", line 118, in worker_process
with open(json_file, 'w', encoding='utf-8') as fp:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 44] No such file or directory: '/tmp/test_python_json_7sbyggem'
Kill <WorkerThread #1 running test=test_venv pid=1118986 time=423 ms> process group
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Ok, I wrote a clean fix from this debug PR: PR #109340.
I merged this PR instead of this DEBUG PR.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|