GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-109276: libregrtest: WASM use filename for JSON by vstinner · Pull Request #109340 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Sep 12, 2023 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

On Emscripten and WASI platforms, or if --python command line option is used, libregrtest now uses a filename for the JSON file.

Emscripten and WASI buildbot workers run the main test process with a different Python (Linux) which spawns Emscripten/WASI processes using the command specified in --python command line option. Passing a file descriptor from the parent process to the child process doesn't work in this case.

  • Add RunTests.json_file_use_filename() method.
  • Add a test in test_regrtest on the --python command line option.
  • test_regrtest: add parallel=False parameter.
  • Issue: Enhance Python regrtest test runner (test.libregrtest) #109276

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@vstinner vstinner added the skip news label Sep 12, 2023
@bedevere-app bedevere-app Bot added the awaiting review label Sep 12, 2023
@bedevere-app bedevere-app Bot mentioned this pull request Sep 12, 2023
Enhance Python regrtest test runner (test.libregrtest) #109276
Closed
@vstinner vstinner mentioned this pull request Sep 12, 2023
gh-109276: libregrtest: WASM use filename for JSON #109326
Closed
@vstinner
vstinner force-pushed the regrtest_json_file3 branch from 2948d9c to 5e88a04 Compare September 12, 2023 21:00
@vstinner
pythongh-109276: libregrtest: WASM use filename for JSON
d19861f 
On Emscripten and WASI platforms, or if --python command line option
is used, libregrtest now uses a filename for the JSON file.

Emscripten and WASI buildbot workers run the main test process with a
different Python (Linux) which spawns Emscripten/WASI processes using
the command specified in --python command line option. Passing a file
descriptor from the parent process to the child process doesn't work
in this case.

* Add JsonFile and JsonFileType classes
* Add RunTests.json_file_use_filename() method.
* Add a test in test_regrtest on the --python command line option.
* test_regrtest: add parallel=False parameter.
* Split long RunWorkers._runtest() function into sub-functions.
@vstinner
vstinner force-pushed the regrtest_json_file3 branch from 5e88a04 to d19861f Compare September 12, 2023 22:16
@vstinner
vstinner merged commit 75cdd9a into python:main Sep 12, 2023
@vstinner
vstinner deleted the regrtest_json_file3 branch September 12, 2023 22:41
@bedevere-app bedevere-app Bot removed the awaiting review label Sep 12, 2023
@vstinner

vstinner commented Sep 12, 2023

Copy link
Copy Markdown
Member Author

First, I wrote PR #109326 to debug the issue and understand what's going on. Then I wrote this clean PR.

Well. Sadly, this PR doesn't work as expected because I misunderstood how Emscripten/WASI buildbot workers run tests.

On these buildbots, a first Linux x86-64 Python binary is used to run python -m test. This one lists tests and then spawns test worker processes.

These test worker processes use the Emscripten/WASI runtime! So the parent and child processes don't run the same "platform"!

Before, the result was serialized as JSON at the end of stdout. But it causes issues, so I enhanced libregrtest to write the JSON into a separated file descriptor (change made 2 days ago).

First probem: apparently, currently, it's not possible to pass a file descriptor from the Linux x86-64 Python to the Emscripten/WASI Python: it fails with OSError: [Errno 8] Bad file descriptor.

So I wrote this PR to use a named filename for the JSON file. Second problem: the Emscripten/WASI Python runtime doesn't have a full access to the filesystem, and paths are mapped differently.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
vstinner added a commit to vstinner/cpython that referenced this pull request Sep 13, 2023
@vstinner
pythongh-109276: libregrtest: WASM use filename for JSON (python#109340)
3fab186 
On Emscripten and WASI platforms, or if --python command line option
is used, libregrtest now uses a filename for the JSON file.

Emscripten and WASI buildbot workers run the main test process with a
different Python (Linux) which spawns Emscripten/WASI processes using
the command specified in --python command line option. Passing a file
descriptor from the parent process to the child process doesn't work
in this case.

* Add JsonFile and JsonFileType classes
* Add RunTests.json_file_use_filename() method.
* Add a test in test_regrtest on the --python command line option.
* test_regrtest: add parallel=False parameter.
* Split long RunWorkers._runtest() function into sub-functions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

1 participant

@vstinner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.