GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — test: report ERROR wpt test results by legendecas · Pull Request #50429 · nodejs/node

Conversation

@legendecas

@legendecas legendecas commented Oct 27, 2023

Copy link
Copy Markdown
Member

When a wpt test file is exited for uncaught error, its result should be
recorded in the wptreport.json and uploaded to wpt.fyi.

For instance, html/webappapis/timers/evil-spec-example.any.js is
exited for uncaught error in Node.js but it shows as "MISSING" at
https://wpt.fyi/results/html/webappapis/timers?label=master&label=experimental&product=chrome&product=node.js&aligned.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to Node.js core tests and test infrastructure. labels Oct 27, 2023
@legendecas
legendecas force-pushed the wpt/report branch 2 times, most recently from 780117a to 111c759 Compare October 27, 2023 15:27
@legendecas

legendecas commented Oct 27, 2023

Copy link
Copy Markdown
Member Author

@panva It would be great if you could take a look at this when you get a chance since you originally created wpt.fyi report workflow. Thank you!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@panva
panva self-requested a review October 27, 2023 17:38
@panva

panva commented Oct 27, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

evaluation in WPTRunner.runJsTests() gets reported as a test in the ERRORed file, see https://staging.wpt.fyi/results/resource-timing/delivery-type.tentative.any.html?run_id=5544793184665600&run_id=5129270332489728, i ran your PR against staging.wpt.fyi

I believe it should just be the harness that has an error and then no tests in it, like so https://wpt.fyi/results/streams/readable-byte-streams/construct-byob-request.any.html?run_id=5105883752693760&run_id=5146271209226240&run_id=5855453021470720&run_id=5148705381941248&run_id=6261279079006208

Same for timeout in our case.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@legendecas

legendecas commented Oct 28, 2023

Copy link
Copy Markdown
Member Author

Right. I've removed the synthetic sub-tests from the report to wpt.fyi. Thanks for the suggestion.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@panva

panva commented Oct 28, 2023

Copy link
Copy Markdown
Member

I'll re-run the upload tonight to check what it looks like now.
legendecas reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@panva

panva commented Oct 30, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

The evaluation in WPTRunner.runJsTests() looks alright now, it's gone.

But why is there a newly added for entry for all idlharness-shadowrealm.window.html which isn't present on main and wasn't present before the last fixup?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
legendecas added 2 commits October 30, 2023 20:23
@legendecas
test: report error wpt test results
fa7b054 
When a wpt test file is exited for uncaught error, its result should be
recorded in the `wptreport.json` and uploaded to wpt.fyi.

For instance, `html/webappapis/timers/evil-spec-example.any.js` is
exited for uncaught error in Node.js but it shows as "MISSING" at
https://wpt.fyi/results/html/webappapis/timers?label=master&label=experimental&product=chrome&product=node.js&aligned.
@legendecas
test: recognize wpt completion error
e842497
@legendecas
legendecas force-pushed the wpt/report branch from 9e65978 to e842497 Compare October 30, 2023 12:23
@legendecas

legendecas commented Oct 30, 2023

Copy link
Copy Markdown
Member Author

Well, it seems a completion with error status was not recognized as a failed test and it was not reported to wpt.fyi. Fixed with e842497.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
panva
panva approved these changes Oct 30, 2023
View reviewed changes
@legendecas legendecas added commit-queue-rebase PRs the Commit Queue should land as multiple self-contained commits. request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. labels Oct 30, 2023
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 30, 2023
@panva panva added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Oct 30, 2023
@nodejs-github-bot

nodejs-github-bot commented Oct 31, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55342/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@github-actions github-actions Bot mentioned this pull request Nov 1, 2023
CI Reliability 2023-11-01 nodejs/reliability#704
Open
24 tasks
@nodejs-github-bot

nodejs-github-bot commented Nov 1, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55374/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@nodejs-github-bot

nodejs-github-bot commented Nov 1, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55378/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
This was referenced Nov 2, 2023
CI Reliability 2023-11-02 nodejs/reliability#705
Open
CI Reliability 2023-11-03 nodejs/reliability#706
Open
@legendecas legendecas added the commit-queue PRs queued for automated landing through the Commit Queue. label Nov 3, 2023
@nodejs-github-bot nodejs-github-bot added commit-queue-failed PRs whose Commit Queue landing failed and need manual intervention before retrying. and removed commit-queue PRs queued for automated landing through the Commit Queue. labels Nov 3, 2023
@nodejs-github-bot

nodejs-github-bot commented Nov 3, 2023

Copy link
Copy Markdown
Collaborator
Commit Queue failed
- Loading data for nodejs/node/pull/50429
✔  Done loading data for nodejs/node/pull/50429
----------------------------------- PR info ------------------------------------
Title      test: report ERROR wpt test results (#50429)
   ⚠  Could not retrieve the email or name of the PR author's from user's GitHub profile!
Branch     legendecas:wpt/report -> nodejs:main
Labels     test, author ready, needs-ci, commit-queue-rebase
Commits    2
 - test: report error wpt test results
 - test: recognize wpt completion error
Committers 1
 - legendecas 
PR-URL: https://github.com/nodejs/node/pull/50429
Reviewed-By: Filip Skokan 
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/50429
Reviewed-By: Filip Skokan 
--------------------------------------------------------------------------------
   ℹ  This PR was created on Fri, 27 Oct 2023 15:23:01 GMT
   ✔  Approvals: 1
   ✔  - Filip Skokan (@panva): https://github.com/nodejs/node/pull/50429#pullrequestreview-1704170475
   ✘  This PR needs to wait 0 more hours to land (or 0 hours if there is one more approval)
   ✔  Last GitHub CI successful
   ℹ  Last Full PR CI on 2023-11-01T06:12:59Z: https://ci.nodejs.org/job/node-test-pull-request/55378/
- Querying data for job/node-test-pull-request/55378/
   ✔  Last Jenkins CI successful
--------------------------------------------------------------------------------
   ✔  Aborted `git node land` session in /home/runner/work/node/node/.ncu
https://github.com/nodejs/node/actions/runs/6746717439
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@legendecas legendecas removed the commit-queue-failed PRs whose Commit Queue landing failed and need manual intervention before retrying. label Nov 3, 2023
@legendecas legendecas added the commit-queue PRs queued for automated landing through the Commit Queue. label Nov 3, 2023
@nodejs-github-bot nodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Nov 3, 2023
@nodejs-github-bot

nodejs-github-bot commented Nov 3, 2023

Copy link
Copy Markdown
Collaborator

Landed in fd2368e...a00f0b1
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@nodejs-github-bot nodejs-github-bot closed this Nov 3, 2023
nodejs-github-bot pushed a commit that referenced this pull request Nov 3, 2023
@legendecas @nodejs-github-bot
test: report error wpt test results
075ae82 
When a wpt test file is exited for uncaught error, its result should be
recorded in the `wptreport.json` and uploaded to wpt.fyi.

For instance, `html/webappapis/timers/evil-spec-example.any.js` is
exited for uncaught error in Node.js but it shows as "MISSING" at
https://wpt.fyi/results/html/webappapis/timers?label=master&label=experimental&product=chrome&product=node.js&aligned.

PR-URL: #50429
Reviewed-By: Filip Skokan <[email protected]>
nodejs-github-bot pushed a commit that referenced this pull request Nov 3, 2023
@legendecas @nodejs-github-bot
test: recognize wpt completion error
a00f0b1 
PR-URL: #50429
Reviewed-By: Filip Skokan <[email protected]>
@legendecas
legendecas deleted the wpt/report branch November 3, 2023 16:12
@github-actions github-actions Bot mentioned this pull request Nov 4, 2023
CI Reliability 2023-11-04 nodejs/reliability#707
Open
14 tasks
targos pushed a commit that referenced this pull request Nov 11, 2023
@legendecas @targos
test: report error wpt test results
4c93f96 
When a wpt test file is exited for uncaught error, its result should be
recorded in the `wptreport.json` and uploaded to wpt.fyi.

For instance, `html/webappapis/timers/evil-spec-example.any.js` is
exited for uncaught error in Node.js but it shows as "MISSING" at
https://wpt.fyi/results/html/webappapis/timers?label=master&label=experimental&product=chrome&product=node.js&aligned.

PR-URL: #50429
Reviewed-By: Filip Skokan <[email protected]>
targos pushed a commit that referenced this pull request Nov 11, 2023
@legendecas @targos
test: recognize wpt completion error
dadf77c 
PR-URL: #50429
Reviewed-By: Filip Skokan <[email protected]>
@targos targos mentioned this pull request Nov 12, 2023
v21.2.0 release proposal #50681
Merged
targos pushed a commit that referenced this pull request Nov 14, 2023
@legendecas @targos
test: report error wpt test results
7e3eb02 
When a wpt test file is exited for uncaught error, its result should be
recorded in the `wptreport.json` and uploaded to wpt.fyi.

For instance, `html/webappapis/timers/evil-spec-example.any.js` is
exited for uncaught error in Node.js but it shows as "MISSING" at
https://wpt.fyi/results/html/webappapis/timers?label=master&label=experimental&product=chrome&product=node.js&aligned.

PR-URL: #50429
Reviewed-By: Filip Skokan <[email protected]>
targos pushed a commit that referenced this pull request Nov 14, 2023
@legendecas @targos
test: recognize wpt completion error
ee75110 
PR-URL: #50429
Reviewed-By: Filip Skokan <[email protected]>
UlisesGascon pushed a commit that referenced this pull request Dec 11, 2023
@legendecas @UlisesGascon
test: report error wpt test results
ecfc951 
When a wpt test file is exited for uncaught error, its result should be
recorded in the `wptreport.json` and uploaded to wpt.fyi.

For instance, `html/webappapis/timers/evil-spec-example.any.js` is
exited for uncaught error in Node.js but it shows as "MISSING" at
https://wpt.fyi/results/html/webappapis/timers?label=master&label=experimental&product=chrome&product=node.js&aligned.

PR-URL: #50429
Reviewed-By: Filip Skokan <[email protected]>
UlisesGascon pushed a commit that referenced this pull request Dec 11, 2023
@legendecas @UlisesGascon
test: recognize wpt completion error
1db44b9 
PR-URL: #50429
Reviewed-By: Filip Skokan <[email protected]>
@UlisesGascon UlisesGascon mentioned this pull request Dec 12, 2023
v20.11.0 proposal #51124
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@panva panva panva approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue-rebase PRs the Commit Queue should land as multiple self-contained commits. needs-ci PRs that need a full CI run. test Issues and PRs related to Node.js core tests and test infrastructure.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@legendecas @panva @nodejs-github-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.