GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — test: make test-perf-hooks more robust and work with workers by joyeecheung · Pull Request #49197 · nodejs/node

Conversation

@joyeecheung

@joyeecheung joyeecheung commented Aug 16, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Previously the test makes several assumptions about the absolute values of the nodeTiming fields, which can make the test flaky on slow machines. This patch rewrites the test to check the relative values instead. It also updates the test to make it work with workers instead of directly skipping in workers.

Refs: nodejs/reliability#638

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bricss reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@joyeecheung
test: make test-perf-hooks more robust and work with workers
3ecbef5
Previously the test makes several assumptions about the absolute
values of the nodeTiming fields, which can make the test flaky
on slow machines. This patch rewrites the test to check the
relative values instead. It also updates the test to make it
work with workers instead of directly skipping in workers.
@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Aug 16, 2023
@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to Node.js core tests and test infrastructure. labels Aug 16, 2023
@joyeecheung

joyeecheung commented Aug 16, 2023

Copy link
Copy Markdown
Member Author

From nodejs/reliability#638 this test has been failing 6 recent PRs, hopefully this makes the flake go away..

Reason sequential/test-perf-hooks
Type JS_TEST_FAILURE
Failed PR 6 (#49104, #49127, #49149, #49162, #49164, #46391)
Appeared test-equinix-ubuntu1804_container-arm64-2, test-osuosl-ubuntu2004_sharedlibs_container-arm64-1, test-osuosl-ubuntu2004_container-armv7l-1, test-equinix-ubuntu2004_container-armv7l-2, test-azure_msft-win11_vs2022-arm64-4, test-equinix-ubuntu2004_container-arm64-2
First CI https://ci.nodejs.org/job/node-test-pull-request/53267/
Last CI https://ci.nodejs.org/job/node-test-pull-request/53336/
Example
not ok 3928 sequential/test-perf-hooks
  ---
  duration_ms: 426.79600
  severity: fail
  exitcode: 1
  stack: |-
    {
      name: 'node',
      entryType: 'node',
      startTime: 0,
      duration: { around: 285.6393041610718 },
      nodeStart: { around: 0 },
      v8Start: { around: 0 },
      bootstrapComplete: { around: 285.5992240905762, delay: 2500 },
      environment: { around: 0 },
      loopStart: -1,
      loopExit: -1
    }
    node:assert:399
        throw err;
        ^
    
    AssertionError [ERR_ASSERTION]: environment: 252.57520198822021 >= 250
        at checkNodeTiming (/home/iojs/build/workspace/node-test-commit-arm/test/sequential/test-perf-hooks.js:31:7)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-arm/test/sequential/test-perf-hooks.js:43:1)
        at Module._compile (node:internal/modules/cjs/loader:1241:14)
        at Module._extensions..js (node:internal/modules/cjs/loader:1295:10)
        at Module.load (node:internal/modules/cjs/loader:1091:32)
   ...
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Aug 16, 2023
@nodejs-github-bot

nodejs-github-bot commented Aug 16, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/53373/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

debadree25
debadree25 reviewed Aug 16, 2023
View reviewed changes
Comment thread test/sequential/test-perf-hooks.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

debadree25
debadree25 approved these changes Aug 16, 2023
View reviewed changes

@debadree25 debadree25 left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!
Thank you!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@debadree25 debadree25 added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Aug 16, 2023
@debadree25

debadree25 commented Aug 16, 2023

Copy link
Copy Markdown
Contributor

i tried stress testing this too locally with python3 tools/test.py test/sequential/test-perf-hooks.js --repeat 50 -j10 it passed well! so maybe could graduate to parallel too?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung

joyeecheung commented Aug 16, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

I am not sure why it was in sequential in the first place, but I think on the safe side we can just keep it there in this PR and if it looks well in the CI, move it to parallel afterwards (otherwise one doesn't know which is causing a regression if there is a regression).

debadree25 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@debadree25

debadree25 commented Aug 16, 2023

Copy link
Copy Markdown
Contributor

Yeah makes sense!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

aduh95
aduh95 approved these changes Aug 16, 2023
View reviewed changes
Comment thread test/sequential/test-perf-hooks.js
// Use a fairly large epsilon value, since we can only guarantee that the node
// process started up in 15 seconds.
assert(Math.abs(performance.timeOrigin - Date.now()) < 15000);
assert(testStartTime < 15000, `${testStartTime} >= 15000`);

@aduh95 aduh95 Aug 16, 2023

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit

Suggested change
assert(testStartTime < 15000, `${testStartTime} >= 15000`);
assert(testStartTime < 15000, `${testStartTime} ≥ 15000`);

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@joyeecheung joyeecheung Aug 17, 2023

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in general we use >= in this codebase instead?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@joyeecheung joyeecheung added the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 18, 2023
@joyeecheung

joyeecheung commented Aug 18, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

By the way the failure also shows up on v20.x-staging https://ci.nodejs.org/job/node-test-binary-armv7l/7027/RUN_SUBSET=js,nodes=ubuntu2004-armv7l/testReport/junit/(root)/sequential/test_perf_hooks/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot nodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 18, 2023
@nodejs-github-bot
nodejs-github-bot merged commit ecde9d9 into nodejs:main Aug 18, 2023
@nodejs-github-bot

nodejs-github-bot commented Aug 18, 2023

Copy link
Copy Markdown
Collaborator

Landed in ecde9d9

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

UlisesGascon pushed a commit that referenced this pull request Sep 10, 2023
@joyeecheung @UlisesGascon
test: make test-perf-hooks more robust and work with workers
9610008
Previously the test makes several assumptions about the absolute
values of the nodeTiming fields, which can make the test flaky
on slow machines. This patch rewrites the test to check the
relative values instead. It also updates the test to make it
work with workers instead of directly skipping in workers.

PR-URL: #49197
Refs: nodejs/reliability#638
Reviewed-By: Debadree Chatterjee <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
@UlisesGascon UlisesGascon mentioned this pull request Sep 10, 2023
v20.7.0 proposal #49592
Merged
targos pushed a commit that referenced this pull request Nov 27, 2023
@joyeecheung @targos
test: make test-perf-hooks more robust and work with workers
0261c5d
Previously the test makes several assumptions about the absolute
values of the nodeTiming fields, which can make the test flaky
on slow machines. This patch rewrites the test to check the
relative values instead. It also updates the test to make it
work with workers instead of directly skipping in workers.

PR-URL: #49197
Refs: nodejs/reliability#638
Reviewed-By: Debadree Chatterjee <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
@targos targos mentioned this pull request Nov 28, 2023
v18.19.0 release proposal #50953
Merged
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
test: make test-perf-hooks more robust and work with workers
e952ee9
Previously the test makes several assumptions about the absolute
values of the nodeTiming fields, which can make the test flaky
on slow machines. This patch rewrites the test to check the
relative values instead. It also updates the test to make it
work with workers instead of directly skipping in workers.

PR-URL: nodejs/node#49197
Refs: nodejs/reliability#638
Reviewed-By: Debadree Chatterjee <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
test: make test-perf-hooks more robust and work with workers
3cee2c0
Previously the test makes several assumptions about the absolute
values of the nodeTiming fields, which can make the test flaky
on slow machines. This patch rewrites the test to check the
relative values instead. It also updates the test to make it
work with workers instead of directly skipping in workers.

PR-URL: nodejs/node#49197
Refs: nodejs/reliability#638
Reviewed-By: Debadree Chatterjee <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aduh95 aduh95 aduh95 approved these changes
+1 more reviewer
@debadree25 debadree25 debadree25 approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. needs-ci PRs that need a full CI run. test Issues and PRs related to Node.js core tests and test infrastructure.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@joyeecheung @nodejs-github-bot @debadree25 @aduh95
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.