GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — fs: improve error performance for `fsyncSync` by pluris · Pull Request #49880 · nodejs/node

Conversation

@pluris

@pluris pluris commented Sep 26, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor
                                                  confidence improvement accuracy (*)   (**)   (***)
fs/bench-fsyncSync.js n=10000 type='existing'            ***      3.37 %       ±1.49% ±1.99%  ±2.60%
fs/bench-fsyncSync.js n=10000 type='non-existing'        ***    116.95 %       ±6.64% ±8.87% ±11.64%

Be aware that when doing many comparisons the risk of a false-positive result increases.
In this case, there are 2 comparisons, you can thus expect the following amount of false-positive results:
  0.10 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.02 false positives, when considering a   1% risk acceptance (**, ***),
  0.00 false positives, when considering a 0.1% risk acceptance (***)

Refs: nodejs/performance#106

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bricss reacted with rocket emoji
All reactions
  • 🚀 1 reaction
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. labels Sep 26, 2023
@pluris pluris changed the title fs: improve error preformance fori fsyncSync fs: improve error performance for fsyncSync Sep 26, 2023
@pluris
pluris force-pushed the perf/fsync branch from e6a80ee to fb0eeb8 Compare September 26, 2023 15:35
@anonrig anonrig mentioned this pull request Sep 26, 2023
Performance of node:fs nodejs/performance#106
Open
anonrig
anonrig reviewed Sep 26, 2023
View reviewed changes
Comment thread benchmark/fs/bench-fsyncSync.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread benchmark/fs/bench-fsyncSync.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread benchmark/fs/bench-fsyncSync.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/node_file.cc Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/internal/fs/sync.js Outdated

@anonrig anonrig Sep 26, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: If you moved this function to c++, it would improve the happy path (existing file)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

pluris reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@pluris pluris Sep 26, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good opinion!
I think you just need to add CHECK_GE(fd,0) to C++.
Do you have a better idea?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@anonrig anonrig Sep 26, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to replicate the behavior of getValidatedFd in C++ and throw the correct error.

const getValidatedFd = hideStackFrames((fd, propName = 'fd') => {
  if (ObjectIs(fd, -0)) {
    return 0;
  }

  validateInt32(fd, propName, 0);

  return fd;
});

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@pluris pluris Sep 27, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the implementation in C++.
I'm not sure if this implementation is correct.
also I updated the benchmark.
When I check it locally, the performance is strange.

Please review again.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/node_file.cc Resolved

Uh oh!

There was an error while loading. Please reload this page.

@anonrig anonrig added the performance Issues and PRs related to the performance of Node.js. label Sep 26, 2023
@pluris
pluris force-pushed the perf/fsync branch from 05cce21 to e0702cf Compare September 26, 2023 16:15
anonrig
anonrig reviewed Sep 26, 2023
View reviewed changes
Comment thread typings/internalBinding/fs.d.ts Resolved

Uh oh!

There was an error while loading. Please reload this page.

@pluris
pluris force-pushed the perf/fsync branch 2 times, most recently from e1ebe02 to c710f94 Compare September 27, 2023 02:43
@pluris
pluris requested a review from anonrig September 27, 2023 07:57
@pluris
pluris force-pushed the perf/fsync branch 3 times, most recently from 764386f to ad76d06 Compare September 27, 2023 08:09
anonrig
anonrig reviewed Sep 27, 2023
View reviewed changes
Comment thread src/node_file.cc Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/node_file.cc Resolved

Uh oh!

There was an error while loading. Please reload this page.

@anonrig anonrig added the commit-queue-squash PRs the Commit Queue should land as one squashed commit. label Sep 27, 2023
@pluris
pluris force-pushed the perf/fsync branch from 09b5de3 to e769fc5 Compare September 27, 2023 14:17
anonrig
anonrig approved these changes Sep 27, 2023
View reviewed changes
anonrig
anonrig approved these changes Sep 27, 2023
View reviewed changes
@anonrig anonrig added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 27, 2023
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 27, 2023
@nodejs-github-bot

nodejs-github-bot commented Sep 27, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54293/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Sep 27, 2023
fs: improve error performance for fdatasyncSync #49898
Merged
src: add GetValidatedFd() to node_file #49903
Closed
joyeecheung
joyeecheung reviewed Sep 27, 2023
View reviewed changes

@joyeecheung joyeecheung left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the diff can be a lot simpler if you just change the sync branch of the original implementations instead of repeating the code in a new binding..(and if you only introduce new bindings, the original sync branch would be dead code..) also I think this breaks --trace-sync-io?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
This was referenced Sep 28, 2023
CI Reliability 2023-09-28 nodejs/reliability#677
Open
CI Reliability 2023-09-30 nodejs/reliability#679
Open
@anonrig

anonrig commented Sep 30, 2023

Copy link
Copy Markdown
Member

Hey @pluris can you rebase this pull request? You'll see that sync.js file is now removed and merged into lib/fs.js

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@anonrig
anonrig requested a review from tniessen October 1, 2023 16:50
joyeecheung
joyeecheung reviewed Oct 2, 2023
View reviewed changes
Comment thread src/node_file.cc Outdated

inline int GetValidatedFd(Environment* env, Local<Value> value) {
if (!value->IsInt32()) {
env->isolate()->ThrowException(ERR_INVALID_ARG_TYPE(

@joyeecheung joyeecheung Oct 2, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be a ValidateInt32 if we want to start moving the validation to C++. Also this is not entirely on-par with the original implementation - the original implementation would print the value in a readable format in the case it's not a number, and this does not give any information about the invalid argument, which would make it harder for users to fix the error. We could also just don't move the error validation code now and leave it in JS if it's not ready to take on implementing proper argument validation + printing in C++.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@pluris pluris Oct 2, 2023

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joyeecheung Thank you for your comment.
You are right. As you said, I think we need to think more about the implementation of GetValidateFd(). First, I will remove this addition to C++ and change it to the original implementation.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
pluris
src: remove GetValidatedFd
fcffde8
@pluris
pluris force-pushed the perf/fsync branch 2 times, most recently from cfd9354 to fcffde8 Compare October 2, 2023 14:01
@pluris
pluris requested a review from joyeecheung October 2, 2023 14:11
joyeecheung
joyeecheung approved these changes Oct 4, 2023
View reviewed changes
Comment thread typings/internalBinding/fs.d.ts Resolved

Uh oh!

There was an error while loading. Please reload this page.

@anonrig anonrig added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 4, 2023
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 4, 2023
@nodejs-github-bot

nodejs-github-bot commented Oct 4, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54530/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Jungku Lee @joyeecheung
Update typings/internalBinding/fs.d.ts
6396e80
Co-authored-by: Joyee Cheung <[email protected]>
@anonrig anonrig added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 4, 2023
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 4, 2023
@nodejs-github-bot

nodejs-github-bot commented Oct 4, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54533/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot

nodejs-github-bot commented Oct 9, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54637/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Oct 10, 2023
CI Reliability 2023-10-10 nodejs/reliability#689
Open
CI Reliability 2023-10-11 nodejs/reliability#690
Open
anonrig
anonrig approved these changes Oct 12, 2023
View reviewed changes
@nodejs-github-bot

nodejs-github-bot commented Oct 12, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54732/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@anonrig anonrig added the commit-queue PRs queued for automated landing through the Commit Queue. label Oct 20, 2023
@nodejs-github-bot nodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Oct 20, 2023
@nodejs-github-bot
nodejs-github-bot merged commit fbd08ec into nodejs:main Oct 20, 2023
@nodejs-github-bot

nodejs-github-bot commented Oct 20, 2023

Copy link
Copy Markdown
Collaborator

Landed in fbd08ec

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

targos pushed a commit that referenced this pull request Oct 23, 2023
Jungku Lee @targos
fs: improve error performance for fsyncSync
5942edb
PR-URL: #49880
Refs: nodejs/performance#106
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
@targos targos mentioned this pull request Oct 23, 2023
v21.1.0 release proposal #50335
Merged
@pluris
pluris deleted the perf/fsync branch November 8, 2023 02:24
targos pushed a commit that referenced this pull request Nov 11, 2023
Jungku Lee @targos
fs: improve error performance for fsyncSync
eb33f70
PR-URL: #49880
Refs: nodejs/performance#106
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
@targos targos mentioned this pull request Nov 12, 2023
v20.10.0 release proposal #50682
Merged
@juanarbol juanarbol mentioned this pull request Dec 6, 2023
2023-12-06, N|Solid v5.0.1 Iron Release nodesource/nsolid#41
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@anonrig anonrig anonrig approved these changes
@joyeecheung joyeecheung joyeecheung approved these changes
@bnoordhuis bnoordhuis Awaiting requested review from bnoordhuis
@santigimeno santigimeno Awaiting requested review from santigimeno
@tniessen tniessen Awaiting requested review from tniessen

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. commit-queue-squash PRs the Commit Queue should land as one squashed commit. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. performance Issues and PRs related to the performance of Node.js.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@pluris @nodejs-github-bot @anonrig @joyeecheung
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.