GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — fs: remove custom Buffer pool for streams by ronag · Pull Request #33981 · nodejs/node

Conversation

@ronag

@ronag ronag commented Jun 19, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member
The performance benefit of using a custom pool are negligable.
Furthermore, it causes problems with Workers and transferrable.
Rather than further adding complexity for compat with Workers,
just remove the pooling logic.

Refs: #33880 (comment)
Fixes: #31733

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bricss reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@ronag ronag added the stream Issues and PRs related to Node.js streams. label Jun 19, 2020
@ronag
ronag requested a review from addaleax June 19, 2020 22:03
@nodejs-github-bot nodejs-github-bot added fs Issues and PRs related to file-system APIs and the fs module. quic Issues and PRs related to the QUIC transport implementation. labels Jun 19, 2020
@ronag
ronag force-pushed the fs-pool branch 2 times, most recently from cf19ba7 to b58a7bb Compare June 19, 2020 22:04
addaleax
addaleax approved these changes Jun 19, 2020
View reviewed changes
@ronag ronag removed the quic Issues and PRs related to the QUIC transport implementation. label Jun 19, 2020
@ronag

ronag commented Jun 19, 2020

Copy link
Copy Markdown
Member Author

Needs a CI and benchmark CI... However, at the moment I'm unable to start either for some reason. Will try again tomorrow.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ronag ronag mentioned this pull request Jun 20, 2020
fs,crypto: AAD decryption of fs stream > 32768 bytes fails #31733
Closed
@ronag
ronag force-pushed the fs-pool branch from b58a7bb to 4300f74 Compare June 20, 2020 08:18
bnoordhuis
bnoordhuis approved these changes Jun 20, 2020
View reviewed changes

@bnoordhuis bnoordhuis left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also update the comment on line 134 in test-crypto-authenticated-stream.js?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread lib/internal/fs/streams.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

@ronag ronag added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Jun 20, 2020
@ronag
fs: remove custom Buffer pool for streams
2bc17a5
The performance benefit of using a custom pool are negligable.
Furthermore, it causes problems with Workers and transferrable.
Rather than further adding complexity for compat with Workers,
just remove the pooling logic.

Refs: nodejs#33880 (comment)
Fixes: nodejs#31733
@ronag
ronag force-pushed the fs-pool branch from b961333 to 2bc17a5 Compare June 20, 2020 08:41
@ronag
ronag requested a review from jasnell June 20, 2020 08:41
@ronag ronag removed the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Jun 20, 2020
@ronag

ronag commented Jun 20, 2020

Copy link
Copy Markdown
Member Author

Needs a CI and benchmark CI

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@addaleax addaleax added needs-benchmark-ci PRs that need a benchmark CI run. needs-ci PRs that need a full CI run. labels Jun 20, 2020
@ronag

ronag commented Jun 20, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

https://ci.nodejs.org/job/node-test-pull-request/31988/
https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/620/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot

nodejs-github-bot commented Jun 20, 2020

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/31995/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ronag

ronag commented Jun 20, 2020

Copy link
Copy Markdown
Member Author

Slower with a low highwatermark < 4k, which is expected and I think that's acceptable as I don't see that being a common case at all.

19:58:22  fs/read-stream-throughput.js n=1024 highWaterMark=1024 filesize=1024000 encodingType='asc'           ***    -11.00 %       ±3.01%  ±4.01%  ±5.22%
19:58:22  fs/read-stream-throughput.js n=1024 highWaterMark=1024 filesize=1024000 encodingType='buf'           ***    -10.15 %       ±3.46%  ±4.61%  ±6.01%
19:58:22  fs/read-stream-throughput.js n=1024 highWaterMark=1024 filesize=1024000 encodingType='utf'           ***     -6.08 %       ±2.53%  ±3.37%  ±4.40%
19:58:22  fs/read-stream-throughput.js n=1024 highWaterMark=1048576 filesize=1024000 encodingType='asc'                 1.28 %       ±4.14%  ±5.52%  ±7.20%
19:58:22  fs/read-stream-throughput.js n=1024 highWaterMark=1048576 filesize=1024000 encodingType='buf'          *     14.52 %      ±13.09% ±17.42% ±22.67%
19:58:22  fs/read-stream-throughput.js n=1024 highWaterMark=1048576 filesize=1024000 encodingType='utf'                17.52 %      ±20.33% ±27.07% ±35.28%
19:58:22  fs/read-stream-throughput.js n=1024 highWaterMark=4096 filesize=1024000 encodingType='asc'                   -0.39 %       ±2.94%  ±3.91%  ±5.10%
19:58:22  fs/read-stream-throughput.js n=1024 highWaterMark=4096 filesize=1024000 encodingType='buf'             *     -4.47 %       ±3.36%  ±4.47%  ±5.83%
19:58:22  fs/read-stream-throughput.js n=1024 highWaterMark=4096 filesize=1024000 encodingType='utf'                   -1.49 %       ±2.29%  ±3.05%  ±3.97%
19:58:22  fs/read-stream-throughput.js n=1024 highWaterMark=65535 filesize=1024000 encodingType='asc'                  -1.46 %       ±8.08% ±10.75% ±13.99%
19:58:22  fs/read-stream-throughput.js n=1024 highWaterMark=65535 filesize=1024000 encodingType='buf'                  -0.84 %       ±7.46%  ±9.94% ±12.96%
19:58:22  fs/read-stream-throughput.js n=1024 highWaterMark=65535 filesize=1024000 encodingType='utf'                  -0.54 %       ±6.16%  ±8.19%
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ronag
fixup
ca1d829
@nodejs-github-bot

nodejs-github-bot commented Jun 21, 2020

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/32000/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ronag

ronag commented Jun 21, 2020

Copy link
Copy Markdown
Member Author

@nodejs/crypto: Got problems with two crypto failures https://ci.nodejs.org/job/node-test-commit-linux-containered/20895/ Any ideas?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bnoordhuis bnoordhuis mentioned this pull request Jun 22, 2020
Upgrade openssl to 1.1.1g on ubuntu1804_sharedlibs_openssl111_x64 nodejs/build#2363
Closed
@bnoordhuis

bnoordhuis commented Jun 22, 2020

Copy link
Copy Markdown
Member

@ronag I've opened nodejs/build#2363 for the setAAD issue. Didn't the crypto check fix the other test failure?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ronag

ronag commented Jun 22, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Didn't the crypto check fix the other test failure?

Doesn't seem so. Which confuses me. I'll try another CI.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot

nodejs-github-bot commented Jun 22, 2020

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/32016/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ronag ronag removed needs-benchmark-ci PRs that need a benchmark CI run. needs-ci PRs that need a full CI run. labels Jun 22, 2020
@ronag

ronag commented Jun 22, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

@bnoordhuis: Still failing... throw new ERR_NO_CRYPTO(); https://ci.nodejs.org/job/node-test-commit-linux-containered/20915/nodes=ubuntu1804_sharedlibs_withoutssl_x64/testReport/junit/(root)/test/parallel_test_crypto_authenticated_stream/

Do I need to do anything further than ca1d829?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

rvagg
rvagg reviewed Jun 23, 2020
View reviewed changes
Comment thread test/parallel/test-crypto-authenticated-stream.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

@ronag
fixup
2facf66
@nodejs-github-bot

nodejs-github-bot commented Jun 23, 2020 •
edited by ronag
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/32042/ ✅

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ronag ronag added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Jun 23, 2020
@ronag

ronag commented Jun 23, 2020

Copy link
Copy Markdown
Member Author

Landed in db0e991

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ronag ronag closed this Jun 23, 2020
ronag added a commit that referenced this pull request Jun 23, 2020
@ronag
fs: remove custom Buffer pool for streams
db0e991
The performance benefit of using a custom pool are negligable.
Furthermore, it causes problems with Workers and transferrable.
Rather than further adding complexity for compat with Workers,
just remove the pooling logic.

Refs: #33880 (comment)
Fixes: #31733

PR-URL: #33981
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
@addaleax addaleax mentioned this pull request Jun 27, 2020
tools: update ESLint from 7.2.0 to 7.3.0 #34000
Closed
2 tasks done
@codebytere

codebytere commented Jun 27, 2020

Copy link
Copy Markdown
Member

@ronag could you please backport this to v14.x? It conflicted in a handful of places and I didn't want to try my luck 😅

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@codebytere codebytere added the backport-requested-v14.x label Jun 27, 2020
@Zclhlmgqzc Zclhlmgqzc mentioned this pull request Oct 14, 2020
[Bug report] fs.createReadStream set highWaterMark, the same file get different result #35465
Closed
@gireeshpunathil gireeshpunathil mentioned this pull request Nov 4, 2020
worker: motivating examples? #33880
Closed
@targos targos added backport-open-v14.x and removed author ready PRs with CI started, the required approvals, and no outstanding review comments. backport-requested-v14.x labels Apr 25, 2021
targos pushed a commit to targos/node that referenced this pull request Apr 25, 2021
@ronag @targos
fs: remove custom Buffer pool for streams
0211c7a
The performance benefit of using a custom pool are negligable.
Furthermore, it causes problems with Workers and transferrable.
Rather than further adding complexity for compat with Workers,
just remove the pooling logic.

Refs: nodejs#33880 (comment)
Fixes: nodejs#31733

PR-URL: nodejs#33981
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
@targos targos mentioned this pull request Apr 25, 2021
[v14.x] fs: remove custom Buffer pool for streams #38397
Closed
targos pushed a commit that referenced this pull request Apr 26, 2021
@ronag @targos
fs: remove custom Buffer pool for streams
443cace
The performance benefit of using a custom pool are negligable.
Furthermore, it causes problems with Workers and transferrable.
Rather than further adding complexity for compat with Workers,
just remove the pooling logic.

Refs: #33880 (comment)
Fixes: #31733

PR-URL: #33981
Backport-PR-URL: #38397
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
@targos targos added backported-to-v14.x and removed backport-open-v14.x labels Apr 26, 2021
@danielleadams danielleadams mentioned this pull request May 3, 2021
v14.17.0 proposal #38507
Merged
@richardlau richardlau mentioned this pull request Apr 6, 2023
test: remove stale entry from known_issues.status #47454
Merged
nodejs-github-bot pushed a commit that referenced this pull request Apr 8, 2023
@richardlau
test: remove stale entry from known_issues.status
5ae2bce
The `test-crypto-authenticated-stream` test was moved out of
`test/known_issues` and now lives in `test/parallel`

PR-URL: #47454
Refs: #33981
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
RafaelGSS pushed a commit that referenced this pull request Apr 13, 2023
@richardlau @RafaelGSS
test: remove stale entry from known_issues.status
86f8905
The `test-crypto-authenticated-stream` test was moved out of
`test/known_issues` and now lives in `test/parallel`

PR-URL: #47454
Refs: #33981
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
danielleadams pushed a commit that referenced this pull request Jul 6, 2023
@richardlau @danielleadams
test: remove stale entry from known_issues.status
dc65124
The `test-crypto-authenticated-stream` test was moved out of
`test/known_issues` and now lives in `test/parallel`

PR-URL: #47454
Refs: #33981
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
MoLow pushed a commit to MoLow/node that referenced this pull request Jul 6, 2023
@richardlau @MoLow
test: remove stale entry from known_issues.status
095ca5c
The `test-crypto-authenticated-stream` test was moved out of
`test/known_issues` and now lives in `test/parallel`

PR-URL: nodejs#47454
Refs: nodejs#33981
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bnoordhuis bnoordhuis bnoordhuis approved these changes
@addaleax addaleax addaleax approved these changes
@jasnell jasnell Awaiting requested review from jasnell
+1 more reviewer
@rvagg rvagg rvagg left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

fs Issues and PRs related to file-system APIs and the fs module. stream Issues and PRs related to Node.js streams.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

fs,crypto: AAD decryption of fs stream > 32768 bytes fails

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@ronag @nodejs-github-bot @bnoordhuis @codebytere @rvagg @addaleax @targos
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.