GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — encoding: use AliasedUint32Array for encodeInto results by joyeecheung · Pull Request #46658 · nodejs/node

Conversation

@joyeecheung

@joyeecheung joyeecheung commented Feb 14, 2023

Copy link
Copy Markdown
Member

The first commits comes from #46620

benchmark: stablize encode benchmark

  • Increase the number of iteration to 1e6 to reduce flakes. 1e4
    can introduce flakes even when comparing the main branch
    against itself
  • Replace the 1024 * 32 length test with 1024 * 8 since it would
    otherwise take too long to complete.
  • Check the results of the encoding methods at the end.

src: move encoding bindings to a new binding

Move the bindings used by TextEncoder to a new binding for
more self-contained code.

encoding: use AliasedUint32Array for encodeInto results

Getting the buffer from a TypedArray created from the JS land
incurs a copy. For encodeInto() results we can just use an
AliasedArray and let the binding always own the store.

                                                                               confidence improvement accuracy (*)   (**)  (***)
util/text-encoder.js op='encodeInto' type='ascii' n=1000000 len=1024                           0.41 %       ±1.33% ±1.78% ±2.33%
util/text-encoder.js op='encodeInto' type='ascii' n=1000000 len=16                    ***      3.66 %       ±1.35% ±1.79% ±2.33%
util/text-encoder.js op='encodeInto' type='ascii' n=1000000 len=256                            0.29 %       ±0.88% ±1.17% ±1.53%
util/text-encoder.js op='encodeInto' type='ascii' n=1000000 len=32                    ***      4.01 %       ±1.34% ±1.79% ±2.34%
util/text-encoder.js op='encodeInto' type='ascii' n=1000000 len=8192                          -0.32 %       ±0.66% ±0.88% ±1.15%
util/text-encoder.js op='encodeInto' type='one-byte-string' n=1000000 len=1024          *     -0.48 %       ±0.37% ±0.50% ±0.65%
util/text-encoder.js op='encodeInto' type='one-byte-string' n=1000000 len=16          ***      5.39 %       ±2.49% ±3.34% ±4.41%
util/text-encoder.js op='encodeInto' type='one-byte-string' n=1000000 len=256         ***      1.44 %       ±0.48% ±0.63% ±0.83%
util/text-encoder.js op='encodeInto' type='one-byte-string' n=1000000 len=32          ***      4.67 %       ±1.01% ±1.34% ±1.74%
util/text-encoder.js op='encodeInto' type='one-byte-string' n=1000000 len=8192                 0.15 %       ±0.24% ±0.32% ±0.42%
util/text-encoder.js op='encodeInto' type='two-byte-string' n=1000000 len=1024        ***      1.29 %       ±0.49% ±0.66% ±0.86%
util/text-encoder.js op='encodeInto' type='two-byte-string' n=1000000 len=16                   2.23 %       ±2.90% ±3.90% ±5.16%
util/text-encoder.js op='encodeInto' type='two-byte-string' n=1000000 len=256         ***      1.10 %       ±0.48% ±0.64% ±0.84%
util/text-encoder.js op='encodeInto' type='two-byte-string' n=1000000 len=32          ***      6.31 %       ±1.60% ±2.14% ±2.82%
util/text-encoder.js op='encodeInto' type='two-byte-string' n=1000000 len=8192                 0.19 %       ±0.21% ±0.28% ±0.37%

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bricss and mcollina reacted with thumbs up emoji
All reactions
  • 👍 2 reactions
@nodejs-github-bot

nodejs-github-bot commented Feb 14, 2023

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/gyp
  • @nodejs/http
  • @nodejs/http2
  • @nodejs/net
  • @nodejs/startup
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run. labels Feb 14, 2023
@anonrig anonrig added the performance Issues and PRs related to the performance of Node.js. label Feb 14, 2023
anonrig
anonrig reviewed Feb 14, 2023
View reviewed changes
Comment thread src/encoding.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/node_binding.cc Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/internal/encoding.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/env-inl.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

BridgeAR
BridgeAR reviewed Feb 14, 2023
View reviewed changes
Comment thread benchmark/util/text-encoder.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung
joyeecheung force-pushed the encode branch from 535e1bf to 86b9a0b Compare February 15, 2023 01:40
@nodejs-github-bot

nodejs-github-bot commented Feb 20, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/49791/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Feb 21, 2023
CI Reliability 2023-02-21 nodejs/reliability#502
Open
CI Reliability 2023-02-22 nodejs/reliability#503
Open
@joyeecheung
benchmark: stablize encode benchmark
29ef3b0
- Increase the number of iteration to 1e6 to reduce flakes. 1e4
  can introduce flakes even when comparing the main branch
  against itself
- Replace the 1024 * 32 length test with 1024 * 8 since it would
  otherwise take too long to complete. Remove the 16 length test
  since it's not too different from 32.
- Check the results of the encoding methods at the end.
@joyeecheung
joyeecheung force-pushed the encode branch from 86b9a0b to 06584cc Compare February 28, 2023 14:38
joyeecheung added 2 commits February 28, 2023 15:41
@joyeecheung
src: move encoding bindings to a new binding
57bb217
Move the bindings used by TextEncoder to a new binding for
more self-contained code.
@joyeecheung
encoding: use AliasedUint32Array for encodeInto results
9b54d80
Getting the buffer from a TypedArray created from the JS land
incurs a copy. For encodeInto() results we can just use an
AliasedArray and let the binding always own the store.
@joyeecheung
joyeecheung force-pushed the encode branch from 06584cc to 9b54d80 Compare February 28, 2023 14:41
@joyeecheung

joyeecheung commented Feb 28, 2023

Copy link
Copy Markdown
Member Author

Rebased to fix merge conflict

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot

nodejs-github-bot commented Feb 28, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/50104/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Mar 1, 2023
CI Reliability 2023-03-01 nodejs/reliability#510
Open
CI Reliability 2023-03-02 nodejs/reliability#511
Open
CI Reliability 2023-03-03 nodejs/reliability#512
Open
@joyeecheung

joyeecheung commented Mar 3, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

cc @nodejs/buffer @nodejs/cpp-reviewers can I have some reviews please? Thanks.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Mar 4, 2023
CI Reliability 2023-03-04 nodejs/reliability#513
Open
CI Reliability 2023-03-05 nodejs/reliability#514
Open
RaisinTen
RaisinTen approved these changes Mar 6, 2023
View reviewed changes

@RaisinTen RaisinTen left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code change LGTM (% lint fixes) but I'm curious about this part:

Getting the buffer from a TypedArray created from the JS land
incurs a copy.

Where does the copy take place?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@joyeecheung

joyeecheung commented Mar 7, 2023

Copy link
Copy Markdown
Member Author

Where does the copy take place?

v8::ArrayBufferView::Buffer().

RaisinTen reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung
fixup! encoding: use AliasedUint32Array for encodeInto results
1233e7f
@joyeecheung

joyeecheung commented Mar 7, 2023

Copy link
Copy Markdown
Member Author

Fixed linter error (looks like the bot is not working again: https://ci.nodejs.org/job/node-test-pull-request/50249/)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung
fixup! src: move encoding bindings to a new binding
4ea7f2e
@joyeecheung
joyeecheung force-pushed the encode branch from ea70ca8 to 4ea7f2e Compare March 7, 2023 16:59
@nodejs-github-bot

nodejs-github-bot commented Mar 7, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/50258/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

joyeecheung added a commit that referenced this pull request Mar 7, 2023
@joyeecheung
benchmark: stablize encode benchmark
044021d
- Increase the number of iteration to 1e6 to reduce flakes. 1e4
  can introduce flakes even when comparing the main branch
  against itself
- Replace the 1024 * 32 length test with 1024 * 8 since it would
  otherwise take too long to complete. Remove the 16 length test
  since it's not too different from 32.
- Check the results of the encoding methods at the end.

PR-URL: #46658
Reviewed-By: Darshan Sen <[email protected]>
joyeecheung added a commit that referenced this pull request Mar 7, 2023
@joyeecheung
src: move encoding bindings to a new binding
25c2186
Move the bindings used by TextEncoder to a new binding for
more self-contained code.

PR-URL: #46658
Reviewed-By: Darshan Sen <[email protected]>
joyeecheung added a commit that referenced this pull request Mar 7, 2023
@joyeecheung
src: use AliasedUint32Array for encodeInto results
e5933c8
Getting the buffer from a TypedArray created from the JS land
incurs a copy. For encodeInto() results we can just use an
AliasedArray and let the binding always own the store.

PR-URL: #46658
Reviewed-By: Darshan Sen <[email protected]>
@joyeecheung

joyeecheung commented Mar 7, 2023

Copy link
Copy Markdown
Member Author

Landed in 3b0c047...e5933c8

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung joyeecheung closed this Mar 7, 2023
@github-actions github-actions Bot mentioned this pull request Mar 8, 2023
CI Reliability 2023-03-08 nodejs/reliability#517
Open
19 tasks
@MoLow

MoLow commented Mar 8, 2023

Copy link
Copy Markdown
Member

this commit has landed without a passing CI/GH actions, introducing a lint error: #47007

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@cjihrig

cjihrig commented Mar 8, 2023

Copy link
Copy Markdown
Contributor

Linter issue is addressed in #47003.

MoLow reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung

joyeecheung commented Mar 8, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

The CI was actually passing in https://ci.nodejs.org/job/node-test-pull-request/50258/, which included the fixup commit that should remove the unused using statement causing the lint error: 4ea7f2e. But somehow git was only able to apply that fixup commit partially during landing, and I didn't see any warnings?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Mar 9, 2023
CI Reliability 2023-03-09 nodejs/reliability#518
Open
CI Reliability 2023-03-10 nodejs/reliability#519
Open
CI Reliability 2023-03-11 nodejs/reliability#520
Open
CI Reliability 2023-03-12 nodejs/reliability#521
Open
CI Reliability 2023-03-13 nodejs/reliability#522
Open
targos pushed a commit that referenced this pull request Mar 13, 2023
@joyeecheung @targos
benchmark: stablize encode benchmark
d654f46
- Increase the number of iteration to 1e6 to reduce flakes. 1e4
  can introduce flakes even when comparing the main branch
  against itself
- Replace the 1024 * 32 length test with 1024 * 8 since it would
  otherwise take too long to complete. Remove the 16 length test
  since it's not too different from 32.
- Check the results of the encoding methods at the end.

PR-URL: #46658
Reviewed-By: Darshan Sen <[email protected]>
targos pushed a commit that referenced this pull request Mar 13, 2023
@joyeecheung @targos
src: move encoding bindings to a new binding
db4fa93
Move the bindings used by TextEncoder to a new binding for
more self-contained code.

PR-URL: #46658
Reviewed-By: Darshan Sen <[email protected]>
targos pushed a commit that referenced this pull request Mar 13, 2023
@joyeecheung @targos
src: use AliasedUint32Array for encodeInto results
7ec0dcb
Getting the buffer from a TypedArray created from the JS land
incurs a copy. For encodeInto() results we can just use an
AliasedArray and let the binding always own the store.

PR-URL: #46658
Reviewed-By: Darshan Sen <[email protected]>
targos pushed a commit that referenced this pull request Mar 14, 2023
@joyeecheung @targos
benchmark: stablize encode benchmark
0617c5e
- Increase the number of iteration to 1e6 to reduce flakes. 1e4
  can introduce flakes even when comparing the main branch
  against itself
- Replace the 1024 * 32 length test with 1024 * 8 since it would
  otherwise take too long to complete. Remove the 16 length test
  since it's not too different from 32.
- Check the results of the encoding methods at the end.

PR-URL: #46658
Reviewed-By: Darshan Sen <[email protected]>
targos pushed a commit that referenced this pull request Mar 14, 2023
@joyeecheung @targos
src: move encoding bindings to a new binding
dcba3a0
Move the bindings used by TextEncoder to a new binding for
more self-contained code.

PR-URL: #46658
Reviewed-By: Darshan Sen <[email protected]>
targos pushed a commit that referenced this pull request Mar 14, 2023
@joyeecheung @targos
src: use AliasedUint32Array for encodeInto results
6b60f38
Getting the buffer from a TypedArray created from the JS land
incurs a copy. For encodeInto() results we can just use an
AliasedArray and let the binding always own the store.

PR-URL: #46658
Reviewed-By: Darshan Sen <[email protected]>
@targos targos mentioned this pull request Mar 14, 2023
v19.8.0 release proposal #47087
Merged
@danielleadams

danielleadams commented Apr 3, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

@joyeecheung This breaks the build when landing in v18.x - do you mind opening a backport PR?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@danielleadams danielleadams added the backport-requested-v18.x label Apr 3, 2023
debadree25
debadree25 reviewed May 5, 2023
View reviewed changes
Comment thread src/encoding_binding.cc
CHECK_NOT_NULL(binding);
}

void BindingData::EncodeInto(const FunctionCallbackInfo<Value>& args) {

@debadree25 debadree25 May 5, 2023

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity I exploring places where fast API could be applied, came across this function and PR, I experimented a bit it seems v8::FastApiTypedArray<uint8_t>& is indeed supported and it does get caught in fast call, but I am not sure how one could extract ->Buffer() from v8::FastApiTypedArray<uint8_t>&, is that even possible? just wondering do let me know your thoughts cc @anonrig @joyeecheung

Thank You!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
targos pushed a commit that referenced this pull request Nov 10, 2023
@joyeecheung @targos
benchmark: stablize encode benchmark
24d3fcf
- Increase the number of iteration to 1e6 to reduce flakes. 1e4
  can introduce flakes even when comparing the main branch
  against itself
- Replace the 1024 * 32 length test with 1024 * 8 since it would
  otherwise take too long to complete. Remove the 16 length test
  since it's not too different from 32.
- Check the results of the encoding methods at the end.

PR-URL: #46658
Reviewed-By: Darshan Sen <[email protected]>
@targos targos mentioned this pull request Nov 28, 2023
v18.19.0 release proposal #50953
Merged
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
benchmark: stablize encode benchmark
bf4207c
- Increase the number of iteration to 1e6 to reduce flakes. 1e4
  can introduce flakes even when comparing the main branch
  against itself
- Replace the 1024 * 32 length test with 1024 * 8 since it would
  otherwise take too long to complete. Remove the 16 length test
  since it's not too different from 32.
- Check the results of the encoding methods at the end.

PR-URL: nodejs/node#46658
Reviewed-By: Darshan Sen <[email protected]>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
benchmark: stablize encode benchmark
710b0ac
- Increase the number of iteration to 1e6 to reduce flakes. 1e4
  can introduce flakes even when comparing the main branch
  against itself
- Replace the 1024 * 32 length test with 1024 * 8 since it would
  otherwise take too long to complete. Remove the 16 length test
  since it's not too different from 32.
- Check the results of the encoding methods at the end.

PR-URL: nodejs/node#46658
Reviewed-By: Darshan Sen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@anonrig anonrig anonrig left review comments
@BridgeAR BridgeAR BridgeAR left review comments
@RaisinTen RaisinTen RaisinTen approved these changes
+1 more reviewer
@debadree25 debadree25 debadree25 left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run. performance Issues and PRs related to the performance of Node.js.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

9 participants

@joyeecheung @nodejs-github-bot @MoLow @cjihrig @danielleadams @anonrig @BridgeAR @debadree25 @RaisinTen
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.