GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — src: implement structuredClone in native by joyeecheung · Pull Request #50330 · nodejs/node

Conversation

@joyeecheung

@joyeecheung joyeecheung commented Oct 22, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Simplify the implementation by implementing it directly in C++. This improves performance and also makes structuredClone supported in custom snapshots.

Local benchmark numbers:

                                                    confidence improvement accuracy (*)   (**)  (***)
misc/structured-clone.js n=10000 type='arraybuffer'        ***     12.79 %       ±1.72% ±2.29% ±2.99%
misc/structured-clone.js n=10000 type='object'             ***      7.65 %       ±2.22% ±2.96% ±3.85%
misc/structured-clone.js n=10000 type='string'             ***     75.01 %       ±3.84% ±5.14% ±6.75%

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
bricss, JMTK, KhafraDev, H4ad, GeoffreyBooth, acutmore, lin72h, robpalme, tonivj5, dnalborczyk, and alpharder reacted with heart emoji
All reactions
  • ❤️ 11 reactions
@nodejs-github-bot

nodejs-github-bot commented Oct 22, 2023

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/crypto
  • @nodejs/net
  • @nodejs/startup
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run. labels Oct 22, 2023
@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 22, 2023
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 22, 2023
@nodejs-github-bot

nodejs-github-bot commented Oct 22, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55129/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@KhafraDev

KhafraDev commented Oct 22, 2023

Copy link
Copy Markdown
Member

will this fix #49940 too?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
mcollina
mcollina approved these changes Oct 22, 2023
View reviewed changes

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Am I correct in assuming this will improve all messaging through worker threads?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
anonrig reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@benjamingr

benjamingr commented Oct 23, 2023

Copy link
Copy Markdown
Member

@mcollina no, if you look at the changes this instead of creating a MessageChannel object in JS land creates it in C++ land - basically moving that part from JS to C++, see https://github.com/nodejs/node/pull/50330/files#diff-512ab8348327408c9a5ba3f7c60d8f8977a672a22b96345b8cfc2562936062f5R1582-R1583
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
benjamingr
benjamingr approved these changes Oct 23, 2023
View reviewed changes
legendecas
legendecas reviewed Oct 24, 2023
View reviewed changes
Comment thread src/node_messaging.cc Resolved

Uh oh!

There was an error while loading. Please reload this page.
@joyeecheung
joyeecheung force-pushed the native-structured-clone branch from d04776a to 81ff9d9 Compare October 24, 2023 10:13
@joyeecheung

joyeecheung commented Oct 24, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

FYI this is not yet ready because I want to see if we can just avoid the message ports all together without doing a lot of refactoring (there are a lot of code in place to check that the port isn't in the arguments, which then rely on having a port instance to check on) - if not, I'll just try getting this in first before following up with a PR to get rid of the message ports
JMTK, mcollina, and benjamingr reacted with heart emoji
All reactions
  • ❤️ 3 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@joyeecheung
joyeecheung force-pushed the native-structured-clone branch from 81ff9d9 to d7538ee Compare October 24, 2023 12:43
@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 24, 2023
@joyeecheung

joyeecheung commented Oct 24, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

From what I can tell simply using the existing Message::Serialize/Message::Deserialize should yield equivalent results. This should also get rid of the previous peculiar behavior of emitting deserialization errors as messageerror events instead of throwing them directly.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 24, 2023
@nodejs-github-bot

nodejs-github-bot commented Oct 24, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55182/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@joyeecheung

joyeecheung commented Oct 24, 2023

Copy link
Copy Markdown
Member Author

will this fix #49940 too?

I don't think so, the assertion comes from the ReadIterable implementation and that's still unfixed - on the other hand it might be better to just implement that method in JS land and do a glue of structuredClone with that method. This calls into JS so many times that doing it in C++ would be slower. I can do that as a follow-up.
KhafraDev reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@joyeecheung
joyeecheung marked this pull request as ready for review October 24, 2023 13:24
@joyeecheung

joyeecheung commented Oct 24, 2023

Copy link
Copy Markdown
Member Author

I removed the messaging namespace rename (this should still be done in another PR though) and the BindingData because it now no longer relies on MessagePorts. The benchmark numbers are improved a bit too due to the removal of unnecessary hoops:

                                                    confidence improvement accuracy (*)   (**)  (***)
misc/structured-clone.js n=10000 type='arraybuffer'        ***     12.79 %       ±1.72% ±2.29% ±2.99%
misc/structured-clone.js n=10000 type='object'             ***      7.65 %       ±2.22% ±2.96% ±3.85%
misc/structured-clone.js n=10000 type='string'             ***     75.01 %       ±3.84% ±5.14% ±6.75%

This should be ready for review now - I'd appreciate another pair of eyeballs to confirm that this still yields equivalent results.
JMTK, benjamingr, and bricss reacted with heart emoji
All reactions
  • ❤️ 3 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@joyeecheung
joyeecheung force-pushed the native-structured-clone branch from d7538ee to 7d8ecdd Compare October 24, 2023 13:30
@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 24, 2023
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 24, 2023
@nodejs-github-bot

nodejs-github-bot commented Oct 24, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55186/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@joyeecheung
src: implement structuredClone in native
b69261c 
Simplify the implementation by implementing it directly in C++.
This improves performance and also makes structuredClone supported
in custom snapshots.
@joyeecheung
joyeecheung force-pushed the native-structured-clone branch from 7d8ecdd to b69261c Compare October 24, 2023 17:53
aduh95
aduh95 reviewed Oct 24, 2023
View reviewed changes

@aduh95 aduh95 left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please update the message in the linter error:

node/lib/.eslintrc.yaml

Lines 174 to 175 in fd21429

- name: structuredClone
message: Use `const { structuredClone } = require('internal/structured_clone');` instead of the global.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
benjamingr
benjamingr approved these changes Oct 24, 2023
View reviewed changes
@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 25, 2023
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 25, 2023
@nodejs-github-bot

nodejs-github-bot commented Oct 25, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55224/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
aduh95
aduh95 approved these changes Oct 25, 2023
View reviewed changes
@nodejs-github-bot

nodejs-github-bot commented Oct 25, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55232/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
daeyeon
daeyeon approved these changes Oct 25, 2023
View reviewed changes
@joyeecheung joyeecheung added commit-queue PRs queued for automated landing through the Commit Queue. commit-queue-squash PRs the Commit Queue should land as one squashed commit. labels Oct 25, 2023
@nodejs-github-bot nodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Oct 25, 2023
@nodejs-github-bot
nodejs-github-bot merged commit c3a41d8 into nodejs:main Oct 25, 2023
@nodejs-github-bot

nodejs-github-bot commented Oct 25, 2023

Copy link
Copy Markdown
Collaborator

Landed in c3a41d8
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@joyeecheung joyeecheung mentioned this pull request Oct 26, 2023
Improve performance of structuredClone #50320
Closed
This was referenced Oct 26, 2023
CI Reliability 2023-10-26 nodejs/reliability#698
Open
CI Reliability 2023-10-27 nodejs/reliability#699
Open
CI Reliability 2023-10-28 nodejs/reliability#700
Open
targos pushed a commit that referenced this pull request Nov 11, 2023
@joyeecheung @targos
src: implement structuredClone in native
5b453d4 
Simplify the implementation by implementing it directly in C++.
This improves performance and also makes structuredClone supported
in custom snapshots.

PR-URL: #50330
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Daeyeon Jeong <[email protected]>
@targos targos mentioned this pull request Nov 12, 2023
v21.2.0 release proposal #50681
Merged
UlisesGascon pushed a commit that referenced this pull request Dec 11, 2023
@joyeecheung @UlisesGascon
src: implement structuredClone in native
4ef1d68 
Simplify the implementation by implementing it directly in C++.
This improves performance and also makes structuredClone supported
in custom snapshots.

PR-URL: #50330
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Daeyeon Jeong <[email protected]>
@UlisesGascon UlisesGascon mentioned this pull request Dec 12, 2023
v20.11.0 proposal #51124
Merged
@jazelly jazelly mentioned this pull request Oct 8, 2024
lib: convert transfer sequence to array in js #55317
Merged
nodejs-github-bot pushed a commit that referenced this pull request Oct 13, 2024
@jazelly
lib: convert transfer sequence to array in js
50b4ada 
This commit lets `tranfer` passed to `structuredClone` get validated at
JS layer by doing webidl conversion. This avoids the C++ to JS function
call overhead in the native implementaiton of `structuredClone`

PR-URL: #55317
Fixes: #55280
Refs: #50330
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matthew Aitken <[email protected]>
aduh95 pushed a commit that referenced this pull request Oct 19, 2024
@jazelly @aduh95
lib: convert transfer sequence to array in js
07ad987 
This commit lets `tranfer` passed to `structuredClone` get validated at
JS layer by doing webidl conversion. This avoids the C++ to JS function
call overhead in the native implementaiton of `structuredClone`

PR-URL: #55317
Fixes: #55280
Refs: #50330
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matthew Aitken <[email protected]>
jazelly added a commit to jazelly/node that referenced this pull request Nov 27, 2024
@jazelly
lib: convert transfer sequence to array in js
8901e1e 
This commit lets `tranfer` passed to `structuredClone` get validated at
JS layer by doing webidl conversion. This avoids the C++ to JS function
call overhead in the native implementaiton of `structuredClone`

PR-URL: nodejs#55317
Fixes: nodejs#55280
Refs: nodejs#50330
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matthew Aitken <[email protected]>
@jazelly jazelly mentioned this pull request Nov 27, 2024
[v22.x backport] lib: convert transfer sequence to array in js #56015
Closed
ruyadorno pushed a commit that referenced this pull request Nov 27, 2024
@jazelly @ruyadorno
lib: convert transfer sequence to array in js
d48c5da 
This commit lets `tranfer` passed to `structuredClone` get validated at
JS layer by doing webidl conversion. This avoids the C++ to JS function
call overhead in the native implementaiton of `structuredClone`

PR-URL: #55317
Fixes: #55280
Refs: #50330
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matthew Aitken <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@legendecas legendecas legendecas approved these changes
@mcollina mcollina mcollina approved these changes
@benjamingr benjamingr benjamingr approved these changes
@daeyeon daeyeon daeyeon approved these changes
@aduh95 aduh95 aduh95 approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. commit-queue-squash PRs the Commit Queue should land as one squashed commit. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

8 participants

@joyeecheung @nodejs-github-bot @KhafraDev @benjamingr @mcollina @daeyeon @legendecas @aduh95
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.