GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — [v18.x] backport various patches of internal refactorings for snapshot and realms by joyeecheung · Pull Request #45007 · nodejs/node

Conversation

@joyeecheung

@joyeecheung joyeecheung commented Oct 14, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This backports various refactoring patches for snapshot and shadow realms support to smooth out the backports of other patches. AFAICT only the shadow realms support patch changes the behavior a bit (but only if users uses the --experimental-shadow-realms flag and we don't have any compatibility guarantee for that anyway).

src: add initial shadow realm support

Add initial shadow realm support behind an off-by-default flag
--experimental-shadow-realm.

PR-URL: #42869
Refs: #42528
Reviewed-By: Antoine du Hamel [email protected]
Reviewed-By: Darshan Sen [email protected]

src: per-environment time origin value

According to https://html.spec.whatwg.org/#environment-settings-object,
the timeOrigin is a per-environment value. Worker's timeOrigin is the
time when the worker is created.

PR-URL: #43781
Reviewed-By: Matteo Collina [email protected]
Reviewed-By: Anna Henningsen [email protected]
Reviewed-By: Paolo Insogna [email protected]
Reviewed-By: Joyee Cheung [email protected]

src: fix cppgc incompatibility in v8

This patch updates the layout of the BaseObjects to make sure
that the first embedder field of them is a "type" pointer, the
first 16 bits of which are the Node.js embedder ID, so that
cppgc will always skip over them. In addition we now use this
field to determine if the native object should be interpreted
as a Node.js embedder object in the serialization and deserialization
callbacks for the startup snapshot to improve the reliability.

Co-authored-by: Joyee Cheung [email protected]
PR-URL: #43521
Reviewed-By: James M Snell [email protected]
Reviewed-By: Darshan Sen [email protected]

bootstrap: check more metadata when loading the snapshot

This patch stores the metadata about the Node.js binary
into the SnapshotData and adds fields denoting how the
snapshot was generated, on what platform it was
generated as well as the V8 cached data version flag.
Instead of simply crashing when the metadata doesn't
match, Node.js now prints an error message and exit with
1 for the customized snapshot, or ignore the snapshot
and start from scratch if it's the default one.

PR-URL: #44132
Reviewed-By: Anna Henningsen [email protected]
Reviewed-By: Chengzhong Wu [email protected]

src: iterate over base objects to prepare for snapshot

Instead of iterating over the bindings, iterate over the base
objects that are snapshottable. This allows us to snapshot
base objects that are not bindings. In addition this refactors
the InternalFieldInfo class to eliminate potential undefined
behaviors, and renames it to InternalFieldInfoBase.
The {de}serialize callbacks now expect a InternalFieldInfo struct
nested in Snapshotable classes that can be used to carry
serialization data around. This allows us to create structs
inheriting from InternalFieldInfo for Snapshotable objects
that need custom fields.

PR-URL: #44192
Refs: #37476
Reviewed-By: Anna Henningsen [email protected]
Reviewed-By: Chengzhong Wu [email protected]

src: support WeakReference in snapshot

Move util::WeakReference to a separate header and implement
{de}serialization for it to be snapshotable.

PR-URL: #44193
Refs: #44014
Refs: #37476
Reviewed-By: Matteo Collina [email protected]
Reviewed-By: Chengzhong Wu [email protected]

src: introduce node::Realm

To distinguish per-context values from the node::Environment, split
those values to a new node::Realm structure and consolidate
bootstrapping methods with it.

PR-URL: #44179
Refs: #42528
Reviewed-By: Joyee Cheung [email protected]
Reviewed-By: James M Snell [email protected]

src: create BaseObject with node::Realm

BaseObject is a wrapper around JS objects. These objects should be
created in a node::Realm and destroyed when their associated realm is
cleaning up.

PR-URL: #44348
Refs: #42528
Reviewed-By: Joyee Cheung [email protected]

src: refactor BaseObject methods

  • Wrap the initialization of the kSlot and kEmbedderType fields
    into a BaseObject::SetInternalFields() method.
  • Move the tagging of kEmbedderType field into
    BaseObject::TagNodeObject()
  • Add a variant of BaseObject::MakeLazilyInitializedJSTemplate()
    that only needs IsolateData.
    This makes it easier to create BaseObject subclasses.

PR-URL: #44796
Reviewed-By: Chengzhong Wu [email protected]
Reviewed-By: James M Snell [email protected]

benchmark: add vm context global proxy benchmark

PR-URL: #44796
Reviewed-By: Chengzhong Wu [email protected]
Reviewed-By: James M Snell [email protected]

vm: make ContextifyContext a BaseObject

Instead of adding a reference to the ContextifyContext by using
a v8::External, we make ContextifyContext a weak BaseObject that
whose wrapper is referenced by the sandbox via a private symbol.
This makes it easier to snapshot the contexts, in addition to
reusing the BaseObject lifetime management for ContextifyContexts.

PR-URL: #44796
Reviewed-By: Chengzhong Wu [email protected]
Reviewed-By: James M Snell [email protected]

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
legendecas and others added 7 commits October 14, 2022 16:08
@legendecas @joyeecheung
src: add initial shadow realm support
c13a4bc
Add initial shadow realm support behind an off-by-default flag
`--experimental-shadow-realm`.

PR-URL: nodejs#42869
Refs: nodejs#42528
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Darshan Sen <[email protected]>
@legendecas @joyeecheung
src: per-environment time origin value
99e2a11
According to https://html.spec.whatwg.org/#environment-settings-object,
the timeOrigin is a per-environment value. Worker's timeOrigin is the
time when the worker is created.

PR-URL: nodejs#43781
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Paolo Insogna <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
@codebytere @joyeecheung
src: fix cppgc incompatibility in v8
4ce5f39
This patch updates the layout of the BaseObjects to make sure
that the first embedder field of them is a "type" pointer, the
first 16 bits of which are the Node.js embedder ID, so that
cppgc will always skip over them. In addition we now use this
field to determine if the native object should be interpreted
as a Node.js embedder object in the serialization and deserialization
callbacks for the startup snapshot to improve the reliability.

Co-authored-by: Joyee Cheung <[email protected]>
PR-URL: nodejs#43521
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Darshan Sen <[email protected]>
@joyeecheung
bootstrap: check more metadata when loading the snapshot
7847629
This patch stores the metadata about the Node.js binary
into the SnapshotData and adds fields denoting how the
snapshot was generated, on what platform it was
generated as well as the V8 cached data version flag.
Instead of simply crashing when the metadata doesn't
match, Node.js now prints an error message and exit with
1 for the customized snapshot, or ignore the snapshot
and start from scratch if it's the default one.

PR-URL: nodejs#44132
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
@joyeecheung
src: iterate over base objects to prepare for snapshot
5144634
Instead of iterating over the bindings, iterate over the base
objects that are snapshottable. This allows us to snapshot
base objects that are not bindings. In addition this refactors
the InternalFieldInfo class to eliminate potential undefined
behaviors, and renames it to InternalFieldInfoBase.
The {de}serialize callbacks now expect a InternalFieldInfo struct
nested in Snapshotable classes that can be used to carry
serialization data around. This allows us to create structs
inheriting from InternalFieldInfo for Snapshotable objects
that need custom fields.

PR-URL: nodejs#44192
Refs: nodejs#37476
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
@joyeecheung
src: support WeakReference in snapshot
4670f7c
Move util::WeakReference to a separate header and implement
{de}serialization for it to be snapshotable.

PR-URL: nodejs#44193
Refs: nodejs#44014
Refs: nodejs#37476
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
@legendecas @joyeecheung
src: introduce node::Realm
4dd707a
To distinguish per-context values from the node::Environment, split
those values to a new node::Realm structure and consolidate
bootstrapping methods with it.

PR-URL: nodejs#44179
Refs: nodejs#42528
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@nodejs-github-bot

nodejs-github-bot commented Oct 14, 2022

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/gyp
  • @nodejs/http2
  • @nodejs/net
  • @nodejs/startup
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run. v18.x labels Oct 14, 2022
legendecas and others added 4 commits October 17, 2022 14:31
@legendecas @joyeecheung
src: create BaseObject with node::Realm
95d5daf
BaseObject is a wrapper around JS objects. These objects should be
created in a node::Realm and destroyed when their associated realm is
cleaning up.

PR-URL: nodejs#44348
Refs: nodejs#42528
Reviewed-By: Joyee Cheung <[email protected]>
@joyeecheung
src: refactor BaseObject methods
7499284
- Wrap the initialization of the kSlot and kEmbedderType fields
  into a BaseObject::SetInternalFields() method.
- Move the tagging of kEmbedderType field into
  BaseObject::TagNodeObject()
- Add a variant of BaseObject::MakeLazilyInitializedJSTemplate()
  that only needs IsolateData.
This makes it easier to create BaseObject subclasses.

PR-URL: nodejs#44796
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@joyeecheung
benchmark: add vm context global proxy benchmark
d2192e6
PR-URL: nodejs#44796
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@joyeecheung
vm: make ContextifyContext a BaseObject
45f23ef
Instead of adding a reference to the ContextifyContext by using
a v8::External, we make ContextifyContext a weak BaseObject that
whose wrapper is referenced by the sandbox via a private symbol.
This makes it easier to snapshot the contexts, in addition to
reusing the BaseObject lifetime management for ContextifyContexts.

PR-URL: nodejs#44796
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@nodejs-github-bot

nodejs-github-bot commented Oct 18, 2022

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/47324/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung

joyeecheung commented Oct 18, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

cc @legendecas @nodejs/releasers

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gengjiawen
gengjiawen approved these changes Oct 18, 2022
View reviewed changes
legendecas
legendecas approved these changes Oct 18, 2022
View reviewed changes

@legendecas legendecas left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing the backports!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@joyeecheung
fixup! src: introduce node::Realm
9f90e65
@joyeecheung

joyeecheung commented Oct 26, 2022

Copy link
Copy Markdown
Member Author

Added back some changes to test/pummel/test-heapdump-env.js that were left out

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot

nodejs-github-bot commented Oct 26, 2022

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/47493/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

mcollina
mcollina approved these changes Oct 26, 2022
View reviewed changes

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
RafaelGSS
RafaelGSS approved these changes Oct 26, 2022
View reviewed changes
@nodejs-github-bot

nodejs-github-bot commented Oct 31, 2022

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/47592/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot mentioned this pull request Nov 1, 2022
CI Reliability 2022-11-01 nodejs/reliability#417
Open
17 tasks
@nodejs-github-bot

nodejs-github-bot commented Nov 1, 2022

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/47619/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung

joyeecheung commented Nov 1, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Landed in 0774b64...c37dc7c

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung joyeecheung closed this Nov 1, 2022
@github-actions github-actions Bot mentioned this pull request Nov 2, 2022
CI Reliability 2022-11-02 nodejs/reliability#418
Open
15 tasks
@joyeecheung joyeecheung mentioned this pull request Nov 2, 2022
Node.js Technical Steering Committee (TSC) Meeting 2022-11-02 nodejs/TSC#1302
Closed
@danielleadams danielleadams mentioned this pull request Jan 2, 2023
v18.13.0 release proposal #46025
Merged
@danielleadams

danielleadams commented Jan 3, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

#43781 was dropped from v18.13.0 (#46025) because it is semver-major change - if its decided that this is a bugfix for v18.x, we can open another backport and drop the semver-major label.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcollina mcollina mcollina approved these changes
@gengjiawen gengjiawen gengjiawen approved these changes
@legendecas legendecas legendecas approved these changes
@RafaelGSS RafaelGSS RafaelGSS approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

8 participants

@joyeecheung @nodejs-github-bot @danielleadams @mcollina @gengjiawen @legendecas @RafaelGSS @codebytere
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.