GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — src: generate snapshot with --predictable by joyeecheung · Pull Request #48749 · nodejs/node

Conversation

@joyeecheung

@joyeecheung joyeecheung commented Jul 12, 2023

Copy link
Copy Markdown
Member

To improve determinism of snapshot generation, add --predictable to the V8 flags used to initialize a process launched to generate snapshot. Also add a kGeneratePredictableSnapshot flag to ProcessInitializationFlags for this and moves the configuration of these flags into node::InitializeOncePerProcess() so that it can be shared by embedders.

Refs: nodejs/build#3043

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bricss reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@nodejs-github-bot

nodejs-github-bot commented Jul 12, 2023

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/startup
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Jul 12, 2023
richardlau
richardlau approved these changes Jul 12, 2023
View reviewed changes
anonrig
anonrig approved these changes Jul 12, 2023
View reviewed changes
Comment thread src/node.cc Resolved

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung joyeecheung added the blocked PRs that are blocked by other issues or PRs. label Jul 12, 2023
@joyeecheung

joyeecheung commented Jul 12, 2023

Copy link
Copy Markdown
Member Author

This needs https://chromium-review.googlesource.com/c/v8/v8/+/4681766

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@targos

targos commented Jul 17, 2023

Copy link
Copy Markdown
Member

https://chromium-review.googlesource.com/c/v8/v8/+/4681766 landed.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung
joyeecheung force-pushed the predictable branch from 04193b0 to 697fef6 Compare July 20, 2023 08:56
@joyeecheung joyeecheung removed the blocked PRs that are blocked by other issues or PRs. label Jul 20, 2023
@joyeecheung

joyeecheung commented Aug 14, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

It turns out all the flags implied by --predictable need to be ignored in the flag hash computation too:

--no-concurrent-recompilation
--no-parallel-scavenge
--no-concurrent-marking
--no-concurrent-array-buffer-sweeping
--no-parallel-marking
--no-concurrent-sweeping
--no-parallel-compaction
--no-parallel-pointer-update
--no-memory-reducer
--single-threaded-gc

Although I now wonder if --predictable is actually needed for our current use case (it seems --predictable currently does not actually introduce additional changes to the snapshot?) This might still be needed down the line though, as V8 does use --predictable for their own snapshot. I may come back and pick this up later when it turns out to be necessary

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung
joyeecheung force-pushed the predictable branch from 697fef6 to 1e6968d Compare August 14, 2023 14:19
@joyeecheung joyeecheung mentioned this pull request Aug 24, 2023
Reproducible builds on Linux nodejs/build#3043
Closed
@joyeecheung

joyeecheung commented Aug 24, 2023

Copy link
Copy Markdown
Member Author

hmm, looks like this is necessary after all or otherwise we get 2 more differences: nodejs/build#3043 (comment)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung
joyeecheung force-pushed the predictable branch from 1e6968d to 232afe3 Compare September 7, 2023 14:57
@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 7, 2023
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 7, 2023
@nodejs-github-bot

nodejs-github-bot commented Sep 7, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/53788/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung
joyeecheung force-pushed the predictable branch from 232afe3 to df94eb0 Compare September 7, 2023 19:08
joyeecheung added a commit to joyeecheung/node that referenced this pull request Sep 18, 2023
@joyeecheung
deps: V8: cherry-pick b33bf2dfd261
8f4e632
Original commit message:

    Ignore --predictable when computing flag list hashes

    This allows reproducible code cache generation.

    Refs: nodejs#48749

    Change-Id: Ib4693de60ddff1fe41d95c10980f763463db3f95
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4681766
    Reviewed-by: Leszek Swirski <[email protected]>
    Commit-Queue: Joyee Cheung <[email protected]>
    Cr-Commit-Position: refs/heads/main@{#88943}

Refs: v8/v8@b33bf2d
@joyeecheung
joyeecheung force-pushed the predictable branch from df94eb0 to 4cfdafb Compare September 18, 2023 13:03
@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 18, 2023
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 18, 2023
@nodejs-github-bot

nodejs-github-bot commented Sep 18, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54037/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Sep 19, 2023
CI Reliability 2023-09-19 nodejs/reliability#670
Open
CI Reliability 2023-09-20 nodejs/reliability#671
Open
CI Reliability 2023-09-21 nodejs/reliability#672
Open
nodejs-github-bot pushed a commit that referenced this pull request Sep 25, 2023
@joyeecheung @nodejs-github-bot
deps: V8: cherry-pick b33bf2dfd261
e074512
Original commit message:

    Ignore --predictable when computing flag list hashes

    This allows reproducible code cache generation.

    Refs: #48749

    Change-Id: Ib4693de60ddff1fe41d95c10980f763463db3f95
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4681766
    Reviewed-by: Leszek Swirski <[email protected]>
    Commit-Queue: Joyee Cheung <[email protected]>
    Cr-Commit-Position: refs/heads/main@{#88943}

Refs: v8/v8@b33bf2d
PR-URL: #49703
Refs: v8/v8@de9a5de
Reviewed-By: Jiawen Geng <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
@joyeecheung
src: generate default snapshot with --predictable
d95be40
To improve determinism of snapshot generation, add --predictable
to the V8 flags used to initialize a process launched to generate
snapshot. Also add a kGeneratePredictableSnapshot flag
to ProcessInitializationFlags for this and moves the configuration
of these flags into node::InitializeOncePerProcess() so that
it can be shared by embedders.
@joyeecheung
joyeecheung force-pushed the predictable branch from 4cfdafb to d95be40 Compare September 26, 2023 17:11
@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 26, 2023
@joyeecheung

joyeecheung commented Sep 26, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Rebased after #49703 is backported and also moved the flag settings into InitializeNodeWithArgsInternal(), together with other V8::SetFlagsFromString calls. We also do not need to set --harmony-import-assertions now because a) that's true by default now and b) we take care of that already in ProcessGlobalArgsInternal()

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 26, 2023
@nodejs-github-bot

nodejs-github-bot commented Sep 26, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54260/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot

nodejs-github-bot commented Sep 27, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54278/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@joyeecheung

joyeecheung commented Sep 27, 2023

Copy link
Copy Markdown
Member Author

@anonrig @richardlau This has been rebased and updated. Can I get some new LGTM please?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot mentioned this pull request Sep 28, 2023
CI Reliability 2023-09-28 nodejs/reliability#677
Open
19 tasks
ruyadorno pushed a commit that referenced this pull request Sep 28, 2023
@joyeecheung @ruyadorno
deps: V8: cherry-pick b33bf2dfd261
1f16df2
Original commit message:

    Ignore --predictable when computing flag list hashes

    This allows reproducible code cache generation.

    Refs: #48749

    Change-Id: Ib4693de60ddff1fe41d95c10980f763463db3f95
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4681766
    Reviewed-by: Leszek Swirski <[email protected]>
    Commit-Queue: Joyee Cheung <[email protected]>
    Cr-Commit-Position: refs/heads/main@{#88943}

Refs: v8/v8@b33bf2d
PR-URL: #49703
Refs: v8/v8@de9a5de
Reviewed-By: Jiawen Geng <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
ruyadorno pushed a commit that referenced this pull request Sep 28, 2023
@joyeecheung @ruyadorno
deps: V8: cherry-pick b33bf2dfd261
cdd1c66
Original commit message:

    Ignore --predictable when computing flag list hashes

    This allows reproducible code cache generation.

    Refs: #48749

    Change-Id: Ib4693de60ddff1fe41d95c10980f763463db3f95
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4681766
    Reviewed-by: Leszek Swirski <[email protected]>
    Commit-Queue: Joyee Cheung <[email protected]>
    Cr-Commit-Position: refs/heads/main@{#88943}

Refs: v8/v8@b33bf2d
PR-URL: #49703
Refs: v8/v8@de9a5de
Reviewed-By: Jiawen Geng <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
richardlau
richardlau approved these changes Oct 6, 2023
View reviewed changes
legendecas
legendecas approved these changes Oct 9, 2023
View reviewed changes
@legendecas legendecas added the commit-queue PRs queued for automated landing through the Commit Queue. label Oct 9, 2023
@nodejs-github-bot nodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Oct 9, 2023
@nodejs-github-bot
nodejs-github-bot merged commit 387e292 into nodejs:main Oct 9, 2023
@nodejs-github-bot

nodejs-github-bot commented Oct 9, 2023

Copy link
Copy Markdown
Collaborator

Landed in 387e292

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@RafaelGSS RafaelGSS mentioned this pull request Oct 9, 2023
2023-10-17, Version 21.0.0 (Current) #49870
Merged
targos pushed a commit that referenced this pull request Nov 11, 2023
@joyeecheung @targos
src: generate default snapshot with --predictable
c99e51e
To improve determinism of snapshot generation, add --predictable
to the V8 flags used to initialize a process launched to generate
snapshot. Also add a kGeneratePredictableSnapshot flag
to ProcessInitializationFlags for this and moves the configuration
of these flags into node::InitializeOncePerProcess() so that
it can be shared by embedders.

PR-URL: #48749
Refs: nodejs/build#3043
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
@targos targos mentioned this pull request Nov 12, 2023
v20.10.0 release proposal #50682
Merged
@juanarbol juanarbol mentioned this pull request Dec 6, 2023
2023-12-06, N|Solid v5.0.1 Iron Release nodesource/nsolid#41
Merged
codebytere added a commit to electron/electron that referenced this pull request Dec 8, 2023
@codebytere
src: generate snapshot with --predictable
96344de
nodejs/node#48749
codebytere added a commit to electron/electron that referenced this pull request Dec 8, 2023
@codebytere
src: generate snapshot with --predictable
53ed3fa
nodejs/node#48749
codebytere added a commit to electron/electron that referenced this pull request Dec 11, 2023
@electron-roller @ckerr @codebytere
chore: bump node to v20.10.0 (main) (#40675)
890a557
* chore: bump node in DEPS to v20.10.0

* chore: update feat_initialize_asar_support.patch

no code changes; patch just needed an update due to nearby upstream changes

Xref: nodejs/node#49986

* chore: update pass_all_globals_through_require.patch

no manual changes; patch applied with fuzz

Xref: nodejs/node#49657

* chore: update refactor_allow_embedder_overriding_of_internal_fs_calls

Xref: nodejs/node#49912

no code changes; patch just needed an update due to nearby upstream changes

* chore: update chore_allow_the_node_entrypoint_to_be_a_builtin_module.patch

Xref: nodejs/node#49986

minor manual changes needed to sync with upstream change

* update fix_expose_the_built-in_electron_module_via_the_esm_loader.patch

Xref: nodejs/node#50096
Xref: nodejs/node#50314
in lib/internal/modules/esm/load.js, update the code that checks for
`format === 'electron'`. I'd like 👀 on this

Xref: nodejs/node#49657
add braces in lib/internal/modules/esm/translators.js to sync with upstream

* fix: lazyload fs in esm loaders to apply asar patches

* nodejs/node#50127
* nodejs/node#50096

* esm: jsdoc for modules code

nodejs/node#49523

* test: set test-cli-node-options as flaky

nodejs/node#50296

* deps: update c-ares to 1.20.1

nodejs/node#50082

* esm: bypass CommonJS loader under --default-type=module

nodejs/node#49986

* deps: update uvwasi to 0.0.19

nodejs/node#49908

* lib,test: do not hardcode Buffer.kMaxLength

nodejs/node#49876

* crypto: account for disabled SharedArrayBuffer

nodejs/node#50034

* test: fix edge snapshot stack traces

nodejs/node#49659

* src: generate snapshot with --predictable

nodejs/node#48749

* chore: fixup patch indices

* fs: throw errors from sync branches instead of separate implementations

nodejs/node#49913

* crypto: ensure valid point on elliptic curve in SubtleCrypto.importKey

nodejs/node#50234

* esm: detect ESM syntax in ambiguous JavaScrip

nodejs/node#50096

* fixup! test: fix edge snapshot stack traces

* esm: unflag extensionless ES module JavaScript and Wasm in module scope

nodejs/node#49974

* [tagged-ptr] Arrowify objects

https://chromium-review.googlesource.com/c/v8/v8/+/4705331

---------

Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com>
Co-authored-by: Charles Kerr <[email protected]>
Co-authored-by: Shelley Vohr <[email protected]>
debadree25 pushed a commit to debadree25/node that referenced this pull request Apr 15, 2024
@joyeecheung @debadree25
src: generate default snapshot with --predictable
135b8f0
To improve determinism of snapshot generation, add --predictable
to the V8 flags used to initialize a process launched to generate
snapshot. Also add a kGeneratePredictableSnapshot flag
to ProcessInitializationFlags for this and moves the configuration
of these flags into node::InitializeOncePerProcess() so that
it can be shared by embedders.

PR-URL: nodejs#48749
Refs: nodejs/build#3043
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bnoordhuis bnoordhuis bnoordhuis left review comments
@anonrig anonrig anonrig approved these changes
@richardlau richardlau richardlau approved these changes
@legendecas legendecas legendecas approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@joyeecheung @nodejs-github-bot @targos @bnoordhuis @anonrig @richardlau @legendecas
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.