GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — lib: load fewer builtins when bootstrapping without a snapshot by codebytere · Pull Request #65329 · nodejs/node Conversation
|
|
Review requested:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@ ## main #65329 +/- ## ========================================== - Coverage 90.13% 90.09% -0.05% ========================================== Files 752 752 Lines 251568 252348 +780 Branches 47270 47462 +192 ========================================== + Hits 226759 227355 +596 - Misses 16168 16292 +124 - Partials 8641 8701 +60
🚀 New features to boost your workflow:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
5d49228 to
088f769
Compare
There was an error while loading. Please reload this page.
|
Benchmark GHA (misc / startup-core): https://github.com/nodejs/node/actions/runs/32009241408 Results
Benchmark results:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
The modified test is failing on Windows, indicating this PR is adding flakiness. Can you have a look? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
088f769 to
0d3b8a7
Compare
There was an error while loading. Please reload this page.
|
@aduh95 fixed! |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
0d3b8a7 to
f95b3c7
Compare
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@mcollina mind re-reviewing? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Contexts that are not deserialized from the built-in snapshot -- worker threads, and the main context of embedders that create their own isolate or of `node --no-node-snapshot` -- compile (with the code cache at best) every builtin the bootstrap touches, so each eagerly required builtin is startup time (~0.15-0.4 ms apiece). A number of them are only required eagerly so that they end up in the snapshot, or for features the bootstrap path never uses. Load lazily what those paths do not need: - is_main_thread.js: preload util, url, the ESM loader (translators, resolver, module_job/map, source maps, node:module, vm modules, mime, data_url, the TypeScript stripper), internal/blob and internal/dns/utils only while building a snapshot; they load on first use otherwise. - fs: internal/blob (+ internal/encoding and its tables) is only used by fs.openAsBlob(). - internal/url: internal/data_url (+ internal/mime) is only used by the Buffer-returning file URL helpers. - internal/process/execution, the CommonJS loader, esm/translators and esm/load: the TypeScript stripper and data: URL helpers are only needed for TypeScript sources / data: URLs. - pre_execution: internal/dns/utils (+ internal/net) is only needed up front to validate an explicit --dns-result-order or to register the resolver's snapshot serializer; the default order becomes the variable's initializer. - internal/worker: event_loop_utilization and error_serdes are only needed once a sub-worker's ELU is read or it reports an error. - worker_threads: `locks` is defined lazily, like util's lazy exports. Main-thread startup with the snapshot is unchanged (the same modules are preloaded into it; the bootstrap-modules test lists are adjusted). A bare worker compiles 95 -> 83 builtins (cold start -5%); without the snapshot an empty CommonJS entry point compiles 76 -> 59 builtins and an empty ES module entry point 76 -> 69. Signed-off-by: Shelley Vohr <[email protected]>
f95b3c7 to
0cc532a
Compare
|
I am in favor of "for snapshot less mode, try not to eager load that much" in general, although some of the removals seem somewhat too conservative (e.g. the url and util modules are commonly used by most applications) and may only favor the startup of empty scripts but penalise the startup of a real application that actually does something. So I think we should check the performance impact of the scripts that does something beyond being empty as well. If the number Comes from that scenario then LGTM, otherwise can you run the benchmark across the startup-core and startup-cli-version data sets? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@joyeecheung yes, that row is for the "app that actually does something" case i ran so nothing gets penalized: with the snapshot the preloaded set is unchanged (they sit behind |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Landed in e38eede |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Contexts that are not deserialized from the built-in snapshot -- worker threads, and the main context of embedders that create their own isolate or of `node --no-node-snapshot` -- compile (with the code cache at best) every builtin the bootstrap touches, so each eagerly required builtin is startup time (~0.15-0.4 ms apiece). A number of them are only required eagerly so that they end up in the snapshot, or for features the bootstrap path never uses. Load lazily what those paths do not need: - is_main_thread.js: preload util, url, the ESM loader (translators, resolver, module_job/map, source maps, node:module, vm modules, mime, data_url, the TypeScript stripper), internal/blob and internal/dns/utils only while building a snapshot; they load on first use otherwise. - fs: internal/blob (+ internal/encoding and its tables) is only used by fs.openAsBlob(). - internal/url: internal/data_url (+ internal/mime) is only used by the Buffer-returning file URL helpers. - internal/process/execution, the CommonJS loader, esm/translators and esm/load: the TypeScript stripper and data: URL helpers are only needed for TypeScript sources / data: URLs. - pre_execution: internal/dns/utils (+ internal/net) is only needed up front to validate an explicit --dns-result-order or to register the resolver's snapshot serializer; the default order becomes the variable's initializer. - internal/worker: event_loop_utilization and error_serdes are only needed once a sub-worker's ELU is read or it reports an error. - worker_threads: `locks` is defined lazily, like util's lazy exports. Main-thread startup with the snapshot is unchanged (the same modules are preloaded into it; the bootstrap-modules test lists are adjusted). A bare worker compiles 95 -> 83 builtins (cold start -5%); without the snapshot an empty CommonJS entry point compiles 76 -> 59 builtins and an empty ES module entry point 76 -> 69. Signed-off-by: Shelley Vohr <[email protected]> PR-URL: #65329 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Chengzhong Wu <[email protected]> Reviewed-By: Aviv Keller <[email protected]>
Contexts that are not deserialized from the built-in snapshot -- worker threads, and the main context of embedders that create their own isolate or of `node --no-node-snapshot` -- compile (with the code cache at best) every builtin the bootstrap touches, so each eagerly required builtin is startup time (~0.15-0.4 ms apiece). A number of them are only required eagerly so that they end up in the snapshot, or for features the bootstrap path never uses. Load lazily what those paths do not need: - is_main_thread.js: preload util, url, the ESM loader (translators, resolver, module_job/map, source maps, node:module, vm modules, mime, data_url, the TypeScript stripper), internal/blob and internal/dns/utils only while building a snapshot; they load on first use otherwise. - fs: internal/blob (+ internal/encoding and its tables) is only used by fs.openAsBlob(). - internal/url: internal/data_url (+ internal/mime) is only used by the Buffer-returning file URL helpers. - internal/process/execution, the CommonJS loader, esm/translators and esm/load: the TypeScript stripper and data: URL helpers are only needed for TypeScript sources / data: URLs. - pre_execution: internal/dns/utils (+ internal/net) is only needed up front to validate an explicit --dns-result-order or to register the resolver's snapshot serializer; the default order becomes the variable's initializer. - internal/worker: event_loop_utilization and error_serdes are only needed once a sub-worker's ELU is read or it reports an error. - worker_threads: `locks` is defined lazily, like util's lazy exports. Main-thread startup with the snapshot is unchanged (the same modules are preloaded into it; the bootstrap-modules test lists are adjusted). A bare worker compiles 95 -> 83 builtins (cold start -5%); without the snapshot an empty CommonJS entry point compiles 76 -> 59 builtins and an empty ES module entry point 76 -> 69. Signed-off-by: Shelley Vohr <[email protected]> PR-URL: #65329 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Chengzhong Wu <[email protected]> Reviewed-By: Aviv Keller <[email protected]>
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.