GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — sea: allow requiring core modules with the "node:" prefix by RaisinTen · Pull Request #47779 · nodejs/node

Conversation

@RaisinTen

@RaisinTen RaisinTen commented Apr 29, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Previously, the require() function exposed to the embedded SEA code was calling the internal require() function if the module name belonged to the list of public core modules but the internal require() function does not support loading modules with the "node:" prefix, so this change forwards the calls to another require() function that supports this.

Fixes: nodejs/single-executable#69


bootstrap: fix bug in the snapshot require function and share the code with the SEA require

Previously, requiring node:test while building a snapshot
was failing with a Error: Cannot find module 'node:test'. error, so
this change fixes that and uses the same code path for normalizing ids
for both the snapshot require function as well as the SEA require
function.

Details

Previously:

$ node --snapshot-blob snapshot.blob --build-snapshot <(echo "require('node:test')")
node:internal/main/mksnapshot:105
    throw err;
    ^

Error: Cannot find module 'node:test'.
    at requireForUserSnapshot (node:internal/main/mksnapshot:101:17)
    at /dev/fd/13:1:1
    at main (node:internal/main/mksnapshot:168:5)
    at node:internal/main/mksnapshot:185:1

$ node --snapshot-blob snapshot.blob --build-snapshot <(echo "require('test')")
(node:5389) Warning: built-in module test is not yet supported in user snapshots
(Use `node --trace-warnings ...` to show where the warning was created)
Unknown external reference 0x102d226e0.
<unresolved>
[1]    5389 trace trap  node --snapshot-blob snapshot.blob --build-snapshot <(echo "require('test')")

Now:

$ ./node --snapshot-blob snapshot.blob --build-snapshot <(echo "require('node:test')")
(node:5394) Warning: built-in module node:test is not yet supported in user snapshots
(Use `node --trace-warnings ...` to show where the warning was created)
Unknown external reference 0x105cb25e0.
<unresolved>
[1]    5394 trace trap  ./node --snapshot-blob snapshot.blob --build-snapshot

$ ./node --snapshot-blob snapshot.blob --build-snapshot <(echo "require('test')")
node:internal/main/mksnapshot:106
    throw err;
    ^

Error: Cannot find module 'test'.
    at requireForUserSnapshot (node:internal/main/mksnapshot:102:17)
    at /dev/fd/13:1:1
    at main (node:internal/main/mksnapshot:169:5)
    at node:internal/main/mksnapshot:186:1

Since the Unknown external reference error, which happens now for
the node:test module, was already present for the test module,
it should be okay to fix in a separate PR.

More info about the crash in #47832.

Refs: #47779 (comment)
Signed-off-by: Darshan Sen [email protected]


cc @nodejs/single-executable

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bricss and benjamingr reacted with thumbs up emoji
All reactions
  • 👍 2 reactions
@RaisinTen
sea: allow requiring core modules with the "node:" prefix
614323f
Previously, the `require()` function exposed to the embedded SEA code
was calling the internal `require()` function if the module name
belonged to the list of public core modules but the internal `require()`
function does not support loading modules with the "node:" prefix, so
this change forwards the calls to another `require()` function that
supports this.

Fixes: nodejs/single-executable#69
Signed-off-by: Darshan Sen <[email protected]>
@RaisinTen RaisinTen added the single-executable Issues and PRs related to single-executable applications. label Apr 29, 2023
@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. util Issues and PRs related to the built-in util module. labels Apr 29, 2023
@RaisinTen RaisinTen mentioned this pull request Apr 29, 2023
Missing internal module nodejs/single-executable#69
Closed
@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
ljharb
ljharb approved these changes Apr 29, 2023
View reviewed changes
cjihrig
cjihrig approved these changes Apr 29, 2023
View reviewed changes
@RaisinTen RaisinTen added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Apr 29, 2023
lpinca
lpinca approved these changes Apr 29, 2023
View reviewed changes
@github-actions github-actions Bot mentioned this pull request Apr 30, 2023
CI Reliability 2023-04-30 nodejs/reliability#551
Open
33 tasks
@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
joyeecheung
joyeecheung reviewed Apr 30, 2023
View reviewed changes

@joyeecheung joyeecheung left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should update the documentation to remove the suggestion of Module.createRequire() now that we create one for them?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@github-actions github-actions Bot mentioned this pull request May 1, 2023
CI Reliability 2023-05-01 nodejs/reliability#552
Open
33 tasks
mhdawson
mhdawson approved these changes May 1, 2023
View reviewed changes

@mhdawson mhdawson left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@RaisinTen

RaisinTen commented May 1, 2023

Copy link
Copy Markdown
Member Author

I think we should update the documentation to remove the suggestion of Module.createRequire() now that we create one for them?

@joyeecheung with this change, the require() function that's exposed to the embedded SEA code is still not fs-based because of the Module.isBuiltin check, so if users need access to the file system, they would still need to use Module.createRequire() to build another require() function.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

joyeecheung
joyeecheung reviewed May 1, 2023
View reviewed changes
Comment thread lib/internal/util/embedding.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

This was referenced May 2, 2023
CI Reliability 2023-05-02 nodejs/reliability#553
Open
CI Reliability 2023-05-03 nodejs/reliability#554
Open
@RaisinTen
bootstrap: fix bug in the snapshot require function and share the cod…
2d9e3c5
…e with the SEA require

Previously, this code:
```sh
node --snapshot-blob snapshot.blob --build-snapshot <(echo "require('node:test')")`
```
was failing with a "Error: Cannot find module 'node:test'." error, so
this change fixes that and uses the same code path for normalizing ids
for both the snapshot require function as well as the SEA require
function.

However, building a snapshot when `node:test` is required now fails with:
```sh
./node --snapshot-blob snapshot.blob --build-snapshot <(echo "require('node:test')")
(node:5362) Warning: built-in module node:test is not yet supported in user snapshots
(Use `node --trace-warnings ...` to show where the warning was created)
Unknown external reference 0x10201dd70.
<unresolved>
[1]    5362 trace trap  ./node --snapshot-blob snapshot.blob --build-snapshot
```
which could be solved in a separate PR.

Refs: nodejs#47779 (comment)
Signed-off-by: Darshan Sen <[email protected]>
aymen94
aymen94 approved these changes May 3, 2023
View reviewed changes
@RaisinTen RaisinTen mentioned this pull request May 3, 2023
Support snapshotting node:test #47832
Closed
@nodejs-github-bot

nodejs-github-bot commented May 3, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/51595/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

tony-go
tony-go approved these changes May 3, 2023
View reviewed changes
Comment thread lib/internal/bootstrap/realm.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot mentioned this pull request May 4, 2023
CI Reliability 2023-05-04 nodejs/reliability#555
Open
28 tasks
joyeecheung
joyeecheung approved these changes May 4, 2023
View reviewed changes
@RaisinTen RaisinTen added commit-queue PRs queued for automated landing through the Commit Queue. commit-queue-rebase PRs the Commit Queue should land as multiple self-contained commits. commit-queue-squash PRs the Commit Queue should land as one squashed commit. and removed commit-queue-rebase PRs the Commit Queue should land as multiple self-contained commits. labels May 4, 2023
@nodejs-github-bot nodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label May 4, 2023
@nodejs-github-bot
nodejs-github-bot merged commit c868aad into nodejs:main May 4, 2023
@nodejs-github-bot

nodejs-github-bot commented May 4, 2023

Copy link
Copy Markdown
Collaborator

Landed in c868aad

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@RaisinTen
RaisinTen deleted the sea-allow-requiring-core-modules-with-node-prefix branch May 4, 2023 14:33
@github-actions github-actions Bot mentioned this pull request May 5, 2023
CI Reliability 2023-05-05 nodejs/reliability#556
Open
22 tasks
RaisinTen added a commit to RaisinTen/node that referenced this pull request May 6, 2023
@RaisinTen
module: refactor to use normalizeRequirableId in the CJS module loader
1b0d145
`BuiltinModule.normalizeRequirableId()` was introduced in
nodejs#47779 to fix a bug in the require
function of SEAs and Snapshots, so that built-in modules with the
`node:` scheme could be required correctly. This change makes more use
of this API instead of `BuiltinModule.canBeRequiredByUsers()` and
`BuiltinModule.canBeRequiredWithoutScheme()` to reduce chances of
such bugs.

Signed-off-by: Darshan Sen <[email protected]>
@RaisinTen RaisinTen mentioned this pull request May 6, 2023
module: refactor to use normalizeRequirableId in the CJS module loader #47896
Merged
nodejs-github-bot pushed a commit that referenced this pull request May 8, 2023
@RaisinTen
module: refactor to use normalizeRequirableId in the CJS module loader
7fe4745
`BuiltinModule.normalizeRequirableId()` was introduced in
#47779 to fix a bug in the require
function of SEAs and Snapshots, so that built-in modules with the
`node:` scheme could be required correctly. This change makes more use
of this API instead of `BuiltinModule.canBeRequiredByUsers()` and
`BuiltinModule.canBeRequiredWithoutScheme()` to reduce chances of
such bugs.

Signed-off-by: Darshan Sen <[email protected]>
PR-URL: #47896
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Mohammed Keyvanzadeh <[email protected]>
targos pushed a commit that referenced this pull request May 12, 2023
@RaisinTen @targos
sea: allow requiring core modules with the "node:" prefix
1a7fc18
Previously, the `require()` function exposed to the embedded SEA code
was calling the internal `require()` function if the module name
belonged to the list of public core modules but the internal `require()`
function does not support loading modules with the "node:" prefix, so
this change forwards the calls to another `require()` function that
supports this.

Fixes: nodejs/single-executable#69
Signed-off-by: Darshan Sen <[email protected]>
PR-URL: #47779
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
targos pushed a commit that referenced this pull request May 12, 2023
@RaisinTen @targos
module: refactor to use normalizeRequirableId in the CJS module loader
c19385c
`BuiltinModule.normalizeRequirableId()` was introduced in
#47779 to fix a bug in the require
function of SEAs and Snapshots, so that built-in modules with the
`node:` scheme could be required correctly. This change makes more use
of this API instead of `BuiltinModule.canBeRequiredByUsers()` and
`BuiltinModule.canBeRequiredWithoutScheme()` to reduce chances of
such bugs.

Signed-off-by: Darshan Sen <[email protected]>
PR-URL: #47896
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Mohammed Keyvanzadeh <[email protected]>
@targos targos mentioned this pull request May 15, 2023
v20.2.0 release proposal #48020
Merged
@danielleadams danielleadams added the backport-requested-v18.x label Jul 5, 2023
@aduh95

aduh95 commented Sep 13, 2023

Copy link
Copy Markdown
Contributor

@RaisinTen any chance you would have time to backport this to v18.x?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@RaisinTen RaisinTen mentioned this pull request Sep 14, 2023
[v18.x backport] sea: allow requiring core modules with the "node:" prefix #49648
Closed
@RaisinTen

RaisinTen commented Sep 14, 2023

Copy link
Copy Markdown
Member Author

Sure, done - #49648

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@RaisinTen RaisinTen added backport-open-v18.x and removed backport-requested-v18.x labels Sep 14, 2023
targos pushed a commit that referenced this pull request Nov 10, 2023
@RaisinTen @targos
sea: allow requiring core modules with the "node:" prefix
b8634ee
Previously, the `require()` function exposed to the embedded SEA code
was calling the internal `require()` function if the module name
belonged to the list of public core modules but the internal `require()`
function does not support loading modules with the "node:" prefix, so
this change forwards the calls to another `require()` function that
supports this.

Fixes: nodejs/single-executable#69
Signed-off-by: Darshan Sen <[email protected]>
PR-URL: #47779
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Signed-off-by: Darshan Sen <[email protected]>
@RaisinTen RaisinTen added backported-to-v18.x and removed backport-open-v18.x labels Nov 10, 2023
targos pushed a commit that referenced this pull request Nov 10, 2023
@RaisinTen @targos
module: refactor to use normalizeRequirableId in the CJS module loader
6f0458d
`BuiltinModule.normalizeRequirableId()` was introduced in
#47779 to fix a bug in the require
function of SEAs and Snapshots, so that built-in modules with the
`node:` scheme could be required correctly. This change makes more use
of this API instead of `BuiltinModule.canBeRequiredByUsers()` and
`BuiltinModule.canBeRequiredWithoutScheme()` to reduce chances of
such bugs.

Signed-off-by: Darshan Sen <[email protected]>
PR-URL: #47896
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Mohammed Keyvanzadeh <[email protected]>
@targos targos mentioned this pull request Nov 28, 2023
v18.19.0 release proposal #50953
Merged
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
sea: allow requiring core modules with the "node:" prefix
befe328
Previously, the `require()` function exposed to the embedded SEA code
was calling the internal `require()` function if the module name
belonged to the list of public core modules but the internal `require()`
function does not support loading modules with the "node:" prefix, so
this change forwards the calls to another `require()` function that
supports this.

Fixes: nodejs/single-executable#69
Signed-off-by: Darshan Sen <[email protected]>
PR-URL: nodejs/node#47779
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Signed-off-by: Darshan Sen <[email protected]>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
module: refactor to use normalizeRequirableId in the CJS
7e5db3d
`BuiltinModule.normalizeRequirableId()` was introduced in
nodejs/node#47779 to fix a bug in the require
function of SEAs and Snapshots, so that built-in modules with the
`node:` scheme could be required correctly. This change makes more use
of this API instead of `BuiltinModule.canBeRequiredByUsers()` and
`BuiltinModule.canBeRequiredWithoutScheme()` to reduce chances of
such bugs.

Signed-off-by: Darshan Sen <[email protected]>
PR-URL: nodejs/node#47896
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Mohammed Keyvanzadeh <[email protected]>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
sea: allow requiring core modules with the "node:" prefix
11494df
Previously, the `require()` function exposed to the embedded SEA code
was calling the internal `require()` function if the module name
belonged to the list of public core modules but the internal `require()`
function does not support loading modules with the "node:" prefix, so
this change forwards the calls to another `require()` function that
supports this.

Fixes: nodejs/single-executable#69
Signed-off-by: Darshan Sen <[email protected]>
PR-URL: nodejs/node#47779
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Signed-off-by: Darshan Sen <[email protected]>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
module: refactor to use normalizeRequirableId in the CJS
197530e
`BuiltinModule.normalizeRequirableId()` was introduced in
nodejs/node#47779 to fix a bug in the require
function of SEAs and Snapshots, so that built-in modules with the
`node:` scheme could be required correctly. This change makes more use
of this API instead of `BuiltinModule.canBeRequiredByUsers()` and
`BuiltinModule.canBeRequiredWithoutScheme()` to reduce chances of
such bugs.

Signed-off-by: Darshan Sen <[email protected]>
PR-URL: nodejs/node#47896
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Mohammed Keyvanzadeh <[email protected]>
abhishekumar-tyagi pushed a commit to abhishekumar-tyagi/node that referenced this pull request May 5, 2024
@RaisinTen
sea: allow requiring core modules with the "node:" prefix
003f6ba
Previously, the `require()` function exposed to the embedded SEA code
was calling the internal `require()` function if the module name
belonged to the list of public core modules but the internal `require()`
function does not support loading modules with the "node:" prefix, so
this change forwards the calls to another `require()` function that
supports this.

Fixes: nodejs/single-executable#69
Signed-off-by: Darshan Sen <[email protected]>
PR-URL: nodejs/node#47779
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Signed-off-by: Darshan Sen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ljharb ljharb ljharb approved these changes
@lpinca lpinca lpinca approved these changes
@cjihrig cjihrig cjihrig approved these changes
@joyeecheung joyeecheung joyeecheung approved these changes
+3 more reviewers
@tony-go tony-go tony-go approved these changes
@mhdawson mhdawson mhdawson approved these changes
@aymen94 aymen94 aymen94 approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue-squash PRs the Commit Queue should land as one squashed commit. needs-ci PRs that need a full CI run. single-executable Issues and PRs related to single-executable applications. util Issues and PRs related to the built-in util module.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Missing internal module

Uh oh!

There was an error while loading. Please reload this page.

11 participants

@RaisinTen @nodejs-github-bot @aduh95 @ljharb @lpinca @cjihrig @joyeecheung @mhdawson @aymen94 @tony-go @danielleadams
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.