GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — esm: add chaining to loaders by JakobJingleheimer · Pull Request #42623 · nodejs/node

Conversation

@JakobJingleheimer

@JakobJingleheimer JakobJingleheimer commented Apr 6, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Notable changes

  • ESMLoader now provides more detailed error messages for hooks, including the specific file and hook from which the error was caused (ex the hook broke the chain; but not SyntaxError within the hook—which was already working). In order to facilitate this:
    • ESMLoader's internal collections of hooks have changed from hook fn[] → Array<{ url: specifier, fn: hook }>
    • ESMLoader::import()'s return has changed from ModuleNamespace[] | ModuleNamespace → Array<{ url: specifier, fn: hook }> | ModuleNamespace

TODOs:

  • hook up next()s
  • custom loaders FIFO → LIFO
  • detect & throw on unsignalled short-circuit
  • update existing hook output error messages to include newly available specifier
  • update code docs
  • update Error docs
  • update ESM docs
  • verify/fix node-land breakages caused the change to ESMLoader::import() (I think this will not be a breaking change to user-land as the functionality seen in user-land continues to be where ESMLoader::import() returns a single ModuleNamespace)
  • update current tests
  • add new tests for chaining

Trying it out

CLI
./node \
    --loader=/PATH/TO/loaderA.mjs \
    --loader=/PATH/TO/loaderB.mjs \
    --input-type=module \
    -e "import fs from 'node:fs'; console.log(2)"
loaderA.mjs
export const resolve = async function resolveA(
	specifier,
	context,
	next,
) {
	console.log('resolveA running');
	return {
		shortCircuit: true,
		url: 'file:///foo/bar.js',
	};
}
export const load = async function loadA(
	resolvedUrl,
	context,
	next,
) {
	console.log('loadA running');
	return {
		format: 'module',
		shortCircuit: true,
		source: 'export default 42',
	}
}
loaderB.mjs
export const resolve = async function resolveB(
	specifier,
	context,
	next,
) {
	console.log('resolveB running');
	return next(specifier);
}
export const load = async function loadB(
	resolvedUrl,
	context,
	next,
) {
	console.log('loadB running');
	return next(resolvedUrl);
}

42 should be printed to stdout when the hooks are properly chained; when disabled, the node:fs module should be printed to stdout.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

GeoffreyBooth, bricss, lin72h, MomenNano, hpx7, RDIL, and wooorm reacted with hooray emoji
All reactions
  • 🎉 7 reactions
@nodejs-github-bot

nodejs-github-bot commented Apr 6, 2022 •
edited by GeoffreyBooth
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/modules
  • @nodejs/loaders
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run. labels Apr 6, 2022
@JakobJingleheimer JakobJingleheimer added esm Issues and PRs related to the ECMAScript Modules implementation. loaders Issues and PRs related to ES module loaders. labels Apr 6, 2022
aduh95
aduh95 reviewed Apr 6, 2022
View reviewed changes
Comment thread lib/internal/modules/esm/loader.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

aduh95
aduh95 reviewed Apr 6, 2022
View reviewed changes
Comment thread lib/internal/modules/esm/loader.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

JakobJingleheimer
JakobJingleheimer commented Apr 6, 2022
View reviewed changes
Comment thread lib/internal/modules/esm/loader.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

targos
targos reviewed Apr 6, 2022
View reviewed changes
Comment thread src/node_options.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

VoltrexKeyva
VoltrexKeyva reviewed Apr 6, 2022
View reviewed changes
Comment thread lib/internal/errors.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/internal/modules/esm/loader.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

@JakobJingleheimer
JakobJingleheimer force-pushed the esm/add-chaining-to-loaders branch 2 times, most recently from ff0c53e to e2c1337 Compare April 6, 2022 18:26
@JakobJingleheimer

JakobJingleheimer commented Apr 6, 2022

Copy link
Copy Markdown
Member Author

Hrrrm. I've got all the existing es-module tests updated except for es-module/test-esm-symlink-type.js: all of its cases are failing with the same TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension "" for […]. That error seems legit based on where it's coming from (ESMLoader doesn't support extensionless files), but the test seems to think it should be okay.

I didn't see anything related to symlinks when making my changes, so I'm not sure what's up 🤔

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bmeck

bmeck commented Apr 6, 2022 via email

Copy link
Copy Markdown
Member
It should have an extension after realpathing right?
…
On Wed, Apr 6, 2022, 4:57 PM Jacob Smith ***@***.***> wrote: Hrrrm. I've got all the existing es-module tests updated except for es-module/test-esm-symlink-type.js: all of its cases are failing with the same TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension "" for […]. That error seems legit based on where it's coming from (ESMLoader doesn't support extensionless files), but the test seems to think it should be okay. I didn't see anything related to symlinks when making my changes, so I'm not sure what's up 🤔 — Reply to this email directly, view it on GitHub <#42623 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABZJIY3J6WPKVHTYM66YD3VDYCEFANCNFSM5SVGTG3A> . You are receiving this because you are on a team that was mentioned.Message ID: ***@***.***>
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@JakobJingleheimer

JakobJingleheimer commented Apr 6, 2022

Copy link
Copy Markdown
Member Author

I presume so based on your question. Do you know whereabouts I could check to confirm that is or isn't happening?

By the time it gets to ESMLoader::load() → defaultLoad() → getFormat() it (supposedly—I didn't print it) doesn't have an extension.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@JakobJingleheimer JakobJingleheimer added the notable-change PRs with changes that should be highlighted in changelogs. label Apr 8, 2022
@GeoffreyBooth GeoffreyBooth mentioned this pull request Apr 8, 2022
Node.js Loaders Team Meeting 2022-04-12 nodejs/loaders#74
Closed
@JakobJingleheimer
JakobJingleheimer marked this pull request as ready for review April 10, 2022 11:09
JakobJingleheimer
JakobJingleheimer commented Apr 10, 2022
View reviewed changes
Comment thread lib/internal/modules/esm/loader.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

@JakobJingleheimer

This comment was marked as outdated.

Sign in to view
@targos
targos marked this pull request as draft April 10, 2022 11:43
@targos
targos marked this pull request as ready for review April 10, 2022 11:43
@targos

This comment was marked as resolved.

Sign in to view
@JakobJingleheimer

This comment was marked as outdated.

Sign in to view
@JakobJingleheimer
JakobJingleheimer requested review from GeoffreyBooth, aduh95, bmeck and guybedford April 10, 2022 16:51
JakobJingleheimer
JakobJingleheimer commented Apr 10, 2022
View reviewed changes
Comment thread test/es-module/test-esm-loader-chaining.mjs Resolved

Uh oh!

There was an error while loading. Please reload this page.

GeoffreyBooth
GeoffreyBooth reviewed Apr 11, 2022
View reviewed changes

@GeoffreyBooth GeoffreyBooth left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven’t reviewed the tests, but I’ve written more than enough notes so far 😄 Overall this is quite good; I agree with the general thrust and design. It shouldn’t take too much work to clean everything up and get this merged.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

JakobJingleheimer reacted with thumbs up emoji JakobJingleheimer reacted with heart emoji
All reactions
  • 👍 1 reaction
  • ❤️ 1 reaction
Comment thread doc/api/errors.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread doc/api/esm.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread doc/api/esm.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread doc/api/esm.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread doc/api/esm.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/internal/modules/esm/loader.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/internal/modules/esm/loader.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/internal/modules/run_main.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/internal/process/esm_loader.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/internal/util/types.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

@JakobJingleheimer

JakobJingleheimer commented Jul 18, 2022

Copy link
Copy Markdown
Member Author

If possible, yes. I think #37468 made it into v16? (if so, it should be okay)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@targos

targos commented Jul 19, 2022

Copy link
Copy Markdown
Member

Yes it did. I cherry-picked this change 👍🏻

JakobJingleheimer reacted with heart emoji JakobJingleheimer reacted with rocket emoji
All reactions
  • ❤️ 1 reaction
  • 🚀 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

targos pushed a commit that referenced this pull request Jul 19, 2022
@JakobJingleheimer @targos
esm: add chaining to loaders
5be7d41
PR-URL: #42623
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
@cspotcode

cspotcode commented Jul 19, 2022

Copy link
Copy Markdown

Double-checking, is #43763 going to land in v16 as well? Otherwise it'll break projects.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@azu azu mentioned this pull request Jul 20, 2022
2022-07-20のJS: Node.js 18.6.0、React NativeのHermesデフォルト化、AWSのデザインシステム jser/jser.github.io#999
Merged
@targos

targos commented Jul 20, 2022

Copy link
Copy Markdown
Member

It wasn't going to be because by default commits are supposed to bake in a Current release before we consider them for LTS, but I just added the lts-watch label and will make sure it lands in v16 too.

cspotcode reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ntucker ntucker mentioned this pull request Jul 21, 2022
internal: Use yarn pnp ntucker/anansi#1586
Closed
targos pushed a commit that referenced this pull request Jul 31, 2022
@JakobJingleheimer @targos
esm: add chaining to loaders
247a6f6
PR-URL: #42623
Reviewed-By: Geoffrey Booth <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
This was referenced Jul 31, 2022
[Bug?]: chain loader privatenumber/tsx#72
Closed
[Bug?]: chain loader yarnpkg/berry#4694
Closed
@targos targos mentioned this pull request Aug 2, 2022
v16.17.0 release proposal #44098
Merged
targos added a commit that referenced this pull request Aug 6, 2022
@targos
2022-08-16, Version 16.17.0 'Gallium' (LTS)
cd03cc3
Notable changes:

Adds `util.parseArgs` helper for higher level command-line argument
parsing.

Contributed by Benjamin Coe, John Gee, Darcy Clarke, Joe Sepi,
Kevin Gibbons, Aaron Casanova, Jessica Nahulan, and Jordan Harband.
#42675

Node.js ESM Loader hooks now support multiple custom loaders, and
composition is achieved via "chaining": `foo-loader` calls `bar-loader`
calls `qux-loader` (a custom loader _must_ now signal a short circuit
when intentionally not calling the next). See the ESM docs
(https://nodejs.org/dist/latest-v16.x/docs/api/esm.html) for details.

Contributed by Jacob Smith, Geoffrey Booth, and Bradley Farias.
#42623

The `node:test` module, which was initially introduced in Node.js
v18.0.0, is now available with all the changes done to it up to Node.js
v18.7.0.

To better align Node.js' experimental implementation of the Web Crypto
API with other runtimes, several changes were made:

* Support for CFRG curves was added, with the `'Ed25519'`, `'Ed448'`,
  `'X25519'`, and `'X448'` algorithms.
* The proprietary `'NODE-DSA'`, `'NODE-DH'`, `'NODE-SCRYPT'`,
  `'NODE-ED25519'`, `'NODE-ED448'`, `'NODE-X25519'`, and `'NODE-X448'`
  algorithms were removed.
* The proprietary `'node.keyObject'` import/export format was removed.

Contributed by Filip Skokan.
#42507
#43310

Updated Corepack to 0.12.1 - #43965
Updated ICU to 71.1 - #42655
Updated npm to 8.15.0 - #43917
Updated Undici to 5.8.0 - #43886

(SEMVER-MINOR) crypto: make authTagLength optional for CC20P1305 (Tobias Nießen) [#42427](#42427)
(SEMVER-MINOR) crypto: align webcrypto RSA key import/export with other implementations (Filip Skokan) [#42816](#42816)
(SEMVER-MINOR) dns: export error code constants from `dns/promises` (Feng Yu) [#43176](#43176)
doc: deprecate ercion to integer in process.exit (Daeyeon Jeong) [#43738](#43738)
(SEMVER-MINOR) doc: deprecate diagnostics\_channel object subscribe method (Stephen Belanger) [#42714](#42714)
(SEMVER-MINOR) errors: add support for cause in aborterror (James M Snell) [#41008](#41008)
(SEMVER-MINOR) events: expose CustomEvent on global with CLI flag (Daeyeon Jeong) [#43885](#43885)
(SEMVER-MINOR) events: add `CustomEvent` (Daeyeon Jeong) [#43514](#43514)
(SEMVER-MINOR) events: propagate abortsignal reason in new AbortError ctor in events (James M Snell) [#41008](#41008)
(SEMVER-MINOR) fs: propagate abortsignal reason in new AbortSignal constructors (James M Snell) [#41008](#41008)
(SEMVER-MINOR) fs: make params in writing methods optional (LiviaMedeiros) [#42601](#42601)
(SEMVER-MINOR) fs: add `read(buffer[, options])` versions (LiviaMedeiros) [#42768](#42768)
(SEMVER-MINOR) http: add drop request event for http server (theanarkh) [#43806](#43806)
(SEMVER-MINOR) http: add diagnostics channel for http client (theanarkh) [#43580](#43580)
(SEMVER-MINOR) http: add perf\_hooks detail for http request and client (theanarkh) [#43361](#43361)
(SEMVER-MINOR) http: add uniqueHeaders option to request and createServer (Paolo Insogna) [#41397](#41397)
(SEMVER-MINOR) http2: propagate abortsignal reason in new AbortError constructor (James M Snell) [#41008](#41008)
(SEMVER-MINOR) http2: compat support for array headers (OneNail) [#42901](#42901)
(SEMVER-MINOR) lib: propagate abortsignal reason in new AbortError constructor in blob (James M Snell) [#41008](#41008)
(SEMVER-MINOR) lib: add abortSignal.throwIfAborted() (James M Snell) [#40951](#40951)
(SEMVER-MINOR) lib: improved diagnostics\_channel subscribe/unsubscribe (Stephen Belanger) [#42714](#42714)
(SEMVER-MINOR) module: add isBuiltIn method (hemanth.hm) [#43396](#43396)
(SEMVER-MINOR) moduleepl**: support 'node:'-only core modules (Colin Ihrig) [#42325](#42325)
(SEMVER-MINOR) net: add drop event for net server (theanarkh) [#43582](#43582)
(SEMVER-MINOR) net: add ability to reset a tcp socket (pupilTong) [#43112](#43112)
(SEMVER-MINOR) nodepi**: emit uncaught-exception on unhandled tsfn callbacks (Chengzhong Wu) [#36510](#36510)
(SEMVER-MINOR) perfhooks**: add PerformanceResourceTiming (RafaelGSS) [#42725](#42725)
(SEMVER-MINOR) report: add more heap infos in process report (theanarkh) [#43116](#43116)
(SEMVER-MINOR) src: add --openssl-legacy-provider option (Daniel Bevenius) [#40478](#40478)
(SEMVER-MINOR) src: define fs.constants.S\_IWUSR & S\_IRUSR for Win (Liviu Ionescu) [#42757](#42757)
(SEMVER-MINOR) srcoc,test**: add --openssl-shared-config option (Daniel Bevenius) [#43124](#43124)
(SEMVER-MINOR) stream: use cause options in AbortError constructors (James M Snell) [#41008](#41008)
(SEMVER-MINOR) stream: add iterator helper find (Nitzan Uziely) [#41849](#41849)
(SEMVER-MINOR) stream: add writableAborted (Robert Nagy) [#40802](#40802)
(SEMVER-MINOR) timers: propagate signal.reason in awaitable timers (James M Snell) [#41008](#41008)
(SEMVER-MINOR) v8: add v8.startupSnapshot utils (Joyee Cheung) [#43329](#43329)
(SEMVER-MINOR) v8: export more fields in getHeapStatistics (theanarkh) [#42784](#42784)
(SEMVER-MINOR) worker: add hasRef() to MessagePort (Darshan Sen) [#42849](#42849)

PR-URL: #44098
targos added a commit that referenced this pull request Aug 15, 2022
@targos
2022-08-16, Version 16.17.0 'Gallium' (LTS)
30187ba
Notable changes:

Adds `util.parseArgs` helper for higher level command-line argument
parsing.
Contributed by Benjamin Coe, John Gee, Darcy Clarke, Joe Sepi,
Kevin Gibbons, Aaron Casanova, Jessica Nahulan, and Jordan Harband.
#42675

Node.js ESM Loader hooks now support multiple custom loaders, and
composition is achieved via "chaining": `foo-loader` calls `bar-loader`
calls `qux-loader` (a custom loader _must_ now signal a short circuit
when intentionally not calling the next). See the ESM docs
(https://nodejs.org/dist/latest-v16.x/docs/api/esm.html) for details.
Contributed by Jacob Smith, Geoffrey Booth, and Bradley Farias.
#42623

The `node:test` module, which was initially introduced in Node.js
v18.0.0, is now available with all the changes done to it up to Node.js
v18.7.0.

To better align Node.js' experimental implementation of the Web Crypto
API with other runtimes, several changes were made:
* Support for CFRG curves was added, with the `'Ed25519'`, `'Ed448'`,
  `'X25519'`, and `'X448'` algorithms.
* The proprietary `'NODE-DSA'`, `'NODE-DH'`, `'NODE-SCRYPT'`,
  `'NODE-ED25519'`, `'NODE-ED448'`, `'NODE-X25519'`, and `'NODE-X448'`
  algorithms were removed.
* The proprietary `'node.keyObject'` import/export format was removed.
Contributed by Filip Skokan.
#42507
#43310

Updated Corepack to 0.12.1 - #43965
Updated ICU to 71.1 - #42655
Updated npm to 8.15.0 - #43917
Updated Undici to 5.8.0 - #43886

(SEMVER-MINOR) crypto: make authTagLength optional for CC20P1305 (Tobias Nießen) #42427
(SEMVER-MINOR) crypto: align webcrypto RSA key import/export with other implementations (Filip Skokan) #42816
(SEMVER-MINOR) dns: export error code constants from `dns/promises` (Feng Yu) #43176
doc: deprecate coercion to integer in process.exit (Daeyeon Jeong) #43738
(SEMVER-MINOR) doc: deprecate diagnostics_channel object subscribe method (Stephen Belanger) #42714
(SEMVER-MINOR) errors: add support for cause in aborterror (James M Snell) #41008
(SEMVER-MINOR) events: expose CustomEvent on global with CLI flag (Daeyeon Jeong) #43885
(SEMVER-MINOR) events: add `CustomEvent` (Daeyeon Jeong) #43514
(SEMVER-MINOR) events: propagate abortsignal reason in new AbortError ctor in events (James M Snell) #41008
(SEMVER-MINOR) fs: propagate abortsignal reason in new AbortSignal constructors (James M Snell) #41008
(SEMVER-MINOR) fs: make params in writing methods optional (LiviaMedeiros) #42601
(SEMVER-MINOR) fs: add `read(buffer[, options])` versions (LiviaMedeiros) #42768
(SEMVER-MINOR) http: add drop request event for http server (theanarkh) #43806
(SEMVER-MINOR) http: add diagnostics channel for http client (theanarkh) #43580
(SEMVER-MINOR) http: add perf_hooks detail for http request and client (theanarkh) #43361
(SEMVER-MINOR) http: add uniqueHeaders option to request and createServer (Paolo Insogna) #41397
(SEMVER-MINOR) http2: propagate abortsignal reason in new AbortError constructor (James M Snell) #41008
(SEMVER-MINOR) http2: compat support for array headers (OneNail) #42901
(SEMVER-MINOR) lib: propagate abortsignal reason in new AbortError constructor in blob (James M Snell) #41008
(SEMVER-MINOR) lib: add abortSignal.throwIfAborted() (James M Snell) #40951
(SEMVER-MINOR) lib: improved diagnostics_channel subscribe/unsubscribe (Stephen Belanger) #42714
(SEMVER-MINOR) module: add isBuiltIn method (hemanth.hm) #43396
(SEMVER-MINOR) module,repl: support 'node:'-only core modules (Colin Ihrig) #42325
(SEMVER-MINOR) net: add drop event for net server (theanarkh) #43582
(SEMVER-MINOR) net: add ability to reset a tcp socket (pupilTong) #43112
(SEMVER-MINOR) node-api: emit uncaught-exception on unhandled tsfn callbacks (Chengzhong Wu) #36510
(SEMVER-MINOR) perf_hooks: add PerformanceResourceTiming (RafaelGSS) #42725
(SEMVER-MINOR) report: add more heap infos in process report (theanarkh) #43116
(SEMVER-MINOR) src: add --openssl-legacy-provider option (Daniel Bevenius) #40478
(SEMVER-MINOR) src: define fs.constants.S_IWUSR & S_IRUSR for Win (Liviu Ionescu) #42757
(SEMVER-MINOR) src,doc,test: add --openssl-shared-config option (Daniel Bevenius) #43124
(SEMVER-MINOR) stream: use cause options in AbortError constructors (James M Snell) #41008
(SEMVER-MINOR) stream: add iterator helper find (Nitzan Uziely) #41849
(SEMVER-MINOR) stream: add writableAborted (Robert Nagy) #40802
(SEMVER-MINOR) timers: propagate signal.reason in awaitable timers (James M Snell) #41008
(SEMVER-MINOR) v8: add v8.startupSnapshot utils (Joyee Cheung) #43329
(SEMVER-MINOR) v8: export more fields in getHeapStatistics (theanarkh) #42784
(SEMVER-MINOR) worker: add hasRef() to MessagePort (Darshan Sen) #42849

PR-URL: #44098
targos added a commit that referenced this pull request Aug 16, 2022
@targos
2022-08-16, Version 16.17.0 'Gallium' (LTS)
5e5fb82
Notable changes:

Adds `util.parseArgs` helper for higher level command-line argument
parsing.
Contributed by Benjamin Coe, John Gee, Darcy Clarke, Joe Sepi,
Kevin Gibbons, Aaron Casanova, Jessica Nahulan, and Jordan Harband.
#42675

Node.js ESM Loader hooks now support multiple custom loaders, and
composition is achieved via "chaining": `foo-loader` calls `bar-loader`
calls `qux-loader` (a custom loader _must_ now signal a short circuit
when intentionally not calling the next). See the ESM docs
(https://nodejs.org/dist/latest-v16.x/docs/api/esm.html) for details.
Contributed by Jacob Smith, Geoffrey Booth, and Bradley Farias.
#42623

The `node:test` module, which was initially introduced in Node.js
v18.0.0, is now available with all the changes done to it up to Node.js
v18.7.0.

To better align Node.js' experimental implementation of the Web Crypto
API with other runtimes, several changes were made:
* Support for CFRG curves was added, with the `'Ed25519'`, `'Ed448'`,
  `'X25519'`, and `'X448'` algorithms.
* The proprietary `'NODE-DSA'`, `'NODE-DH'`, `'NODE-SCRYPT'`,
  `'NODE-ED25519'`, `'NODE-ED448'`, `'NODE-X25519'`, and `'NODE-X448'`
  algorithms were removed.
* The proprietary `'node.keyObject'` import/export format was removed.
Contributed by Filip Skokan.
#42507
#43310

Updated Corepack to 0.12.1 - #43965
Updated ICU to 71.1 - #42655
Updated npm to 8.15.0 - #43917
Updated Undici to 5.8.0 - #43886

(SEMVER-MINOR) crypto: make authTagLength optional for CC20P1305 (Tobias Nießen) #42427
(SEMVER-MINOR) crypto: align webcrypto RSA key import/export with other implementations (Filip Skokan) #42816
(SEMVER-MINOR) dns: export error code constants from `dns/promises` (Feng Yu) #43176
doc: deprecate coercion to integer in process.exit (Daeyeon Jeong) #43738
(SEMVER-MINOR) doc: deprecate diagnostics_channel object subscribe method (Stephen Belanger) #42714
(SEMVER-MINOR) errors: add support for cause in aborterror (James M Snell) #41008
(SEMVER-MINOR) events: expose CustomEvent on global with CLI flag (Daeyeon Jeong) #43885
(SEMVER-MINOR) events: add `CustomEvent` (Daeyeon Jeong) #43514
(SEMVER-MINOR) events: propagate abortsignal reason in new AbortError ctor in events (James M Snell) #41008
(SEMVER-MINOR) fs: propagate abortsignal reason in new AbortSignal constructors (James M Snell) #41008
(SEMVER-MINOR) fs: make params in writing methods optional (LiviaMedeiros) #42601
(SEMVER-MINOR) fs: add `read(buffer[, options])` versions (LiviaMedeiros) #42768
(SEMVER-MINOR) http: add drop request event for http server (theanarkh) #43806
(SEMVER-MINOR) http: add diagnostics channel for http client (theanarkh) #43580
(SEMVER-MINOR) http: add perf_hooks detail for http request and client (theanarkh) #43361
(SEMVER-MINOR) http: add uniqueHeaders option to request and createServer (Paolo Insogna) #41397
(SEMVER-MINOR) http2: propagate abortsignal reason in new AbortError constructor (James M Snell) #41008
(SEMVER-MINOR) http2: compat support for array headers (OneNail) #42901
(SEMVER-MINOR) lib: propagate abortsignal reason in new AbortError constructor in blob (James M Snell) #41008
(SEMVER-MINOR) lib: add abortSignal.throwIfAborted() (James M Snell) #40951
(SEMVER-MINOR) lib: improved diagnostics_channel subscribe/unsubscribe (Stephen Belanger) #42714
(SEMVER-MINOR) module: add isBuiltIn method (hemanth.hm) #43396
(SEMVER-MINOR) module,repl: support 'node:'-only core modules (Colin Ihrig) #42325
(SEMVER-MINOR) net: add drop event for net server (theanarkh) #43582
(SEMVER-MINOR) net: add ability to reset a tcp socket (pupilTong) #43112
(SEMVER-MINOR) node-api: emit uncaught-exception on unhandled tsfn callbacks (Chengzhong Wu) #36510
(SEMVER-MINOR) perf_hooks: add PerformanceResourceTiming (RafaelGSS) #42725
(SEMVER-MINOR) report: add more heap infos in process report (theanarkh) #43116
(SEMVER-MINOR) src: add --openssl-legacy-provider option (Daniel Bevenius) #40478
(SEMVER-MINOR) src: define fs.constants.S_IWUSR & S_IRUSR for Win (Liviu Ionescu) #42757
(SEMVER-MINOR) src,doc,test: add --openssl-shared-config option (Daniel Bevenius) #43124
(SEMVER-MINOR) stream: use cause options in AbortError constructors (James M Snell) #41008
(SEMVER-MINOR) stream: add iterator helper find (Nitzan Uziely) #41849
(SEMVER-MINOR) stream: add writableAborted (Robert Nagy) #40802
(SEMVER-MINOR) timers: propagate signal.reason in awaitable timers (James M Snell) #41008
(SEMVER-MINOR) v8: add v8.startupSnapshot utils (Joyee Cheung) #43329
(SEMVER-MINOR) v8: export more fields in getHeapStatistics (theanarkh) #42784
(SEMVER-MINOR) worker: add hasRef() to MessagePort (Darshan Sen) #42849

PR-URL: #44098
@brev brev mentioned this pull request Aug 23, 2022
ESM Loader API has changed in Node 18 sebamarynissen/create-esm-loader#4
Closed
@localjo localjo mentioned this pull request Aug 23, 2022
[Bug?]: node 16.17.0 update cause yarn 3.2.2 build to fail yarnpkg/berry#4778
Closed
1 task
@localjo

localjo commented Aug 23, 2022

Copy link
Copy Markdown

It seems that this PR had breaking changes, but got merged into the 16.17.0 release. See the discussion in yarnpkg/berry#4778 for details.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@JakobJingleheimer

JakobJingleheimer commented Aug 23, 2022

Copy link
Copy Markdown
Member Author

@localjo it did indeed have breaking changes. It's experimental, so it doesn't require a major version bump. The author of yarn was aware of the incoming change (he is on this team).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@localjo

localjo commented Aug 23, 2022

Copy link
Copy Markdown

@JakobJingleheimer It seems that the breaking changes are breaking a lot of people's builds though. See the discussion here: yarnpkg/berry#4778

Whether it should have been merged or not, I don't know. But I'm mentioning it here in case people are looking for info about why their projects stopped building, and how to solve it. In my case, I solved my issue by locking Node in my CI environment to 16.16.0. Less than ideal, but it works for me.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@GeoffreyBooth

GeoffreyBooth commented Aug 23, 2022

Copy link
Copy Markdown
Member

Whether it should have been merged or not, I don't know.

It was an intentional breaking change. Loaders are still an experimental API.

localjo reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@localjo

localjo commented Aug 23, 2022

Copy link
Copy Markdown

Fair enough. It seems a handful of packages must be using that experimental API and the people (including myself) who are importing those packages are unaware. Likely either an RTFM problem on my part, or lack of documentation on the part of the publishers of the packages. Anyway, thanks for the clarification.

JakobJingleheimer reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@strophy strophy mentioned this pull request Aug 24, 2022
fix: yarn fails under node 16.17.x dashpay/platform#506
Closed
3 of 5 tasks
This was referenced Sep 1, 2022
chore: workaround test-ui ci failure vitest-dev/vitest#1942
Merged
fix: support type: module in Node.js 16.17.0+ and 18.6.0+ cypress-io/cypress#23637
Merged
@andrasbacsai andrasbacsai mentioned this pull request Sep 2, 2022
Fix esm loader webstudio-is/webstudio#253
Merged
@wbt

wbt commented Sep 2, 2022

Copy link
Copy Markdown
Contributor

Breaking changes, especially intentional ones, should not be merged without a breaking-change semver update. This also broke my build process, with GitHub Actions automatically incorporating the latest version of Node. Code that previously worked with no indications about it being experimental was suddenly both labeled as experimental and broken, without a corresponding change in the codebase.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@JakobJingleheimer

JakobJingleheimer commented Sep 2, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

@wbt you are mistaken. This feature has been marked experimental since its inception multiple years ago. This change was also highlighted very prominently at the very top of the release notes as breaking and specifically how and how to update. Experimental features are not semver major.

If your build process is sensitive to experimental changes, you should pin your version appropriately. For example, we do not release breaking experimental changes in semver patch, only semver minor (and if it so happens, major).

If you are dependent on some intermediary that is leveraging an experimental feature and did not pin or release appropriately, take it up with that dependency.

EDIT: If you would like to be notified of notable and breaking changes in Loaders/Customisation API, please subscribe to this issue nodejs/loaders#103

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs nodejs locked and limited conversation to collaborators Sep 2, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@ljharb ljharb ljharb left review comments
@jasnell jasnell jasnell left review comments
@guybedford guybedford guybedford left review comments
@GeoffreyBooth GeoffreyBooth GeoffreyBooth approved these changes
@aduh95 aduh95 aduh95 approved these changes
@bmeck bmeck Awaiting requested review from bmeck
+3 more reviewers
@cspotcode cspotcode cspotcode left review comments
@targos targos targos left review comments
@VoltrexKeyva VoltrexKeyva VoltrexKeyva left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue-squash PRs the Commit Queue should land as one squashed commit. esm Issues and PRs related to the ECMAScript Modules implementation. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. loaders Issues and PRs related to ES module loaders. needs-ci PRs that need a full CI run. notable-change PRs with changes that should be highlighted in changelogs. semver-minor PRs that contain new features and should be released in the next minor version.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

15 participants

@JakobJingleheimer @nodejs-github-bot @bmeck @targos @cspotcode @aduh95 @Flarna @ljharb @GeoffreyBooth @guybedford @wmertens @localjo @wbt @jasnell @VoltrexKeyva
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.