GeezerVison

Copy link

Copy Markdown

Member

peterhirn reacted with heart emoji

mcollina force-pushed the stream-async-iterator-perf branch from 2307152 to 6f9a2f3 Compare July 12, 2026 08:36

aduh95 reviewed Jul 12, 2026

View reviewed changes

aduh95 reviewed Jul 12, 2026

View reviewed changes

Renegade334 reviewed Jul 12, 2026

View reviewed changes

@@            Coverage Diff             @@
##             main   #64447      +/-   ##
==========================================
+ Coverage   90.24%   90.29%   +0.04%     
==========================================
  Files         741      760      +19     
  Lines      241384   247324    +5940     
  Branches    45480    46652    +1172     
==========================================
+ Hits       217844   223310    +5466     
- Misses      15097    15477     +380     
- Partials     8443     8537      +94     

mertcanaltin reviewed Jul 12, 2026

View reviewed changes

mertcanaltin reviewed Jul 12, 2026

View reviewed changes

gurgunday reviewed Jul 12, 2026

View reviewed changes

gurgunday reviewed Jul 12, 2026

View reviewed changes

jasnell reviewed Jul 12, 2026

View reviewed changes

gurgunday approved these changes Jul 13, 2026

View reviewed changes

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

mertcanaltin approved these changes Jul 13, 2026

View reviewed changes

MattiasBuelens reviewed Jul 15, 2026

View reviewed changes

MattiasBuelens approved these changes Jul 15, 2026

View reviewed changes

gurgunday approved these changes Jul 15, 2026

View reviewed changes

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ronag approved these changes Aug 2, 2026

View reviewed changes

- Loading data for nodejs/node/pull/64447
✔  Done loading data for nodejs/node/pull/64447
----------------------------------- PR info ------------------------------------
Title      stream: speed up async iteration of Readable (#64447)
Author     Matteo Collina <[email protected]> (@mcollina)
Branch     mcollina:stream-async-iterator-perf -> nodejs:main
Labels     stream, author ready, needs-ci, commit-queue
Commits    4
 - stream: speed up async iteration of Readable
 - fixup: address review comments
 - fixup: remove [SymbolAsyncIterator]
 - stream: fix lint in readable async iterator
Committers 1
 - Matteo Collina <[email protected]>
PR-URL: https://github.com/nodejs/node/pull/64447
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Mattias Buelens <[email protected]>
Reviewed-By: Robert Nagy <[email protected]>
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/64447
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Mattias Buelens <[email protected]>
Reviewed-By: Robert Nagy <[email protected]>
--------------------------------------------------------------------------------
   ℹ  This PR was created on Sun, 12 Jul 2026 08:27:10 GMT
   ✔  Approvals: 3
   ✔  - Gürgün Dayıoğlu (@gurgunday): https://github.com/nodejs/node/pull/64447#pullrequestreview-4707049635
   ✔  - Mattias Buelens (@MattiasBuelens): https://github.com/nodejs/node/pull/64447#pullrequestreview-4706259826
   ✔  - Robert Nagy (@ronag) (TSC): https://github.com/nodejs/node/pull/64447#pullrequestreview-4838491319
   ✔  Last GitHub CI successful
   ℹ  Last Full PR CI on 2026-08-06T13:07:45Z: https://ci.nodejs.org/job/node-test-pull-request/75563/
- Querying data for job/node-test-pull-request/75563/
✔  Build data downloaded
   ✔  Last Jenkins CI successful
--------------------------------------------------------------------------------
   ✔  No git cherry-pick in progress
   ✔  No git am in progress
   ✔  No git rebase in progress
--------------------------------------------------------------------------------
- Bringing origin/main up to date...
From https://github.com/nodejs/node
 * branch                  main       -> FETCH_HEAD
✔  origin/main is now up-to-date
- Downloading patch for 64447
From https://github.com/nodejs/node
 * branch                  refs/pull/64447/merge -> FETCH_HEAD
✔  Fetched commits as 9e23066b8af4..873a47ef6ae8
--------------------------------------------------------------------------------
Auto-merging lib/internal/streams/readable.js
[main 0a0be7f763] stream: speed up async iteration of Readable
 Author: Matteo Collina <[email protected]>
 Date: Sat Jul 11 23:41:55 2026 +0200
 3 files changed, 307 insertions(+), 35 deletions(-)
Auto-merging lib/internal/streams/readable.js
[main 22401b1cb5] fixup: address review comments
 Author: Matteo Collina <[email protected]>
 Date: Mon Jul 13 10:00:44 2026 +0200
 3 files changed, 73 insertions(+), 26 deletions(-)
Auto-merging lib/internal/streams/readable.js
[main 0b186829dd] fixup: remove [SymbolAsyncIterator]
 Author: Matteo Collina <[email protected]>
 Date: Wed Jul 15 17:34:22 2026 +0200
 1 file changed, 1 insertion(+), 4 deletions(-)
Auto-merging lib/internal/streams/readable.js
[main 656bc697b3] stream: fix lint in readable async iterator
 Author: Matteo Collina <[email protected]>
 Date: Sun Aug 2 09:08:31 2026 +0000
 1 file changed, 1 insertion(+), 1 deletion(-)
   ✔  Patches applied
There are 4 commits in the PR. Attempting autorebase.
(node:388) [DEP0190] DeprecationWarning: Passing args to a child process with shell option true can lead to security vulnerabilities, as the arguments are not escaped, only concatenated.
(Use `node --trace-deprecation ...` to show where the warning was created)
Rebasing (2/8)
Executing: git node land --amend --yes
--------------------------------- New Message ----------------------------------
stream: speed up async iteration of Readable
Replace the async generator backing Symbol.asyncIterator with a

hand-rolled iterator. The generator machinery costs several extra

promise allocations and microtask hops per chunk: yield awaits the

yielded value and resolves the pending request through separate

promises. Buffered chunks are now delivered as an already-resolved

promise, one microtask sooner than before.
Thenable chunks are still awaited before delivery, requests received

while a next() is outstanding are queued, and return()/throw() before

the first next() complete the iterator without touching the stream.
The earlier delivery is observable by code racing an abort against

the first chunk. The flatMap AbortSignal test relied on such a race;

it is reworked to abort deterministically while two mappers are in

flight, asserting the concurrency limit, in-flight cancellation and

rejection, without depending on delivery timing or timers.
streams/readable-async-iterator.js sync='yes': +32.59% ()

streams/readable-async-iterator.js sync='no': +9.84% ()
Assisted-by: Claude Fable 5

Signed-off-by: Matteo Collina <[email protected]>

PR-URL: #64447

Reviewed-By: Gürgün Dayıoğlu <[email protected]>

Reviewed-By: Mattias Buelens <[email protected]>

Reviewed-By: Robert Nagy <[email protected]>
[detached HEAD 6e6bcb4f2f] stream: speed up async iteration of Readable

Author: Matteo Collina <[email protected]>

Date: Sat Jul 11 23:41:55 2026 +0200

3 files changed, 307 insertions(+), 35 deletions(-)

Rebasing (3/8)

Rebasing (4/8)

Executing: git node land --amend --yes

--------------------------------- New Message ----------------------------------

fixup: address review comments
Signed-off-by: Matteo Collina <[email protected]>

PR-URL: #64447

Reviewed-By: Gürgün Dayıoğlu <[email protected]>

Reviewed-By: Mattias Buelens <[email protected]>

Reviewed-By: Robert Nagy <[email protected]>
[detached HEAD a9e0e0ad92] fixup: address review comments

Author: Matteo Collina <[email protected]>

Date: Mon Jul 13 10:00:44 2026 +0200

3 files changed, 73 insertions(+), 26 deletions(-)

Rebasing (5/8)

Rebasing (6/8)

Executing: git node land --amend --yes

--------------------------------- New Message ----------------------------------

fixup: remove [SymbolAsyncIterator]
Signed-off-by: Matteo Collina <[email protected]>

PR-URL: #64447

Reviewed-By: Gürgün Dayıoğlu <[email protected]>

Reviewed-By: Mattias Buelens <[email protected]>

Reviewed-By: Robert Nagy <[email protected]>
[detached HEAD 0ea470ea7b] fixup: remove [SymbolAsyncIterator]

Author: Matteo Collina <[email protected]>

Date: Wed Jul 15 17:34:22 2026 +0200

1 file changed, 1 insertion(+), 4 deletions(-)

Rebasing (7/8)

Rebasing (8/8)

Executing: git node land --amend --yes

--------------------------------- New Message ----------------------------------

stream: fix lint in readable async iterator
Signed-off-by: Matteo Collina <[email protected]>

PR-URL: #64447

Reviewed-By: Gürgün Dayıoğlu <[email protected]>

Reviewed-By: Mattias Buelens <[email protected]>

Reviewed-By: Robert Nagy <[email protected]>
[detached HEAD 0d2b7c1daf] stream: fix lint in readable async iterator

Author: Matteo Collina <[email protected]>

Date: Sun Aug 2 09:08:31 2026 +0000

1 file changed, 1 insertion(+), 1 deletion(-)

Successfully rebased and updated refs/heads/main.
ℹ  Add commit-queue-squash label to land the PR as one commit, or commit-queue-rebase to land as separate commits.

86 checks passed

Replace the async generator backing Symbol.asyncIterator with a
hand-rolled iterator. The generator machinery costs several extra
promise allocations and microtask hops per chunk: yield awaits the
yielded value and resolves the pending request through separate
promises. Buffered chunks are now delivered as an already-resolved
promise, one microtask sooner than before.

Thenable chunks are still awaited before delivery, requests received
while a next() is outstanding are queued, and return()/throw() before
the first next() complete the iterator without touching the stream.

The earlier delivery is observable by code racing an abort against
the first chunk. The flatMap AbortSignal test relied on such a race;
it is reworked to abort deterministically while two mappers are in
flight, asserting the concurrency limit, in-flight cancellation and
rejection, without depending on delivery timing or timers.

streams/readable-async-iterator.js sync='yes': +32.59% (***)
streams/readable-async-iterator.js sync='no': +9.84% (***)

Assisted-by: Claude Fable 5
Signed-off-by: Matteo Collina <[email protected]>
PR-URL: #64447
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Mattias Buelens <[email protected]>
Reviewed-By: Robert Nagy <[email protected]>

Replace the async generator backing Symbol.asyncIterator with a
hand-rolled iterator. The generator machinery costs several extra
promise allocations and microtask hops per chunk: yield awaits the
yielded value and resolves the pending request through separate
promises. Buffered chunks are now delivered as an already-resolved
promise, one microtask sooner than before.

Thenable chunks are still awaited before delivery, requests received
while a next() is outstanding are queued, and return()/throw() before
the first next() complete the iterator without touching the stream.

The earlier delivery is observable by code racing an abort against
the first chunk. The flatMap AbortSignal test relied on such a race;
it is reworked to abort deterministically while two mappers are in
flight, asserting the concurrency limit, in-flight cancellation and
rejection, without depending on delivery timing or timers.

streams/readable-async-iterator.js sync='yes': +32.59% (***)
streams/readable-async-iterator.js sync='no': +9.84% (***)

Assisted-by: Claude Fable 5
Signed-off-by: Matteo Collina <[email protected]>
PR-URL: #64447
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Mattias Buelens <[email protected]>
Reviewed-By: Robert Nagy <[email protected]>

The hand-rolled async iterator introduced in
nodejs#64447 unwraps thenable chunks before
delivering them, but guards the `then` access with `chunk !== null` while
the dereference itself requires `chunk != null`. An object mode stream
carrying an `undefined` chunk therefore threw

  TypeError: Cannot read properties of undefined (reading 'then')

Because the throw happens in a microtask rather than rejecting the
iterator's promise, it surfaces as an uncaught exception that terminates
the process instead of a catchable stream error.

`undefined` is a legal chunk value: doc/api/stream.md documents object
mode chunks as "any JavaScript value other than `null`" and reserves
`null` as the end-of-stream sentinel. `readable.push(undefined)` returns
true in object mode, so the stream accepts the value and then crashes on
the way out.

Read `then` with optional chaining at both sites so that `undefined` is
delivered as a value while `null` keeps signalling end-of-stream. `then`
is still read at most once, so a getter cannot observe a second access.

Refs: nodejs#64447

The hand-rolled async iterator introduced in
nodejs#64447 unwraps thenable chunks
before delivering them, but guards the `then` access with
`chunk !== null` while the dereference itself requires `chunk != null`.
An object mode stream carrying an `undefined` chunk therefore threw

  TypeError: Cannot read properties of undefined (reading 'then')

Because the throw happens in a microtask rather than rejecting the
iterator's promise, it surfaces as an uncaught exception that
terminates the process instead of a catchable stream error.

`undefined` is a legal chunk value: doc/api/stream.md documents object
mode chunks as "any JavaScript value other than `null`" and reserves
`null` as the end-of-stream sentinel. `readable.push(undefined)`
returns true in object mode, so the stream accepts the value and then
crashes on the way out.

Read `then` with optional chaining at both sites so that `undefined` is
delivered as a value while `null` keeps signalling end-of-stream.
`then` is still read at most once, so a getter cannot observe a second
access.

Refs: nodejs#64447

The hand-rolled async iterator introduced in
nodejs#64447 unwraps thenable chunks
before delivering them, but guards the `then` access with
`chunk !== null` while the dereference itself requires `chunk != null`.
An object mode stream carrying an `undefined` chunk therefore threw

  TypeError: Cannot read properties of undefined (reading 'then')

Because the throw happens in a microtask rather than rejecting the
iterator's promise, it surfaces as an uncaught exception that
terminates the process instead of a catchable stream error.

`undefined` is a legal chunk value: doc/api/stream.md documents object
mode chunks as "any JavaScript value other than `null`" and reserves
`null` as the end-of-stream sentinel. `readable.push(undefined)`
returns true in object mode, so the stream accepts the value and then
crashes on the way out.

Read `then` with optional chaining at both sites so that `undefined` is
delivered as a value while `null` keeps signalling end-of-stream.
`then` is still read at most once, so a getter cannot observe a second
access.

Refs: nodejs#64447
Signed-off-by: Caleb ツ Everett <[email protected]>

The hand-rolled async iterator introduced in
nodejs#64447 unwraps thenable chunks
before delivering them, but guards the `then` access with
`chunk !== null` while the dereference itself requires `chunk != null`.
An object mode stream carrying an `undefined` chunk therefore threw

  TypeError: Cannot read properties of undefined (reading 'then')

Because the throw happens in a microtask rather than rejecting the
iterator's promise, it surfaces as an uncaught exception that
terminates the process instead of a catchable stream error.

`undefined` is a legal chunk value: doc/api/stream.md documents object
mode chunks as "any JavaScript value other than `null`" and reserves
`null` as the end-of-stream sentinel. `readable.push(undefined)`
returns true in object mode, so the stream accepts the value and then
crashes on the way out.

Read `then` with optional chaining at both sites so that `undefined` is
delivered as a value while `null` keeps signalling end-of-stream.
`then` is still read at most once, so a getter cannot observe a second
access.

Refs: nodejs#64447
Assisted-by: a closed-source coding agent
Signed-off-by: Caleb ツ Everett <[email protected]>

The hand-rolled async iterator introduced in
#64447 unwraps thenable chunks
before delivering them, but guards the `then` access with
`chunk !== null` while the dereference itself requires `chunk != null`.
An object mode stream carrying an `undefined` chunk therefore threw

  TypeError: Cannot read properties of undefined (reading 'then')

Because the throw happens in a microtask rather than rejecting the
iterator's promise, it surfaces as an uncaught exception that
terminates the process instead of a catchable stream error.

`undefined` is a legal chunk value: doc/api/stream.md documents object
mode chunks as "any JavaScript value other than `null`" and reserves
`null` as the end-of-stream sentinel. `readable.push(undefined)`
returns true in object mode, so the stream accepts the value and then
crashes on the way out.

Read `then` with optional chaining at both sites so that `undefined` is
delivered as a value while `null` keeps signalling end-of-stream.
`then` is still read at most once, so a getter cannot observe a second
access.

Refs: #64447
Assisted-by: a closed-source coding agent
Signed-off-by: Caleb ツ Everett <[email protected]>
PR-URL: #65969
Reviewed-By: Xuguang Mei <[email protected]>
Reviewed-By: James M Snell <[email protected]>

The hand-rolled async iterator introduced in
#64447 unwraps thenable chunks
before delivering them, but guards the `then` access with
`chunk !== null` while the dereference itself requires `chunk != null`.
An object mode stream carrying an `undefined` chunk therefore threw

  TypeError: Cannot read properties of undefined (reading 'then')

Because the throw happens in a microtask rather than rejecting the
iterator's promise, it surfaces as an uncaught exception that
terminates the process instead of a catchable stream error.

`undefined` is a legal chunk value: doc/api/stream.md documents object
mode chunks as "any JavaScript value other than `null`" and reserves
`null` as the end-of-stream sentinel. `readable.push(undefined)`
returns true in object mode, so the stream accepts the value and then
crashes on the way out.

Read `then` with optional chaining at both sites so that `undefined` is
delivered as a value while `null` keeps signalling end-of-stream.
`then` is still read at most once, so a getter cannot observe a second
access.

Refs: #64447
Assisted-by: a closed-source coding agent
Signed-off-by: Caleb ツ Everett <[email protected]>
PR-URL: #65969
Reviewed-By: Xuguang Mei <[email protected]>
Reviewed-By: James M Snell <[email protected]>