GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — stream: reduce webstreams encoding and iteration overhead by mcollina · Pull Request #65414 · nodejs/node

Conversation

@mcollina

@mcollina mcollina commented Aug 20, 2026

Copy link
Copy Markdown
Member

This PR is stacked on #65143 — please review the last two commits only.

Two independent optimizations for WHATWG streams:

TextEncoderStream: encode whole chunks natively. The encode-and-enqueue transform walked every code unit in JS, materializing a one-character string per index and building the output via string concatenation. The only state that crosses chunks is a trailing high (leading) surrogate, and TextEncoder.encode()'s USVString conversion already replaces interior lone surrogates with U+FFFD exactly like the spec loop. The transform now joins a pending high surrogate, holds back a new trailing one, and encodes the rest in a single native call. Verified byte-identical to the previous algorithm over 200k randomized surrogate-heavy chunk sequences, plus the full WPT encoding suite. The streaming decode path also stops allocating a { stream: true } options object per chunk.

ReadableStream.from(): drop the async pull machinery. The pull algorithm was an async function awaiting iterator.next() and then the value — an async frame, two await wrappers, and a controller-side reaction per chunk. It is now callback-style with per-stream cached reaction steps, delivering completion directly to the controller's cached pull reactions (the parked-algorithm-result contract from #65143). Thenable adoption is preserved, including the observable .then lookup on plain object values. The iterator's next method is now looked up once at setup, matching the spec's GetIteratorDirect.

Benchmark results (benchmark/compare.js --runs 10, new rows added since the suite covered neither path):

                                                                confidence improvement accuracy (*)    (**)   (***)
webstreams/encoding-streams.js len=1024 kind='encode' n=100000        ***    546.04 %      ±16.30% ±23.40% ±34.38%
webstreams/encoding-streams.js len=16 kind='encode' n=100000          ***     19.93 %       ±4.31%  ±5.96%  ±8.26%
webstreams/from.js kind='sync' n=1000000                              ***     27.39 %       ±2.15%  ±2.96%  ±4.06%
webstreams/from.js kind='async' n=1000000                             ***     23.30 %       ±2.74%  ±3.76%  ±5.13%

All other webstreams rows (pipe-to ×9, pipe-through, read/read-buffered, async-iterator, tee, creation, js_transfer, decode) are unchanged. Full WPT streams + encoding suites and the parallel webstreams/whatwg test batches pass.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@nodejs-github-bot

nodejs-github-bot commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/performance
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. web streams Issues and PRs related to the Web Streams API. labels Aug 20, 2026
@XinGOfCloude18

XinGOfCloude18 commented Aug 20, 2026

Copy link
Copy Markdown

Review requested:

  • @nodejs/performance
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@codecov

codecov Bot commented Aug 20, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 41.83673% with 57 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.10%. Comparing base (9677349) to head (06951a7).
⚠️ Report is 83 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/webstreams/readablestream.js 24.00% 57 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #65414      +/-   ##
==========================================
- Coverage   90.12%   90.10%   -0.02%     
==========================================
  Files         752      751       -1     
  Lines      252317   252490     +173     
  Branches    47435    47464      +29     
==========================================
+ Hits       227401   227508     +107     
- Misses      16209    16308      +99     
+ Partials     8707     8674      -33     
Files with missing lines Coverage Δ
lib/internal/webstreams/encoding.js 100.00% <100.00%> (ø)
lib/internal/webstreams/util.js 96.48% <100.00%> (ø)
lib/internal/webstreams/readablestream.js 87.20% <24.00%> (-1.08%) ⬇️

... and 47 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jasnell
jasnell reviewed Aug 20, 2026
View reviewed changes
Comment thread lib/internal/webstreams/util.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

@mcollina
mcollina force-pushed the webstream-perf-round14 branch 2 times, most recently from fde1b0b to 4a02249 Compare August 20, 2026 17:29
gurgunday
gurgunday approved these changes Aug 20, 2026
View reviewed changes

@gurgunday gurgunday left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
mcollina added 2 commits August 21, 2026 03:02
@mcollina
stream: encode whole chunks in TextEncoderStream
3650132
The encode-and-enqueue transform walked the chunk code unit by code
unit, materializing a single-character string per index and building
the output with string concatenation. The only state that crosses
chunks is a trailing high (leading) surrogate, and TextEncoder.encode's
USVString conversion already replaces every interior lone surrogate
with U+FFFD, which is exactly what the spec loop produces. Join a
pending high surrogate with the incoming chunk, hold back a new
trailing high surrogate, and encode the rest in a single native call.

The streaming decode path also reuses a single options object instead
of allocating { stream: true } per chunk.

An encoding-streams benchmark is added since the suite had no
TextEncoderStream/TextDecoderStream row. Encoding improves by ~546%
with 1KB string chunks and ~20% with 16-character chunks; decode is
unchanged.

Signed-off-by: Matteo Collina <[email protected]>
@mcollina
stream: avoid async pull machinery in ReadableStream.from
c8e5339
The pull algorithm was an async function that awaited iterator.next()
and then the produced value, costing an async-function frame plus two
await wrappers and a controller-side reaction per chunk. Rewrite it
callback-style: the next() result is adopted exactly like the previous
awaits (including the observable .then lookup on plain object values),
the reaction steps are created once per stream, and completion is
delivered straight to the controller's cached pull reactions using the
parked-algorithm-result contract. The iterator's next method is also
looked up once at setup, per the spec's GetIteratorDirect.

A from benchmark is added since the suite had no ReadableStream.from
row. Iterating a stream built from a sync generator improves by ~27%
and from an async generator by ~23%; all other rows are unchanged.

Signed-off-by: Matteo Collina <[email protected]>
@mcollina
mcollina force-pushed the webstream-perf-round14 branch from 4a02249 to c8e5339 Compare August 21, 2026 01:08
anonrig
anonrig approved these changes Aug 21, 2026
View reviewed changes
@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Aug 22, 2026
@mcollina
stream: use a Symbol for kParkedAlgorithmResult
06951a7
The parked-result sentinel is only ever compared by identity, so a
Symbol describes its purpose better than an empty null-prototype
object.

Signed-off-by: Matteo Collina <[email protected]>
anonrig
anonrig approved these changes Aug 22, 2026
View reviewed changes
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Aug 22, 2026
@nodejs-github-bot

nodejs-github-bot commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/76243/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jasnell
jasnell requested a review from MattiasBuelens August 22, 2026 23:47
MattiasBuelens
MattiasBuelens approved these changes Aug 23, 2026
View reviewed changes
@nodejs-github-bot

nodejs-github-bot commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/76306/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot

nodejs-github-bot commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/76307/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mcollina mcollina added author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue PRs queued for automated landing through the Commit Queue. commit-queue-squash PRs the Commit Queue should land as one squashed commit. and removed needs-ci PRs that need a full CI run. labels Aug 24, 2026
@nodejs-github-bot
nodejs-github-bot merged commit f2c7f1b into nodejs:main Aug 24, 2026
76 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot

nodejs-github-bot commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Landed in f2c7f1b

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot nodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Aug 24, 2026
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
@mcollina @aduh95
stream: encode whole chunks in TextEncoderStream
35a63ae
The encode-and-enqueue transform walked the chunk code unit by code
unit, materializing a single-character string per index and building
the output with string concatenation. The only state that crosses
chunks is a trailing high (leading) surrogate, and TextEncoder.encode's
USVString conversion already replaces every interior lone surrogate
with U+FFFD, which is exactly what the spec loop produces. Join a
pending high surrogate with the incoming chunk, hold back a new
trailing high surrogate, and encode the rest in a single native call.

The streaming decode path also reuses a single options object instead
of allocating { stream: true } per chunk.

An encoding-streams benchmark is added since the suite had no
TextEncoderStream/TextDecoderStream row. Encoding improves by ~546%
with 1KB string chunks and ~20% with 16-character chunks; decode is
unchanged.

Signed-off-by: Matteo Collina <[email protected]>
PR-URL: #65414
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Mattias Buelens <[email protected]>
aduh95 pushed a commit that referenced this pull request Aug 25, 2026
@mcollina @aduh95
stream: encode whole chunks in TextEncoderStream
1d6ec2d
The encode-and-enqueue transform walked the chunk code unit by code
unit, materializing a single-character string per index and building
the output with string concatenation. The only state that crosses
chunks is a trailing high (leading) surrogate, and TextEncoder.encode's
USVString conversion already replaces every interior lone surrogate
with U+FFFD, which is exactly what the spec loop produces. Join a
pending high surrogate with the incoming chunk, hold back a new
trailing high surrogate, and encode the rest in a single native call.

The streaming decode path also reuses a single options object instead
of allocating { stream: true } per chunk.

An encoding-streams benchmark is added since the suite had no
TextEncoderStream/TextDecoderStream row. Encoding improves by ~546%
with 1KB string chunks and ~20% with 16-character chunks; decode is
unchanged.

Signed-off-by: Matteo Collina <[email protected]>
PR-URL: #65414
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Mattias Buelens <[email protected]>
@github-actions github-actions Bot mentioned this pull request Aug 25, 2026
2026-08-26, Version 26.8.0 (Current) #65551
Merged
aduh95 pushed a commit that referenced this pull request Aug 27, 2026
@mcollina @aduh95
stream: encode whole chunks in TextEncoderStream
ad21f90
The encode-and-enqueue transform walked the chunk code unit by code
unit, materializing a single-character string per index and building
the output with string concatenation. The only state that crosses
chunks is a trailing high (leading) surrogate, and TextEncoder.encode's
USVString conversion already replaces every interior lone surrogate
with U+FFFD, which is exactly what the spec loop produces. Join a
pending high surrogate with the incoming chunk, hold back a new
trailing high surrogate, and encode the rest in a single native call.

The streaming decode path also reuses a single options object instead
of allocating { stream: true } per chunk.

An encoding-streams benchmark is added since the suite had no
TextEncoderStream/TextDecoderStream row. Encoding improves by ~546%
with 1KB string chunks and ~20% with 16-character chunks; decode is
unchanged.

Signed-off-by: Matteo Collina <[email protected]>
PR-URL: #65414
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Mattias Buelens <[email protected]>
@github-actions github-actions Bot mentioned this pull request Sep 7, 2026
2026-09-08, Version 24.21.0 'Krypton' (LTS) #65873
Merged
aduh95 pushed a commit that referenced this pull request Sep 7, 2026
@mcollina @aduh95
stream: encode whole chunks in TextEncoderStream
90f41b5
The encode-and-enqueue transform walked the chunk code unit by code
unit, materializing a single-character string per index and building
the output with string concatenation. The only state that crosses
chunks is a trailing high (leading) surrogate, and TextEncoder.encode's
USVString conversion already replaces every interior lone surrogate
with U+FFFD, which is exactly what the spec loop produces. Join a
pending high surrogate with the incoming chunk, hold back a new
trailing high surrogate, and encode the rest in a single native call.

The streaming decode path also reuses a single options object instead
of allocating { stream: true } per chunk.

An encoding-streams benchmark is added since the suite had no
TextEncoderStream/TextDecoderStream row. Encoding improves by ~546%
with 1KB string chunks and ~20% with 16-character chunks; decode is
unchanged.

Signed-off-by: Matteo Collina <[email protected]>
PR-URL: #65414
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Mattias Buelens <[email protected]>
aduh95 pushed a commit that referenced this pull request Sep 7, 2026
@mcollina @aduh95
stream: encode whole chunks in TextEncoderStream
107e96d
The encode-and-enqueue transform walked the chunk code unit by code
unit, materializing a single-character string per index and building
the output with string concatenation. The only state that crosses
chunks is a trailing high (leading) surrogate, and TextEncoder.encode's
USVString conversion already replaces every interior lone surrogate
with U+FFFD, which is exactly what the spec loop produces. Join a
pending high surrogate with the incoming chunk, hold back a new
trailing high surrogate, and encode the rest in a single native call.

The streaming decode path also reuses a single options object instead
of allocating { stream: true } per chunk.

An encoding-streams benchmark is added since the suite had no
TextEncoderStream/TextDecoderStream row. Encoding improves by ~546%
with 1KB string chunks and ~20% with 16-character chunks; decode is
unchanged.

Signed-off-by: Matteo Collina <[email protected]>
PR-URL: #65414
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Mattias Buelens <[email protected]>
This was referenced Oct 14, 2026
[DOCUMENTATION] TextEncoderStream in the recommended pipeline is slow Node before 24.21.0 / 26.8.0 TomStrepsil/replace-content-transformer#88
Closed
[87][88] Decode UTF-8 across Buffer chunks in the Node adapters, and document a faster encoder for the web pipeline TomStrepsil/replace-content-transformer#93
Merged
@github-actions github-actions Bot mentioned this pull request Nov 2, 2026
Release v4.5.0 MattiasBuelens/web-streams-polyfill#243
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jasnell jasnell jasnell left review comments
@MattiasBuelens MattiasBuelens MattiasBuelens approved these changes
@anonrig anonrig anonrig approved these changes
@gurgunday gurgunday gurgunday approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue-squash PRs the Commit Queue should land as one squashed commit. web streams Issues and PRs related to the Web Streams API.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@mcollina @nodejs-github-bot @XinGOfCloude18 @jasnell @MattiasBuelens @anonrig @gurgunday
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.