GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — lib: use utf8 fast path for streaming TextDecoder by ChALkeR · Pull Request #61549 · nodejs/node

Conversation

@ChALkeR

@ChALkeR ChALkeR commented Jan 27, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Tracking: #61041

A continuation of #61409

This is based on the logic in https://github.com/ExodusOSS/bytes

Unifies stream and non-stream codepath for UTF-8 both for intl and no-intl variants.

Previously, ICU and string_decoder were used on with-intl and without-intl variants correspondingly for streaming UTF-8 implementation

Instead, just do minor quick slices on JS side and use the single stateless native decoding API to be fast on large chunks

This:

  1. Improves streaming UTF-8 TextDecoder performance
  2. Brings streaming fatal UTF-8 TextDecoder support to without-intl builds
  3. Unifies the logic, using a single codepath everywhere for UTF-8

Benchmarks

Previously: #61131

Non-chunked, 25.5.0

Test Size Throughput Mean Time
Latin lipsum (ASCII) 84.902 KiB 36.69 GiB/s 0.002 ms
Arabic lipsum 79.771 KiB 2.03 GiB/s 0.037 ms
Chinese lipsum 68.203 KiB 4.10 GiB/s 0.016 ms
Arabic + 2 * ASCII 249.577 KiB 2.72 GiB/s 0.092 ms
Non-ASCII char + ASCII 84.905 KiB 3.84 GiB/s 0.024 ms

Non-chunked, PR

Test Size Throughput Mean Time
Latin lipsum (ASCII) 84.902 KiB 36.19 GiB/s 0.002 ms
Arabic lipsum 79.771 KiB 2.02 GiB/s 0.038 ms
Chinese lipsum 68.203 KiB 4.00 GiB/s 0.016 ms
Arabic + 2 * ASCII 249.577 KiB 3.32 GiB/s 0.075 ms
Non-ASCII char + ASCII 84.905 KiB 4.71 GiB/s 0.022 ms

(should not be affected)

Chunked (1000 byte chunks), 25.5.0

Test Size Throughput Mean Time
Latin lipsum (ASCII) 84.902 KiB 1.31 GiB/s 0.062 ms
Arabic lipsum 79.771 KiB 0.90 GiB/s 0.084 ms
Chinese lipsum 68.203 KiB 0.99 GiB/s 0.066 ms
Arabic + 2 * ASCII 249.577 KiB 1.14 GiB/s 0.209 ms
Non-ASCII char + ASCII 84.905 KiB 1.31 GiB/s 0.062 ms

Chunked (1000 byte chunks), PR

Test Size Throughput Mean Time
Latin lipsum (ASCII) 84.902 KiB 7.45 GiB/s 0.011 ms
Arabic lipsum 79.771 KiB 1.24 GiB/s 0.062 ms
Chinese lipsum 68.203 KiB 1.54 GiB/s 0.042 ms
Arabic + 2 * ASCII 249.577 KiB 2.67 GiB/s 0.091 ms
Non-ASCII char + ASCII 84.905 KiB 7.26 GiB/s 0.011 ms

The benchmark creates about ~70-256 chunks on each test
The improvement is significant

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bricss reacted with rocket emoji
All reactions
  • 🚀 1 reaction
@nodejs-github-bot nodejs-github-bot added encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. needs-ci PRs that need a full CI run. labels Jan 27, 2026
@ChALkeR
ChALkeR force-pushed the chalker/decoder/unify/2 branch from a61503c to 4331d20 Compare January 27, 2026 18:18
@ChALkeR ChALkeR mentioned this pull request Jan 27, 2026
lib: unify ICU and no-ICU TextDecoder #61409
Merged
@ChALkeR
ChALkeR force-pushed the chalker/decoder/unify/2 branch from 4331d20 to 916cd32 Compare January 27, 2026 18:25
anonrig
anonrig reviewed Jan 27, 2026
View reviewed changes
Comment thread lib/internal/encoding/util.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

@anonrig anonrig added the needs-benchmark-ci PRs that need a benchmark CI run. label Jan 27, 2026
@ChALkeR
ChALkeR force-pushed the chalker/decoder/unify/2 branch 5 times, most recently from 9ebe8ce to 111b99e Compare January 27, 2026 19:22
@ChALkeR
ChALkeR requested a review from anonrig January 27, 2026 19:23
@ChALkeR
ChALkeR marked this pull request as ready for review January 27, 2026 19:25
@ChALkeR
ChALkeR force-pushed the chalker/decoder/unify/2 branch 3 times, most recently from 1a77e7a to cd5c966 Compare January 27, 2026 20:12
@ChALkeR
lib: use utf8 fast path for streaming TextDecoder
e9e9252
@ChALkeR
ChALkeR force-pushed the chalker/decoder/unify/2 branch from cd5c966 to e9e9252 Compare January 27, 2026 20:13
@anonrig

anonrig commented Jan 27, 2026

Copy link
Copy Markdown
Member

@ChALkeR can you run benchmark CI that applies to this pull-request?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ChALkeR

ChALkeR commented Jan 27, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

@anonrig I can't do anything until #61553

Also, we don't have a stream: true TextDecoder benchmark.
The bench results above are from https://github.com/lemire/jstextdecoderbench with this modification:

function decodeChunked(bytes) {
  const chunk = 1000
  const max = bytes.length - chunk
  let i = 0
  for (; i < max; i += chunk) decoder.decode(bytes.subarray(i, i + chunk), { stream: true })
  decoder.decode(bytes.subarray(i))
}

And using that instead of decoder.decode

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

anonrig
anonrig approved these changes Jan 27, 2026
View reviewed changes

@anonrig anonrig left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the kind responses and amazing work.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ChALkeR reacted with heart emoji
All reactions
  • ❤️ 1 reaction
@codecov

codecov Bot commented Jan 27, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.77%. Comparing base (e155415) to head (1559954).
⚠️ Report is 38 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff            @@
##             main   #61549    +/-   ##
========================================
  Coverage   89.77%   89.77%            
========================================
  Files         672      673     +1     
  Lines      203755   203875   +120     
  Branches    39167    39190    +23     
========================================
+ Hits       182922   183038   +116     
- Misses      13164    13172     +8     
+ Partials     7669     7665     -4     
Files with missing lines Coverage Δ
lib/internal/encoding.js 100.00% <100.00%> (ø)
lib/internal/encoding/util.js 100.00% <100.00%> (ø)

... and 33 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ChALkeR ChALkeR added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Jan 27, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Jan 27, 2026
@nodejs-github-bot

nodejs-github-bot commented Jan 27, 2026

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/71082/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ChALkeR
benchmark: add streaming TextDecoder benchmark
1559954
@ChALkeR

ChALkeR commented Jan 27, 2026

Copy link
Copy Markdown
Member Author

@anonrig I added a benchmark for streaming Unicode TextDecoder.
Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1787

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ChALkeR ChALkeR added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Jan 27, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Jan 27, 2026
@nodejs-github-bot

nodejs-github-bot commented Feb 2, 2026

Copy link
Copy Markdown
Collaborator
Commit Queue failed
- Loading data for nodejs/node/pull/61549
✔  Done loading data for nodejs/node/pull/61549
----------------------------------- PR info ------------------------------------
Title      lib: use utf8 fast path for streaming TextDecoder (#61549)
Author     Nikita Skovoroda <[email protected]> (@ChALkeR)
Branch     ChALkeR:chalker/decoder/unify/2 -> nodejs:main
Labels     performance, encoding, author ready, needs-ci, needs-benchmark-ci
Commits    2
 - lib: use utf8 fast path for streaming TextDecoder
 - benchmark: add streaming TextDecoder benchmark
Committers 1
 - Nikita Skovoroda <[email protected]>
PR-URL: https://github.com/nodejs/node/pull/61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
--------------------------------------------------------------------------------
   ℹ  This PR was created on Tue, 27 Jan 2026 18:17:16 GMT
   ✔  Approvals: 3
   ✔  - Yagiz Nizipli (@anonrig) (TSC): https://github.com/nodejs/node/pull/61549#pullrequestreview-3713460238
   ✔  - Gürgün Dayıoğlu (@gurgunday): https://github.com/nodejs/node/pull/61549#pullrequestreview-3715962381
   ✔  - Colin Ihrig (@cjihrig): https://github.com/nodejs/node/pull/61549#pullrequestreview-3717219044
   ✔  Last GitHub CI successful
   ℹ  Last Full PR CI on 2026-01-28T15:17:24Z: https://ci.nodejs.org/job/node-test-pull-request/71102/
   ℹ  Last Benchmark CI on 2026-01-27T23:16:20Z: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1787
- Querying data for job/node-test-pull-request/71102/
✔  Build data downloaded
   ✔  Last Jenkins CI successful
--------------------------------------------------------------------------------
   ✔  No git cherry-pick in progress
   ✔  No git am in progress
   ✔  No git rebase in progress
--------------------------------------------------------------------------------
- Bringing origin/main up to date...
From https://github.com/nodejs/node
 * branch                  main       -> FETCH_HEAD
✔  origin/main is now up-to-date
- Downloading patch for 61549
From https://github.com/nodejs/node
 * branch                  refs/pull/61549/merge -> FETCH_HEAD
✔  Fetched commits as 6c44d31e2522..155995438c2a
--------------------------------------------------------------------------------
Auto-merging test/parallel/test-bootstrap-modules.js
[main d495fa4fc6] lib: use utf8 fast path for streaming TextDecoder
 Author: Nikita Skovoroda <[email protected]>
 Date: Tue Jan 27 22:12:49 2026 +0400
 4 files changed, 138 insertions(+), 47 deletions(-)
 create mode 100644 lib/internal/encoding/util.js
[main 8441e884d9] benchmark: add streaming TextDecoder benchmark
 Author: Nikita Skovoroda <[email protected]>
 Date: Wed Jan 28 03:08:19 2026 +0400
 1 file changed, 55 insertions(+)
 create mode 100644 benchmark/util/text-decoder-stream.js
   ✔  Patches applied
There are 2 commits in the PR. Attempting autorebase.
(node:2357) [DEP0190] DeprecationWarning: Passing args to a child process with shell option true can lead to security vulnerabilities, as the arguments are not escaped, only concatenated.
(Use `node --trace-deprecation ...` to show where the warning was created)
Rebasing (2/4)
Executing: git node land --amend --yes
--------------------------------- New Message ----------------------------------
lib: use utf8 fast path for streaming TextDecoder

PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>

[detached HEAD d9f44d2f20] lib: use utf8 fast path for streaming TextDecoder
Author: Nikita Skovoroda <[email protected]>
Date: Tue Jan 27 22:12:49 2026 +0400
4 files changed, 138 insertions(+), 47 deletions(-)
create mode 100644 lib/internal/encoding/util.js
Rebasing (3/4)
Rebasing (4/4)
Executing: git node land --amend --yes
--------------------------------- New Message ----------------------------------
benchmark: add streaming TextDecoder benchmark

PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>

[detached HEAD 7160f13a63] benchmark: add streaming TextDecoder benchmark
Author: Nikita Skovoroda <[email protected]>
Date: Wed Jan 28 03:08:19 2026 +0400
1 file changed, 55 insertions(+)
create mode 100644 benchmark/util/text-decoder-stream.js
Successfully rebased and updated refs/heads/main.

ℹ Add commit-queue-squash label to land the PR as one commit, or commit-queue-rebase to land as separate commits.

https://github.com/nodejs/node/actions/runs/21576063433

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ChALkeR ChALkeR added commit-queue-rebase PRs the Commit Queue should land as multiple self-contained commits. commit-queue PRs queued for automated landing through the Commit Queue. and removed commit-queue-failed PRs whose Commit Queue landing failed and need manual intervention before retrying. labels Feb 2, 2026
@nodejs-github-bot nodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Feb 2, 2026
@nodejs-github-bot

nodejs-github-bot commented Feb 2, 2026

Copy link
Copy Markdown
Collaborator

Landed in 6c44d31...8c57908

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

nodejs-github-bot pushed a commit that referenced this pull request Feb 2, 2026
@ChALkeR @nodejs-github-bot
lib: use utf8 fast path for streaming TextDecoder
e43ac89
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
nodejs-github-bot pushed a commit that referenced this pull request Feb 2, 2026
@ChALkeR @nodejs-github-bot
benchmark: add streaming TextDecoder benchmark
8c57908
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
@nodejs-github-bot nodejs-github-bot closed this Feb 2, 2026
aduh95 pushed a commit that referenced this pull request Feb 2, 2026
@ChALkeR @aduh95
lib: use utf8 fast path for streaming TextDecoder
f9e8761
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
aduh95 pushed a commit that referenced this pull request Feb 2, 2026
@ChALkeR @aduh95
benchmark: add streaming TextDecoder benchmark
d474916
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
@github-actions github-actions Bot mentioned this pull request Feb 2, 2026
2026-02-03, Version 25.6.0 (Current) #61635
Merged
aduh95 pushed a commit that referenced this pull request Feb 2, 2026
@ChALkeR @aduh95
lib: use utf8 fast path for streaming TextDecoder
4bea821
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
aduh95 pushed a commit that referenced this pull request Feb 2, 2026
@ChALkeR @aduh95
benchmark: add streaming TextDecoder benchmark
d23ee89
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
aduh95 pushed a commit that referenced this pull request Feb 10, 2026
@ChALkeR @aduh95
lib: use utf8 fast path for streaming TextDecoder
84a597a
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
aduh95 pushed a commit that referenced this pull request Feb 10, 2026
@ChALkeR @aduh95
benchmark: add streaming TextDecoder benchmark
0f57695
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
aduh95 pushed a commit that referenced this pull request Feb 14, 2026
@ChALkeR @aduh95
lib: use utf8 fast path for streaming TextDecoder
c44e017
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
aduh95 pushed a commit that referenced this pull request Feb 14, 2026
@ChALkeR @aduh95
benchmark: add streaming TextDecoder benchmark
9975bd8
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
@github-actions github-actions Bot mentioned this pull request Feb 22, 2026
2026-02-24, Version 24.14.0 'Krypton' (LTS) #61924
Merged
aduh95 pushed a commit that referenced this pull request Feb 22, 2026
@ChALkeR @aduh95
lib: use utf8 fast path for streaming TextDecoder
6df8be4
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
aduh95 pushed a commit that referenced this pull request Feb 22, 2026
@ChALkeR @aduh95
benchmark: add streaming TextDecoder benchmark
83bcd38
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
juanarbol pushed a commit to juanarbol/node that referenced this pull request Jul 16, 2026
@ChALkeR @juanarbol
benchmark: add streaming TextDecoder benchmark
c858f47
PR-URL: nodejs#61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
@juanarbol juanarbol mentioned this pull request Jul 16, 2026
V22.24.0 proposal #64533
Closed
juanarbol pushed a commit that referenced this pull request Jul 17, 2026
@ChALkeR @juanarbol
benchmark: add streaming TextDecoder benchmark
087971e
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
@juanarbol juanarbol mentioned this pull request Jul 17, 2026
V22.23.2 proposal #64567
Closed
juanarbol pushed a commit that referenced this pull request Jul 19, 2026
@ChALkeR @juanarbol
benchmark: add streaming TextDecoder benchmark
9310acb
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
marco-ippolito pushed a commit that referenced this pull request Jul 29, 2026
@ChALkeR @marco-ippolito
benchmark: add streaming TextDecoder benchmark
08e8adf
PR-URL: #61549
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Gürgün Dayıoğlu <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@anonrig anonrig anonrig approved these changes
@cjihrig cjihrig cjihrig approved these changes
@gurgunday gurgunday gurgunday approved these changes
@richardlau richardlau Awaiting requested review from richardlau
@avivkeller avivkeller Awaiting requested review from avivkeller

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue-rebase PRs the Commit Queue should land as multiple self-contained commits. encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. needs-benchmark-ci PRs that need a benchmark CI run. needs-ci PRs that need a full CI run. performance Issues and PRs related to the performance of Node.js.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@ChALkeR @anonrig @nodejs-github-bot @cjihrig @gurgunday @RafaelGSS
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.