GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — lib: use utf8 fast path for streaming TextDecoder by ChALkeR · Pull Request #61549 · nodejs/node Conversation
|
| Test | Size | Throughput | Mean Time |
|---|---|---|---|
| Latin lipsum (ASCII) | 84.902 KiB | 36.69 GiB/s | 0.002 ms |
| Arabic lipsum | 79.771 KiB | 2.03 GiB/s | 0.037 ms |
| Chinese lipsum | 68.203 KiB | 4.10 GiB/s | 0.016 ms |
| Arabic + 2 * ASCII | 249.577 KiB | 2.72 GiB/s | 0.092 ms |
| Non-ASCII char + ASCII | 84.905 KiB | 3.84 GiB/s | 0.024 ms |
| Test | Size | Throughput | Mean Time |
|---|---|---|---|
| Latin lipsum (ASCII) | 84.902 KiB | 36.19 GiB/s | 0.002 ms |
| Arabic lipsum | 79.771 KiB | 2.02 GiB/s | 0.038 ms |
| Chinese lipsum | 68.203 KiB | 4.00 GiB/s | 0.016 ms |
| Arabic + 2 * ASCII | 249.577 KiB | 3.32 GiB/s | 0.075 ms |
| Non-ASCII char + ASCII | 84.905 KiB | 4.71 GiB/s | 0.022 ms |
(should not be affected)
| Test | Size | Throughput | Mean Time |
|---|---|---|---|
| Latin lipsum (ASCII) | 84.902 KiB | 1.31 GiB/s | 0.062 ms |
| Arabic lipsum | 79.771 KiB | 0.90 GiB/s | 0.084 ms |
| Chinese lipsum | 68.203 KiB | 0.99 GiB/s | 0.066 ms |
| Arabic + 2 * ASCII | 249.577 KiB | 1.14 GiB/s | 0.209 ms |
| Non-ASCII char + ASCII | 84.905 KiB | 1.31 GiB/s | 0.062 ms |
| Test | Size | Throughput | Mean Time |
|---|---|---|---|
| Latin lipsum (ASCII) | 84.902 KiB | 7.45 GiB/s | 0.011 ms |
| Arabic lipsum | 79.771 KiB | 1.24 GiB/s | 0.062 ms |
| Chinese lipsum | 68.203 KiB | 1.54 GiB/s | 0.042 ms |
| Arabic + 2 * ASCII | 249.577 KiB | 2.67 GiB/s | 0.091 ms |
| Non-ASCII char + ASCII | 84.905 KiB | 7.26 GiB/s | 0.011 ms |
The benchmark creates about ~70-256 chunks on each test
The improvement is significant
Sorry, something went wrong.
There was an error while loading. Please reload this page.
a61503c to
4331d20
Compare
4331d20 to
916cd32
Compare
There was an error while loading. Please reload this page.
9ebe8ce to
111b99e
Compare
1a77e7a to
cd5c966
Compare
cd5c966 to
e9e9252
Compare
|
@ChALkeR can you run benchmark CI that applies to this pull-request? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
@anonrig I can't do anything until #61553 Also, we don't have a function decodeChunked(bytes) { const chunk = 1000 const max = bytes.length - chunk let i = 0 for (; i < max; i += chunk) decoder.decode(bytes.subarray(i, i + chunk), { stream: true }) decoder.decode(bytes.subarray(i)) } And using that instead of |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the kind responses and amazing work.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@ ## main #61549 +/- ## ======================================== Coverage 89.77% 89.77% ======================================== Files 672 673 +1 Lines 203755 203875 +120 Branches 39167 39190 +23 ======================================== + Hits 182922 183038 +116 - Misses 13164 13172 +8 + Partials 7669 7665 -4
🚀 New features to boost your workflow:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@anonrig I added a benchmark for streaming Unicode TextDecoder. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Commit Queue failed- Loading data for nodejs/node/pull/61549 ✔ Done loading data for nodejs/node/pull/61549 ----------------------------------- PR info ------------------------------------ Title lib: use utf8 fast path for streaming TextDecoder (#61549) Author Nikita Skovoroda <[email protected]> (@ChALkeR) Branch ChALkeR:chalker/decoder/unify/2 -> nodejs:main Labels performance, encoding, author ready, needs-ci, needs-benchmark-ci Commits 2 - lib: use utf8 fast path for streaming TextDecoder - benchmark: add streaming TextDecoder benchmark Committers 1 - Nikita Skovoroda <[email protected]> PR-URL: https://github.com/nodejs/node/pull/61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> ------------------------------ Generated metadata ------------------------------ PR-URL: https://github.com/nodejs/node/pull/61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> -------------------------------------------------------------------------------- ℹ This PR was created on Tue, 27 Jan 2026 18:17:16 GMT ✔ Approvals: 3 ✔ - Yagiz Nizipli (@anonrig) (TSC): https://github.com/nodejs/node/pull/61549#pullrequestreview-3713460238 ✔ - Gürgün Dayıoğlu (@gurgunday): https://github.com/nodejs/node/pull/61549#pullrequestreview-3715962381 ✔ - Colin Ihrig (@cjihrig): https://github.com/nodejs/node/pull/61549#pullrequestreview-3717219044 ✔ Last GitHub CI successful ℹ Last Full PR CI on 2026-01-28T15:17:24Z: https://ci.nodejs.org/job/node-test-pull-request/71102/ ℹ Last Benchmark CI on 2026-01-27T23:16:20Z: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1787 - Querying data for job/node-test-pull-request/71102/ ✔ Build data downloaded ✔ Last Jenkins CI successful -------------------------------------------------------------------------------- ✔ No git cherry-pick in progress ✔ No git am in progress ✔ No git rebase in progress -------------------------------------------------------------------------------- - Bringing origin/main up to date... From https://github.com/nodejs/node * branch main -> FETCH_HEAD ✔ origin/main is now up-to-date - Downloading patch for 61549 From https://github.com/nodejs/node * branch refs/pull/61549/merge -> FETCH_HEAD ✔ Fetched commits as 6c44d31e2522..155995438c2a -------------------------------------------------------------------------------- Auto-merging test/parallel/test-bootstrap-modules.js [main d495fa4fc6] lib: use utf8 fast path for streaming TextDecoder Author: Nikita Skovoroda <[email protected]> Date: Tue Jan 27 22:12:49 2026 +0400 4 files changed, 138 insertions(+), 47 deletions(-) create mode 100644 lib/internal/encoding/util.js [main 8441e884d9] benchmark: add streaming TextDecoder benchmark Author: Nikita Skovoroda <[email protected]> Date: Wed Jan 28 03:08:19 2026 +0400 1 file changed, 55 insertions(+) create mode 100644 benchmark/util/text-decoder-stream.js ✔ Patches applied There are 2 commits in the PR. Attempting autorebase. (node:2357) [DEP0190] DeprecationWarning: Passing args to a child process with shell option true can lead to security vulnerabilities, as the arguments are not escaped, only concatenated. (Use `node --trace-deprecation ...` to show where the warning was created) Rebasing (2/4) Executing: git node land --amend --yes --------------------------------- New Message ---------------------------------- lib: use utf8 fast path for streaming TextDecoderhttps://github.com/nodejs/node/actions/runs/21576063433 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Landed in 6c44d31...8c57908 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: nodejs#61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
PR-URL: #61549 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Gürgün Dayıoğlu <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.