GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — benchmark: pre-generate data set for URL benchmarks and use WPT test data by joyeecheung · Pull Request #24302 · nodejs/node

Conversation

@joyeecheung

@joyeecheung joyeecheung commented Nov 11, 2018

Copy link
Copy Markdown
Member

benchmark: pre-generate data set for URL benchmarks

This patch:

  • Introduces common.bakeUrlData which can be used to pre-generate
    the data set for the URL benchmarks to loop through instead of
    looping over a constant.
  • Add the option to use WPT data in benchmarks for better diversity
    in the input
  • Add the option to benchmark URL parsing with base URLs (whatwg only)
  • Moves the data in benchmark/fixtures/url-inputs.js to
    benchmark/common.js

benchmark: support URL inputs in create-clientrequest

This patch adds the option in the create-clientrequest benchmark
to accept URL inputs (as strings or as URL objects) so we can
measure the impact of URL parsing in a more sophisticated use case.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
joyeecheung added 2 commits November 11, 2018 16:50
@joyeecheung
benchmark: pre-generate data set for URL benchmarks
c096557 
This patch:

- Introduces `common.bakeUrlData` which can be used to pre-generate
  the data set for the URL benchmarks to loop through instead of
  looping over a constant.
- Add the option to use WPT data in benchmarks for better diversity
  in the input
- Add the option to benchmark URL parsing with base URLs (whatwg only)
- Moves the data in `benchmark/fixtures/url-inputs.js` to
  `benchmark/common.js`
@joyeecheung
benchmark: support URL inputs in create-clientrequest
1ae8ab6 
This patch adds the option in the create-clientrequest benchmark
to accept URL inputs (as strings or as URL objects) so we can
measure the impact of URL parsing in a more sophisticated use case.
@joyeecheung joyeecheung added the whatwg-url Issues and PRs related to the WHATWG URL implementation. label Nov 11, 2018
@nodejs-github-bot

nodejs-github-bot commented Nov 11, 2018

Copy link
Copy Markdown
Collaborator

@joyeecheung build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/1576/pipeline
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@nodejs-github-bot nodejs-github-bot added benchmark Issues and PRs related to Node.js benchmarks and benchmarking infrastructure. http Issues and PRs related to the http subsystem. querystring Issues and PRs related to the built-in querystring module. test Issues and PRs related to Node.js core tests and test infrastructure. url Issues and PRs related to the legacy built-in url module. labels Nov 11, 2018
@joyeecheung

joyeecheung commented Nov 11, 2018

Copy link
Copy Markdown
Member Author

CI: https://ci.nodejs.org/job/node-test-pull-request/18529/

cc @TimothyGu @nodejs/url
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@joyeecheung joyeecheung mentioned this pull request Nov 11, 2018
url: make the context non-enumerable #24218
Closed
3 tasks done
mscdex
mscdex reviewed Nov 11, 2018
View reviewed changes
Comment thread benchmark/http/create-clientrequest.js
.startsWith('http://'));
const bench = common.createBenchmark(main, {
len: [1, 8, 16, 32, 64, 128],
n: [1e6]

@mscdex mscdex Nov 11, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is n being removed in general in this PR? If we're not going to be looping over the same input for each input, then we should at least loop n times over the entire set of inputs.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@joyeecheung joyeecheung Nov 11, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mscdex That should already been covered by the combination e and --runs (of the compare.js): the size of the input is dependent on e so to get a size of 1e6 we can just set e to roughly Math.log(1e6 / 200, 2) (all the data are pre-generated, so when e is 1 and type is long it will iterate over the same kind of string 400 times. Only the type wpt gives you a nonhomogenous array), but I don't think we actually need that many iterations for a single benchmark run, the n was set when Crankshaft existed, and currently I don't observe a significance in the value of the n once it's large enough (by default it's roughly 800/400 with/without base). Instead we can just use --runs to get more samples, but the default 30 is already a pretty good sample size (at least it's the magic number taught in statistics textbooks). So with the default setting for each input you run through the same code roughly 30 * 400 times with the same binary

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@joyeecheung

joyeecheung commented Nov 11, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

@mscdex BTW this is the benchmark results I got for #24218 with this benchmark after setting e to 1, 4, 8 (so the size of the array in every single run is 200 * 2, 200 * 2 ** 4, 200 * 2 ** 8). As shown in the accuracy (the rightmost three columns) below the size of the array in one iteration doesn't matter much at least when it's bigger than 200 * 2 and when there are 30 sample runs for each configuration - the difference of accuracy is only roughly ±1% when all of the risks are just randomly small. (I also ran it through legacy which was not touched by #24218 at all and as expected it doesn't show any false positives)

See results 
                                                                                          confidence improvement accuracy (*)   (**)  (***)
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='auth' withBase='false'                      -1.70 %       ±2.31% ±3.09% ±4.07%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='auth' withBase='true'                        0.98 %       ±2.00% ±2.67% ±3.49%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='dot' withBase='false'                        1.20 %       ±2.61% ±3.49% ±4.57%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='dot' withBase='true'                         0.71 %       ±1.13% ±1.50% ±1.95%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='file' withBase='false'                      -0.05 %       ±2.49% ±3.31% ±4.31%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='file' withBase='true'                       -1.76 %       ±3.42% ±4.58% ±6.00%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='idn' withBase='false'                       -0.32 %       ±1.17% ±1.56% ±2.03%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='idn' withBase='true'                        -0.06 %       ±4.55% ±6.05% ±7.88%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='javascript' withBase='false'                 1.84 %       ±2.98% ±3.97% ±5.18%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='javascript' withBase='true'                  0.39 %       ±1.32% ±1.76% ±2.30%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='long' withBase='false'                      -0.95 %       ±1.79% ±2.39% ±3.12%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='long' withBase='true'                        1.55 %       ±2.30% ±3.07% ±4.02%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='percent' withBase='false'                    1.60 %       ±1.83% ±2.44% ±3.18%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='percent' withBase='true'                     1.03 %       ±2.66% ±3.55% ±4.64%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='short' withBase='false'                      0.34 %       ±1.15% ±1.53% ±2.00%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='short' withBase='true'                       1.11 %       ±2.74% ±3.64% ±4.74%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='wpt' withBase='false'                        3.88 %       ±4.19% ±5.63% ±7.43%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='wpt' withBase='true'                         0.24 %       ±1.52% ±2.03% ±2.65%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='ws' withBase='false'                        -1.93 %       ±3.04% ±4.06% ±5.33%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=1 type='ws' withBase='true'                          1.22 %       ±2.92% ±3.89% ±5.08%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='auth' withBase='false'                      -0.94 %       ±4.24% ±5.64% ±7.34%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='auth' withBase='true'                        2.38 %       ±4.23% ±5.63% ±7.34%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='dot' withBase='false'                        0.46 %       ±3.97% ±5.28% ±6.88%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='dot' withBase='true'                         1.16 %       ±2.91% ±3.90% ±5.13%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='file' withBase='false'                       0.00 %       ±2.32% ±3.08% ±4.01%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='file' withBase='true'                        1.14 %       ±3.12% ±4.15% ±5.42%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='idn' withBase='false'                        0.94 %       ±2.70% ±3.59% ±4.67%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='idn' withBase='true'                        -0.89 %       ±2.23% ±2.97% ±3.88%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='javascript' withBase='false'                 0.60 %       ±2.20% ±2.93% ±3.84%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='javascript' withBase='true'                 -0.61 %       ±1.59% ±2.12% ±2.77%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='long' withBase='false'                      -0.11 %       ±2.38% ±3.17% ±4.12%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='long' withBase='true'                        0.34 %       ±1.90% ±2.53% ±3.29%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='percent' withBase='false'                   -0.98 %       ±1.61% ±2.14% ±2.79%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='percent' withBase='true'                    -1.91 %       ±2.26% ±3.03% ±3.98%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='short' withBase='false'                      0.08 %       ±1.60% ±2.13% ±2.77%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='short' withBase='true'                       1.04 %       ±2.15% ±2.87% ±3.75%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='wpt' withBase='false'                        0.34 %       ±2.47% ±3.30% ±4.30%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='wpt' withBase='true'                        -2.43 %       ±3.25% ±4.35% ±5.74%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='ws' withBase='false'                         0.18 %       ±4.31% ±5.74% ±7.47%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=4 type='ws' withBase='true'                         -1.58 %       ±4.90% ±6.53% ±8.50%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='auth' withBase='false'                      -0.54 %       ±2.15% ±2.86% ±3.72%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='auth' withBase='true'                       -0.23 %       ±2.24% ±2.97% ±3.87%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='dot' withBase='false'                       -1.07 %       ±2.72% ±3.62% ±4.71%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='dot' withBase='true'                         0.06 %       ±1.87% ±2.48% ±3.23%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='file' withBase='false'                      -1.98 %       ±2.31% ±3.07% ±4.01%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='file' withBase='true'                       -1.23 %       ±1.80% ±2.39% ±3.11%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='idn' withBase='false'                        0.57 %       ±2.26% ±3.01% ±3.92%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='idn' withBase='true'                         0.42 %       ±2.52% ±3.35% ±4.36%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='javascript' withBase='false'                 0.60 %       ±1.77% ±2.35% ±3.07%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='javascript' withBase='true'                  0.68 %       ±2.38% ±3.17% ±4.13%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='long' withBase='false'                      -0.04 %       ±1.94% ±2.58% ±3.37%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='long' withBase='true'                       -0.40 %       ±2.14% ±2.85% ±3.71%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='percent' withBase='false'                   -1.13 %       ±2.54% ±3.37% ±4.39%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='percent' withBase='true'                     0.87 %       ±2.37% ±3.15% ±4.11%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='short' withBase='false'                      0.76 %       ±2.62% ±3.49% ±4.54%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='short' withBase='true'                       0.48 %       ±2.00% ±2.66% ±3.46%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='wpt' withBase='false'                        1.32 %       ±2.14% ±2.85% ±3.73%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='wpt' withBase='true'                        -1.32 %       ±2.83% ±3.77% ±4.93%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='ws' withBase='false'                         0.38 %       ±2.40% ±3.20% ±4.16%
 url/legacy-vs-whatwg-url-parse.js method='legacy' e=8 type='ws' withBase='true'                          0.03 %       ±1.64% ±2.18% ±2.83%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='auth' withBase='false'              ***    -11.25 %       ±1.80% ±2.40% ±3.13%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='auth' withBase='true'               ***    -14.26 %       ±3.53% ±4.70% ±6.12%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='dot' withBase='false'               ***    -14.39 %       ±3.99% ±5.32% ±6.96%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='dot' withBase='true'                ***    -14.00 %       ±2.17% ±2.90% ±3.79%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='file' withBase='false'              ***    -14.94 %       ±2.36% ±3.14% ±4.08%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='file' withBase='true'               ***    -12.21 %       ±1.77% ±2.35% ±3.07%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='idn' withBase='false'               ***    -15.15 %       ±1.80% ±2.39% ±3.12%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='idn' withBase='true'                ***    -12.46 %       ±3.01% ±4.00% ±5.22%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='javascript' withBase='false'        ***    -15.95 %       ±1.90% ±2.53% ±3.29%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='javascript' withBase='true'         ***    -15.11 %       ±2.69% ±3.60% ±4.71%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='long' withBase='false'              ***     -8.21 %       ±1.39% ±1.85% ±2.41%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='long' withBase='true'               ***     -9.36 %       ±1.73% ±2.30% ±3.01%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='percent' withBase='false'           ***    -16.61 %       ±2.53% ±3.36% ±4.38%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='percent' withBase='true'            ***    -12.59 %       ±3.49% ±4.66% ±6.10%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='short' withBase='false'             ***    -15.16 %       ±2.32% ±3.09% ±4.03%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='short' withBase='true'              ***    -14.91 %       ±2.94% ±3.92% ±5.14%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='wpt' withBase='false'               ***    -14.07 %       ±2.84% ±3.79% ±4.95%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='wpt' withBase='true'                ***     -9.35 %       ±1.54% ±2.05% ±2.68%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='ws' withBase='false'                ***    -15.84 %       ±1.35% ±1.80% ±2.35%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=1 type='ws' withBase='true'                 ***    -13.88 %       ±2.04% ±2.72% ±3.54%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='auth' withBase='false'              ***    -11.49 %       ±2.11% ±2.82% ±3.68%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='auth' withBase='true'               ***     -9.76 %       ±1.93% ±2.57% ±3.35%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='dot' withBase='false'               ***    -15.30 %       ±2.26% ±3.01% ±3.92%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='dot' withBase='true'                ***    -14.11 %       ±2.67% ±3.56% ±4.64%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='file' withBase='false'              ***     -8.92 %       ±4.15% ±5.58% ±7.35%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='file' withBase='true'               ***    -13.43 %       ±2.06% ±2.74% ±3.58%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='idn' withBase='false'               ***    -12.58 %       ±1.56% ±2.08% ±2.71%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='idn' withBase='true'                ***    -15.71 %       ±2.69% ±3.61% ±4.75%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='javascript' withBase='false'        ***    -14.87 %       ±1.90% ±2.52% ±3.29%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='javascript' withBase='true'         ***    -12.78 %       ±2.53% ±3.38% ±4.42%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='long' withBase='false'              ***     -6.34 %       ±2.28% ±3.03% ±3.94%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='long' withBase='true'               ***     -9.13 %       ±2.87% ±3.81% ±4.96%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='percent' withBase='false'           ***    -14.49 %       ±1.68% ±2.25% ±2.94%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='percent' withBase='true'            ***    -13.99 %       ±2.89% ±3.85% ±5.02%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='short' withBase='false'             ***    -13.38 %       ±1.94% ±2.58% ±3.37%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='short' withBase='true'              ***    -11.86 %       ±2.47% ±3.29% ±4.29%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='wpt' withBase='false'               ***    -13.37 %       ±2.56% ±3.40% ±4.43%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='wpt' withBase='true'                ***    -12.79 %       ±2.33% ±3.12% ±4.08%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='ws' withBase='false'                ***    -12.10 %       ±2.01% ±2.68% ±3.49%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=4 type='ws' withBase='true'                 ***    -12.33 %       ±1.30% ±1.72% ±2.24%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='auth' withBase='false'              ***    -12.70 %       ±2.21% ±2.94% ±3.84%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='auth' withBase='true'               ***    -12.98 %       ±1.93% ±2.57% ±3.35%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='dot' withBase='false'               ***    -14.46 %       ±2.31% ±3.08% ±4.02%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='dot' withBase='true'                ***    -14.12 %       ±1.88% ±2.50% ±3.26%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='file' withBase='false'              ***    -12.76 %       ±1.95% ±2.60% ±3.39%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='file' withBase='true'               ***    -12.59 %       ±1.94% ±2.60% ±3.41%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='idn' withBase='false'               ***    -12.28 %       ±2.47% ±3.29% ±4.29%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='idn' withBase='true'                ***    -14.26 %       ±2.60% ±3.46% ±4.50%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='javascript' withBase='false'        ***    -18.45 %       ±2.80% ±3.73% ±4.86%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='javascript' withBase='true'         ***    -14.16 %       ±2.20% ±2.93% ±3.81%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='long' withBase='false'              ***     -6.78 %       ±1.84% ±2.46% ±3.21%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='long' withBase='true'               ***     -6.99 %       ±1.88% ±2.50% ±3.25%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='percent' withBase='false'           ***    -15.37 %       ±2.10% ±2.80% ±3.64%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='percent' withBase='true'            ***    -13.10 %       ±2.28% ±3.04% ±3.98%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='short' withBase='false'             ***    -14.13 %       ±1.73% ±2.30% ±3.00%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='short' withBase='true'              ***    -12.76 %       ±2.15% ±2.86% ±3.73%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='wpt' withBase='false'               ***    -14.31 %       ±2.08% ±2.77% ±3.61%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='wpt' withBase='true'                ***    -12.25 %       ±1.88% ±2.50% ±3.25%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='ws' withBase='false'                ***    -14.86 %       ±2.04% ±2.71% ±3.53%
 url/legacy-vs-whatwg-url-parse.js method='whatwg' e=8 type='ws' withBase='true'                 ***    -12.15 %       ±2.37% ±3.15% ±4.10%

For comparison, with the old benchmark the accuracy isn't really any different from when e=1 above even though it runs much longer

See results 
                                                                              confidence improvement accuracy (*)   (**)  (***)
 url/legacy-vs-whatwg-url-parse.js n=100000 method='whatwg' type='auth'              ***    -15.23 %       ±2.23% ±2.97% ±3.87%
 url/legacy-vs-whatwg-url-parse.js n=100000 method='whatwg' type='dot'               ***    -14.43 %       ±2.85% ±3.79% ±4.93%
 url/legacy-vs-whatwg-url-parse.js n=100000 method='whatwg' type='file'              ***    -13.10 %       ±2.62% ±3.49% ±4.55%
 url/legacy-vs-whatwg-url-parse.js n=100000 method='whatwg' type='idn'               ***    -13.19 %       ±3.37% ±4.49% ±5.84%
 url/legacy-vs-whatwg-url-parse.js n=100000 method='whatwg' type='javascript'        ***    -19.30 %       ±2.03% ±2.71% ±3.52%
 url/legacy-vs-whatwg-url-parse.js n=100000 method='whatwg' type='long'              ***     -6.29 %       ±3.38% ±4.51% ±5.89%
 url/legacy-vs-whatwg-url-parse.js n=100000 method='whatwg' type='percent'           ***    -17.02 %       ±3.22% ±4.29% ±5.58%
 url/legacy-vs-whatwg-url-parse.js n=100000 method='whatwg' type='short'             ***    -13.14 %       ±3.11% ±4.14% ±5.39%
 url/legacy-vs-whatwg-url-parse.js n=100000 method='whatwg' type='ws'                ***    -14.13 %       ±2.30% ±3.06% ±3.98%
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@joyeecheung
fixup! benchmark: support URL inputs in create-clientrequest
d6f9f89
@joyeecheung

joyeecheung commented Nov 11, 2018

Copy link
Copy Markdown
Member Author

Added a type check for the argument of common.bakeUrlData

CI: https://ci.nodejs.org/job/node-test-pull-request/18534/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@joyeecheung

joyeecheung commented Nov 17, 2018

Copy link
Copy Markdown
Member Author

Ping @nodejs/url @nodejs/performance can I have some reviews please? As the data in #24302 (comment) demonstrates, this should not affect the accuracy of the benchmarks while providing more options for the data set.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
mcollina
mcollina approved these changes Nov 18, 2018
View reviewed changes

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@joyeecheung joyeecheung added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Nov 19, 2018
@joyeecheung

joyeecheung commented Nov 19, 2018

Copy link
Copy Markdown
Member Author

New CI: https://ci.nodejs.org/job/node-test-pull-request/18749/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
jasnell
jasnell approved these changes Nov 19, 2018
View reviewed changes
@Trott

Trott commented Nov 19, 2018

Copy link
Copy Markdown
Member

Since this affects benchmarks, let's run the benchmark CI job on it: https://ci.nodejs.org/job/node-test-commit-custom-suites/766/
joyeecheung reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@joyeecheung

joyeecheung commented Nov 19, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Landed in 5f25dd1...8450d11, thanks!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@joyeecheung joyeecheung closed this Nov 19, 2018
joyeecheung added a commit that referenced this pull request Nov 19, 2018
@joyeecheung
benchmark: pre-generate data set for URL benchmarks
a365bb9 
This patch:

- Introduces `common.bakeUrlData` which can be used to pre-generate
  the data set for the URL benchmarks to loop through instead of
  looping over a constant.
- Add the option to use WPT data in benchmarks for better diversity
  in the input
- Add the option to benchmark URL parsing with base URLs (whatwg only)
- Moves the data in `benchmark/fixtures/url-inputs.js` to
  `benchmark/common.js`

PR-URL: #24302
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
joyeecheung added a commit that referenced this pull request Nov 19, 2018
@joyeecheung
benchmark: support URL inputs in create-clientrequest
8450d11 
This patch adds the option in the create-clientrequest benchmark
to accept URL inputs (as strings or as URL objects) so we can
measure the impact of URL parsing in a more sophisticated use case.

PR-URL: #24302
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
targos pushed a commit that referenced this pull request Nov 19, 2018
@joyeecheung @targos
benchmark: pre-generate data set for URL benchmarks
5b5f309 
This patch:

- Introduces `common.bakeUrlData` which can be used to pre-generate
  the data set for the URL benchmarks to loop through instead of
  looping over a constant.
- Add the option to use WPT data in benchmarks for better diversity
  in the input
- Add the option to benchmark URL parsing with base URLs (whatwg only)
- Moves the data in `benchmark/fixtures/url-inputs.js` to
  `benchmark/common.js`

PR-URL: #24302
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
targos pushed a commit that referenced this pull request Nov 19, 2018
@joyeecheung @targos
benchmark: support URL inputs in create-clientrequest
21f58d4 
This patch adds the option in the create-clientrequest benchmark
to accept URL inputs (as strings or as URL objects) so we can
measure the impact of URL parsing in a more sophisticated use case.

PR-URL: #24302
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
rvagg pushed a commit that referenced this pull request Nov 28, 2018
@joyeecheung @rvagg
benchmark: pre-generate data set for URL benchmarks
f5d4db1 
This patch:

- Introduces `common.bakeUrlData` which can be used to pre-generate
  the data set for the URL benchmarks to loop through instead of
  looping over a constant.
- Add the option to use WPT data in benchmarks for better diversity
  in the input
- Add the option to benchmark URL parsing with base URLs (whatwg only)
- Moves the data in `benchmark/fixtures/url-inputs.js` to
  `benchmark/common.js`

PR-URL: #24302
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
rvagg pushed a commit that referenced this pull request Nov 28, 2018
@joyeecheung @rvagg
benchmark: support URL inputs in create-clientrequest
ef63bb2 
This patch adds the option in the create-clientrequest benchmark
to accept URL inputs (as strings or as URL objects) so we can
measure the impact of URL parsing in a more sophisticated use case.

PR-URL: #24302
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@BridgeAR BridgeAR mentioned this pull request Dec 5, 2018
v11.4.0 proposal #24854
Merged
4 tasks done
This was referenced Dec 7, 2018
2018-12-07 Version 11.4.0 (Current) @BridgeAR beevelop/docker-android-nodejs#261
Closed
2018-12-07 Version 11.4.0 (Current) @BridgeAR beevelop/docker-android-nodejs#262
Closed
2018-12-07 Version 11.4.0 (Current) @BridgeAR beevelop/docker-android-nodejs#263
Closed
2018-12-07 Version 11.4.0 (Current) @BridgeAR beevelop/docker-android-nodejs#264
Closed
2018-12-07 Version 11.4.0 (Current) @BridgeAR beevelop/docker-android-nodejs#265
Closed
@codebytere

codebytere commented Jan 13, 2019

Copy link
Copy Markdown
Member

@joyeecheung is this something you think should be backported to v10.x? I've added the label, but feel free to remove it!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@codebytere codebytere added the backport-requested-v10.x label Jan 13, 2019
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
@joyeecheung @refack
benchmark: pre-generate data set for URL benchmarks
9cf0539 
This patch:

- Introduces `common.bakeUrlData` which can be used to pre-generate
  the data set for the URL benchmarks to loop through instead of
  looping over a constant.
- Add the option to use WPT data in benchmarks for better diversity
  in the input
- Add the option to benchmark URL parsing with base URLs (whatwg only)
- Moves the data in `benchmark/fixtures/url-inputs.js` to
  `benchmark/common.js`

PR-URL: nodejs#24302
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
refack pushed a commit to refack/node that referenced this pull request Jan 14, 2019
@joyeecheung @refack
benchmark: support URL inputs in create-clientrequest
47c69ea 
This patch adds the option in the create-clientrequest benchmark
to accept URL inputs (as strings or as URL objects) so we can
measure the impact of URL parsing in a more sophisticated use case.

PR-URL: nodejs#24302
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcollina mcollina mcollina approved these changes
@jasnell jasnell jasnell approved these changes
+1 more reviewer
@mscdex mscdex mscdex left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. benchmark Issues and PRs related to Node.js benchmarks and benchmarking infrastructure. http Issues and PRs related to the http subsystem. querystring Issues and PRs related to the built-in querystring module. test Issues and PRs related to Node.js core tests and test infrastructure. url Issues and PRs related to the legacy built-in url module. whatwg-url Issues and PRs related to the WHATWG URL implementation.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@joyeecheung @nodejs-github-bot @Trott @codebytere @mcollina @mscdex @jasnell
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.