GeezerVison is showing github.com — crypto: use EVP_MD_fetch and cache EVP_MD for hashes by joyeecheung · Pull Request #51034 · nodejs/node
Conversation
joyeecheung
commented
Dec 3, 2023
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
On OpenSSL 3, migrate from EVP_get_digestbyname() to EVP_MD_fetch() to get the implementation and use a per-Environment cache for it. The EVP_MDs are freed during Environment cleanup.
Drive-by: declare the smart pointer for EVP_MD_CTX as EVPMDCtxPointer instead of EVPMDPointer to avoid confusion with EVP_MD pointers.
Refs: https://www.openssl.org/docs/man3.0/man7/crypto.html#Explicit-fetching
Refs: nodejs/performance#136
On MacOS + M2
crypto/webcrypto-digest.js n=100000 method='SHA-1' data=10 sync='createHash' *** 31.30 % ±5.30% ±7.06% ±9.20%
crypto/webcrypto-digest.js n=100000 method='SHA-1' data=100 sync='createHash' *** 34.33 % ±4.93% ±6.60% ±8.67%
crypto/webcrypto-digest.js n=100000 method='SHA-1' data=20 sync='createHash' *** 29.06 % ±2.33% ±3.10% ±4.05%
crypto/webcrypto-digest.js n=100000 method='SHA-1' data=50 sync='createHash' *** 30.14 % ±3.32% ±4.42% ±5.77%
crypto/webcrypto-digest.js n=100000 method='SHA-256' data=10 sync='createHash' *** 32.14 % ±2.68% ±3.58% ±4.69%
crypto/webcrypto-digest.js n=100000 method='SHA-256' data=100 sync='createHash' *** 28.33 % ±4.23% ±5.68% ±7.50%
crypto/webcrypto-digest.js n=100000 method='SHA-256' data=20 sync='createHash' *** 34.56 % ±3.63% ±4.85% ±6.36%
crypto/webcrypto-digest.js n=100000 method='SHA-256' data=50 sync='createHash' *** 30.81 % ±2.45% ±3.29% ±4.32%
crypto/webcrypto-digest.js n=100000 method='SHA-384' data=10 sync='createHash' *** 28.79 % ±2.94% ±3.93% ±5.15%
crypto/webcrypto-digest.js n=100000 method='SHA-384' data=100 sync='createHash' *** 28.88 % ±3.09% ±4.14% ±5.44%
crypto/webcrypto-digest.js n=100000 method='SHA-384' data=20 sync='createHash' *** 29.88 % ±4.17% ±5.56% ±7.26%
crypto/webcrypto-digest.js n=100000 method='SHA-384' data=50 sync='createHash' *** 30.27 % ±3.20% ±4.26% ±5.56%
crypto/webcrypto-digest.js n=100000 method='SHA-512' data=10 sync='createHash' *** 28.05 % ±2.37% ±3.15% ±4.12%
crypto/webcrypto-digest.js n=100000 method='SHA-512' data=100 sync='createHash' *** 24.92 % ±5.05% ±6.80% ±9.00%
crypto/webcrypto-digest.js n=100000 method='SHA-512' data=20 sync='createHash' *** 28.65 % ±3.83% ±5.10% ±6.64%
crypto/webcrypto-digest.js n=100000 method='SHA-512' data=50 sync='createHash' *** 29.21 % ±4.67% ±6.21% ±8.10%
On Ubuntu + Intel(R) Xeon(R) Platinum 8280 CPU @ 2.70GHz
confidence improvement accuracy (*) (**) (***)
crypto/webcrypto-digest.js n=100000 method='SHA-1' data=10 sync='createHash' *** 25.38 % ±1.50% ±2.03% ±2.72%
crypto/webcrypto-digest.js n=100000 method='SHA-1' data=100 sync='createHash' *** 26.31 % ±2.02% ±2.72% ±3.61%
crypto/webcrypto-digest.js n=100000 method='SHA-1' data=20 sync='createHash' *** 27.41 % ±1.94% ±2.63% ±3.54%
crypto/webcrypto-digest.js n=100000 method='SHA-1' data=50 sync='createHash' *** 25.31 % ±1.94% ±2.65% ±3.61%
crypto/webcrypto-digest.js n=100000 method='SHA-256' data=10 sync='createHash' *** 27.36 % ±1.51% ±2.02% ±2.68%
crypto/webcrypto-digest.js n=100000 method='SHA-256' data=100 sync='createHash' *** 25.73 % ±1.59% ±2.15% ±2.89%
crypto/webcrypto-digest.js n=100000 method='SHA-256' data=20 sync='createHash' *** 27.46 % ±1.68% ±2.27% ±3.03%
crypto/webcrypto-digest.js n=100000 method='SHA-256' data=50 sync='createHash' *** 27.52 % ±1.63% ±2.19% ±2.90%
crypto/webcrypto-digest.js n=100000 method='SHA-384' data=10 sync='createHash' *** 26.15 % ±1.85% ±2.49% ±3.30%
crypto/webcrypto-digest.js n=100000 method='SHA-384' data=100 sync='createHash' *** 26.50 % ±1.78% ±2.40% ±3.21%
crypto/webcrypto-digest.js n=100000 method='SHA-384' data=20 sync='createHash' *** 26.63 % ±2.11% ±2.83% ±3.76%
crypto/webcrypto-digest.js n=100000 method='SHA-384' data=50 sync='createHash' *** 26.67 % ±1.55% ±2.10% ±2.81%
crypto/webcrypto-digest.js n=100000 method='SHA-512' data=10 sync='createHash' *** 28.08 % ±1.95% ±2.64% ±3.52%
crypto/webcrypto-digest.js n=100000 method='SHA-512' data=100 sync='createHash' *** 29.11 % ±1.82% ±2.44% ±3.24%
crypto/webcrypto-digest.js n=100000 method='SHA-512' data=20 sync='createHash' *** 27.91 % ±2.02% ±2.74% ±3.70%
crypto/webcrypto-digest.js n=100000 method='SHA-512' data=50 sync='createHash' *** 29.52 % ±1.41% ±1.90% ±2.52%
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
bricss and meyfa reacted with rocket emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
nodejs-github-bot
added
c++
Issues and PRs that require attention from people who are familiar with C++.
lib / src
Issues and PRs involving general changes in the lib/ or src/ directories.
needs-ci
PRs that need a full CI run.
labels
Dec 3, 2023
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
joyeecheung
commented
Dec 4, 2023
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
|
From CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1479/console
See benchmark results
02:45:29 confidence improvement accuracy (*) (**) (***)
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-1' data=10 sync='createHash' *** 14.68 % ±5.04% ±6.71% ±8.74%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-1' data=10 sync='subtle' 0.68 % ±2.40% ±3.19% ±4.16%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-1' data=100 sync='createHash' *** 21.65 % ±3.24% ±4.31% ±5.61%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-1' data=100 sync='subtle' 2.09 % ±2.70% ±3.59% ±4.68%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-1' data=20 sync='createHash' *** 23.17 % ±4.51% ±6.00% ±7.81%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-1' data=20 sync='subtle' 0.56 % ±2.92% ±3.88% ±5.06%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-1' data=50 sync='createHash' *** 21.32 % ±3.68% ±4.91% ±6.40%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-1' data=50 sync='subtle' -1.80 % ±2.38% ±3.17% ±4.13%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-256' data=10 sync='createHash' *** 26.00 % ±5.31% ±7.09% ±9.27%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-256' data=10 sync='subtle' 2.18 % ±2.79% ±3.72% ±4.84%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-256' data=100 sync='createHash' *** 20.86 % ±4.08% ±5.44% ±7.09%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-256' data=100 sync='subtle' -1.96 % ±2.82% ±3.76% ±4.89%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-256' data=20 sync='createHash' *** 21.05 % ±3.78% ±5.04% ±6.58%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-256' data=20 sync='subtle' 0.34 % ±3.23% ±4.30% ±5.59%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-256' data=50 sync='createHash' *** 24.07 % ±3.63% ±4.83% ±6.29%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-256' data=50 sync='subtle' -1.47 % ±2.59% ±3.44% ±4.48%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-384' data=10 sync='createHash' *** 19.15 % ±4.30% ±5.72% ±7.45%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-384' data=10 sync='subtle' * 3.63 % ±2.82% ±3.76% ±4.89%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-384' data=100 sync='createHash' *** 24.47 % ±3.62% ±4.81% ±6.27%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-384' data=100 sync='subtle' -0.21 % ±2.71% ±3.60% ±4.69%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-384' data=20 sync='createHash' *** 21.98 % ±3.90% ±5.20% ±6.80%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-384' data=20 sync='subtle' 0.53 % ±2.94% ±3.92% ±5.10%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-384' data=50 sync='createHash' *** 23.69 % ±3.59% ±4.80% ±6.27%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-384' data=50 sync='subtle' 0.99 % ±2.91% ±3.87% ±5.04%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-512' data=10 sync='createHash' *** 19.90 % ±4.26% ±5.70% ±7.46%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-512' data=10 sync='subtle' -0.34 % ±2.83% ±3.76% ±4.90%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-512' data=100 sync='createHash' *** 22.92 % ±3.49% ±4.65% ±6.06%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-512' data=100 sync='subtle' -1.94 % ±2.21% ±2.95% ±3.84%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-512' data=20 sync='createHash' *** 25.19 % ±3.88% ±5.16% ±6.72%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-512' data=20 sync='subtle' * -3.76 % ±2.94% ±3.91% ±5.09%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-512' data=50 sync='createHash' *** 21.33 % ±3.29% ±4.39% ±5.73%
02:45:29 crypto/webcrypto-digest.js n=100000 method='SHA-512' data=50 sync='subtle' -0.57 % ±2.68% ±3.56%
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
joyeecheung
commented
Dec 4, 2023
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
|
Updated to cache all the explicitly fetched implementations from getHashes() and map the supported algorithms to indices into the cache (now a vector). Locally on macOS + M2 the performance improvement is better:
See benchmark results
confidence improvement accuracy (*) (**) (***)
crypto/webcrypto-digest.js n=100000 method='SHA-1' data=10 sync='createHash' *** 31.30 % ±5.30% ±7.06% ±9.20%
crypto/webcrypto-digest.js n=100000 method='SHA-1' data=100 sync='createHash' *** 34.33 % ±4.93% ±6.60% ±8.67%
crypto/webcrypto-digest.js n=100000 method='SHA-1' data=20 sync='createHash' *** 29.06 % ±2.33% ±3.10% ±4.05%
crypto/webcrypto-digest.js n=100000 method='SHA-1' data=50 sync='createHash' *** 30.14 % ±3.32% ±4.42% ±5.77%
crypto/webcrypto-digest.js n=100000 method='SHA-256' data=10 sync='createHash' *** 32.14 % ±2.68% ±3.58% ±4.69%
crypto/webcrypto-digest.js n=100000 method='SHA-256' data=100 sync='createHash' *** 28.33 % ±4.23% ±5.68% ±7.50%
crypto/webcrypto-digest.js n=100000 method='SHA-256' data=20 sync='createHash' *** 34.56 % ±3.63% ±4.85% ±6.36%
crypto/webcrypto-digest.js n=100000 method='SHA-256' data=50 sync='createHash' *** 30.81 % ±2.45% ±3.29% ±4.32%
crypto/webcrypto-digest.js n=100000 method='SHA-384' data=10 sync='createHash' *** 28.79 % ±2.94% ±3.93% ±5.15%
crypto/webcrypto-digest.js n=100000 method='SHA-384' data=100 sync='createHash' *** 28.88 % ±3.09% ±4.14% ±5.44%
crypto/webcrypto-digest.js n=100000 method='SHA-384' data=20 sync='createHash' *** 29.88 % ±4.17% ±5.56% ±7.26%
crypto/webcrypto-digest.js n=100000 method='SHA-384' data=50 sync='createHash' *** 30.27 % ±3.20% ±4.26% ±5.56%
crypto/webcrypto-digest.js n=100000 method='SHA-512' data=10 sync='createHash' *** 28.05 % ±2.37% ±3.15% ±4.12%
crypto/webcrypto-digest.js n=100000 method='SHA-512' data=100 sync='createHash' *** 24.92 % ±5.05% ±6.80% ±9.00%
crypto/webcrypto-digest.js n=100000 method='SHA-512' data=20 sync='createHash' *** 28.65 % ±3.83% ±5.10% ±6.64%
crypto/webcrypto-digest.js n=100000 method='SHA-512' data=50 sync='createHash' *** 29.21 % ±4.67% ±6.21% ±8.10%
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Will caching the fetches run into issues given that OpenSSL 3 providers are dynamic (although we don't have an API in Node.js for this but one could feasibly write an addon to unload/load providers)?
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Will caching the fetches run into issues given that OpenSSL 3 providers are dynamic (although we don't have an API in Node.js for this but one could feasibly write an addon to unload/load providers)?
I think for our built-in APIs we always do the implicit fetching with no additional library context or provider property configured (because we use EVP_get_digestbyname()) so it's never reliable for any addon to count on the providers they load to affect the built-in APIs. Whether we want to support an explicit provider from the native API would be a different matter, but I think that would be orthogonal to this issue (we can just hook the cache into the explicit native API, or give up on caching if addons configure an explicit provider).
|
richardlau reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
jasnell
added
request-ci
Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review.
c++
Issues and PRs that require attention from people who are familiar with C++.
lib / src
Issues and PRs involving general changes in the lib/ or src/ directories.
needs-ci
PRs that need a full CI run.
labels
Dec 22, 2023
This comment was marked as outdated.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
jkleinsc
pushed a commit
to electron/electron
that referenced
this pull request
Apr 17, 2024
* chore: bump node in DEPS to v20.12.0
* chore: update build_add_gn_build_files.patch
* chore: update patches
* chore: bump node in DEPS to v20.12.1
* chore: update patches
* build: encode non-ASCII Latin1 characters as one byte in JS2C
nodejs/node#51605
* crypto: use EVP_MD_fetch and cache EVP_MD for hashes
nodejs/node#51034
* chore: update filenames.json
* chore: bump node in DEPS to v20.12.2
* chore: update patches
* src: support configurable snapshot
nodejs/node#50453
* test: remove test-domain-error-types flaky designation
nodejs/node#51717
* src: avoid draining platform tasks at FreeEnvironment
nodejs/node#51290
* chore: fix accidentally deleted v8 dep
* lib: define FormData and fetch etc. in the built-in snapshot
nodejs/node#51598
* chore: rebase on main
* chore: remove stray log
---------
Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com>
Co-authored-by: Cheng <[email protected]>
Co-authored-by: Shelley Vohr <[email protected]>
Co-authored-by: PatchUp <73610968+patchup[bot]@users.noreply.github.com>
codebytere
added a commit
to electron/electron
that referenced
this pull request
Jun 1, 2024
* chore: bump node in DEPS to v20.13.1
* chore: bump node in DEPS to v20.14.0
* chore: update build_add_gn_build_files.patch
* chore: update patches
* chore: update patches
* build: encode non-ASCII Latin1 characters as one byte in JS2C
nodejs/node#51605
* crypto: use EVP_MD_fetch and cache EVP_MD for hashes
nodejs/node#51034
* chore: update filenames.json
* chore: update patches
* src: support configurable snapshot
nodejs/node#50453
* test: remove test-domain-error-types flaky designation
nodejs/node#51717
* src: avoid draining platform tasks at FreeEnvironment
nodejs/node#51290
* chore: fix accidentally deleted v8 dep
* lib: define FormData and fetch etc. in the built-in snapshot
nodejs/node#51598
* chore: remove stray log
* crypto: enable NODE_EXTRA_CA_CERTS with BoringSSL
nodejs/node#52217
* test: skip test for dynamically linked OpenSSL
nodejs/node#52542
* lib, url: add a `windows` option to path parsing
nodejs/node#52509
* src: use dedicated routine to compile function for builtin CJS loader
nodejs/node#52016
* test: mark test as flaky
nodejs/node#52671
* build,tools: add test-ubsan ci
nodejs/node#46297
* src: preload function for Environment
nodejs/node#51539
* deps: update c-ares to 1.28.1
nodejs/node#52285
* chore: fixup
* events: extract addAbortListener for safe internal use
nodejs/node#52081
* module: print location of unsettled top-level await in entry points
nodejs/node#51999
* fs: add stacktrace to fs/promises
nodejs/node#49849
* chore: fixup indices
---------
Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com>
Co-authored-by: Cheng <[email protected]>
Co-authored-by: Shelley Vohr <[email protected]>
Co-authored-by: PatchUp <73610968+patchup[bot]@users.noreply.github.com>
Reviewers
jasnell
jasnell approved these changes
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Labels
c++
Issues and PRs that require attention from people who are familiar with C++.
commit-queue-squash
PRs the Commit Queue should land as one squashed commit.
lib / src
Issues and PRs involving general changes in the lib/ or src/ directories.
needs-ci
PRs that need a full CI run.
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|