GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — async-hooks: introduce async-storage API by vdeturckheim · Pull Request #26540 · nodejs/node

Conversation

@vdeturckheim

@vdeturckheim vdeturckheim commented Mar 9, 2019

Copy link
Copy Markdown
Member

This introduces a new API in core to provide asynchronous storage features.

More and more Node.js users have been needing such API to:

  • build monitoring tools
  • build features that enhence code readability and reduce the need of passing HTTP request objects all the way to one single method that needs it (for instance, a logger).

Some userland modules exist but they:

  • sometimes badly interact with each other
  • introduce potential easy mistakes in usage (need to be required first)
  • might need some specific addons to handle part of the ecosystem properly

Havin such API in core will make the ecosystem more stable and reliable when needing an asynchronous storage.

This is still an early work and there is probably a lot of updates to do to this PR :)

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

NotoriousLisp, romain-gilles-ultra, SalvatorePreviti, darky, Shahor, puzpuzpuz, cliffordfajardo, semarketir, and semyonf reacted with thumbs up emoji mmarchini, aviat, antsmartian, kibertoad, LucasLeandro1204, lin7sh, wentout, puzpuzpuz, ErickWendel, NotoriousLisp, and 18 more reacted with heart emoji
All reactions
  • 👍 9 reactions
  • ❤️ 28 reactions
@vdeturckheim
vdeturckheim requested review from bmeurer, jasnell, mcollina and misterdjules March 9, 2019 10:07
@nodejs-github-bot

This comment has been minimized.

Sign in to view
@nodejs-github-bot nodejs-github-bot added the build Issues and PRs related to Node.js builds or CI infrastructure. label Mar 9, 2019
mcollina
mcollina reviewed Mar 9, 2019
View reviewed changes
Comment thread lib/async_storage.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/async_storage.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/async_storage.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/async_storage.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

mscdex
mscdex previously requested changes Mar 9, 2019
View reviewed changes

@mscdex mscdex left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this is best left as a userland module, especially considering it can be fairly easily implemented using just async_hooks. There are also some assumptions being made design-wise here that others might disagree with and would want more flexibility.

Even if this kind of feature were to be added to node core, it would be better to have it start in userland to allow easier iteration on the idea first, to figure out the best API and behaviors between all parties that would be involved.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

benjamingr and brody2consult reacted with thumbs up emoji
All reactions
  • 👍 2 reactions
addaleax
addaleax reviewed Mar 9, 2019
View reviewed changes
Comment thread doc/api/async_storage.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread doc/api/async_storage.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread doc/api/async_storage.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/async_storage.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

@addaleax

addaleax commented Mar 9, 2019

Copy link
Copy Markdown
Member

@mscdex This was talked about at the diagnostics summit. I don’t think the feature has to exist in this particular form, but having it in core made sense in general, and in particular performance-wise, since a good implementation could allow us to skip calling async_hooks callbacks.

FYI @nodejs/tsc

mcollina, vdeturckheim, BridgeAR, cliedeman, ChALkeR, and ethanresnick reacted with thumbs up emoji
All reactions
  • 👍 6 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mcollina

mcollina commented Mar 9, 2019

Copy link
Copy Markdown
Member

I think this functionality belongs in core. We should offer a single way to do cls that is high performant, stable and reliable. It’s not an ecosystem concern, it’s a runtime concern (as much as a ThreadLocal implementation is for Java/.NET/.. etc).

mmarchini, BridgeAR, Qard, hybrist, cliedeman, wentout, drecdroid, Ragnoroct, puzpuzpuz, Shahor, and 9 more reacted with thumbs up emoji
All reactions
  • 👍 19 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jasnell
jasnell reviewed Mar 9, 2019
View reviewed changes
Comment thread doc/api/async_storage.md Outdated
@@ -0,0 +1,86 @@
# Async Storage

<!--introduced_in=v12.x.x-->

@jasnell jasnell Mar 9, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this supposed to be added: REPLACEME ? Does introduced_in work also?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@addaleax addaleax Mar 9, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it’s a separate system used for the per-page “View another version” dropdown in the generated documentation:

image

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jasnell reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@jasnell

jasnell commented Mar 9, 2019

Copy link
Copy Markdown
Member

I agree that this should be in core and there was consensus among the @nodejs/diagnostics WG members attending the diagnostics summit. Sure, we can bikeshed around the specific API, and I would like to get some of the APM vendors to review this before it lands, it would be generally better to have a single authoritative API and implementation of this than multiple userland implementations so that APM solutions built on top can be implemented consistently.

vdeturckheim, Qard, cliedeman, Shahor, and ronkorving reacted with thumbs up emoji
All reactions
  • 👍 5 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vsemozhetbyt vsemozhetbyt added the tsc-review label Mar 9, 2019
@vsemozhetbyt

vsemozhetbyt commented Mar 9, 2019 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Should we create npm placeholder for this name as per our guide?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mcollina

mcollina commented Mar 9, 2019 via email

Copy link
Copy Markdown
Member
Go ahead. I kind of expect this to go under `@nodejs/async-storage` but better safe than sorry. Il giorno sab 9 mar 2019 alle ore 13:10 Vse Mozhet Byt < [email protected]> ha scritto:
…
Should we create npm placeholder for this name as per our guides? — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#26540 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADL4xdIdUUD6v4zX5DYe6C_E1y5GCinks5vU6SmgaJpZM4bmmI4> .
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vsemozhetbyt

vsemozhetbyt commented Mar 9, 2019 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Sorry, I am not experienced in registering placeholders in npm, so anybody more confident please do it.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mcollina

mcollina commented Mar 9, 2019 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There is https://www.npmjs.com/package/async-storage, which makes calling this module async_storage not a great idea. @nodejs/async_storage is ok.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jasnell

jasnell commented Mar 9, 2019

Copy link
Copy Markdown
Member

Does this require a separate new top level module? Could this be exposed alternatively through the existing async_hooks module? Or via util?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vdeturckheim

vdeturckheim commented Mar 9, 2019

Copy link
Copy Markdown
Member Author

@jasnell I actually think it would make sense to have that in util too. I am just worried potential future features would make this API big enough for us to regret it was not a module. That said, I'd be happy to move this to another module.

addaleax reacted with confused emoji
All reactions
  • 😕 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vdeturckheim

vdeturckheim commented Mar 9, 2019

Copy link
Copy Markdown
Member Author

I will update the PR over the week end and start to contact APM and RASP vendors to ensure thay can comment.

Thanks a lot to all of you for the comments and suggestions!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@devsnek

devsnek commented Mar 9, 2019

Copy link
Copy Markdown
Member

if this is going to be in core I'd want it to be more similar to https://github.com/WICG/kv-storage

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jasnell

jasnell commented Mar 9, 2019

Copy link
Copy Markdown
Member

@vdeturckheim ... One thing to consider is that by putting this into a new module, it automatically becomes semver-major and significantly harder to backport. My hope is that we could at least get this into 10.x also.

vdeturckheim reacted with thumbs up emoji addaleax reacted with confused emoji
All reactions
  • 👍 1 reaction
  • 😕 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vdeturckheim

vdeturckheim commented Mar 9, 2019

Copy link
Copy Markdown
Member Author

@jasnell good point, I'll move it to util then (this is a killer argument for me indeed ;) )

@devsnek I did not know about this proposal. I'd be interested in other opinions here but the solutions I could see would probably be a bit slow.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@devsnek

devsnek commented Mar 9, 2019 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

@vdeturckheim reading over this again it seems i was confused as to what this pr is adding to node.

i think it might lend itself better to a different name like "async_closure" or "async_context" something.

i'd be strongly in favor of making the class available as an async_hooks.Context

vdeturckheim reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@targos

targos commented Mar 9, 2019

Copy link
Copy Markdown
Member

I think it would be better placed in async_hooks instead of util.

vdeturckheim, BridgeAR, and addaleax reacted with thumbs up emoji
All reactions
  • 👍 3 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@devsnek devsnek added the async_hooks Issues and PRs related to the async hooks subsystem. label Mar 9, 2019
@othiym23

othiym23 commented Mar 9, 2019

Copy link
Copy Markdown
Contributor

As a developer and co-maintainer of continuation-local-storage, I’m very happy to see this functionality coming to Node core, and am amused to see how positive the reception is this time. A lot can change in 6 years. If anyone wants to reuse the name, or join this with the existing continuation-local-storage, I’m happy to hand the name over to the Foundation, although I understand that this implementation is simpler to use than CLS.

Shahor and bernardoadc reacted with thumbs up emoji aviat, BridgeAR, vdeturckheim, shisama, Qard, and mariodu reacted with heart emoji
All reactions
  • 👍 2 reactions
  • ❤️ 6 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Trott Trott removed the tsc-review label Mar 10, 2019
@vdeturckheim vdeturckheim changed the title async-storage: introduce async-storage API async-hooks: introduce async-storage API Mar 10, 2019
@addaleax addaleax added the notable-change PRs with changes that should be highlighted in changelogs. label Mar 1, 2020
codebytere added a commit that referenced this pull request Mar 1, 2020
@codebytere
2020-03-03 Version 13.10.0 (Current)
165ecaa
Notable changes:

* async_hooks
  * introduce async-context API (vdeturckheim) #26540
* stream
  * support passing generator functions into pipeline() (Robert Nagy) #31223
* tls
  * expose SSL\_export\_keying\_material (simon) #31814
* vm
  * implement vm.measureMemory() for per-context memory measurement (Joyee Cheung) #31824

PR-URL: #32027
codebytere added a commit that referenced this pull request Mar 3, 2020
@codebytere
2020-03-03 Version 13.10.0 (Current)
e39b006
Notable changes:

* async_hooks
  * introduce async-context API (vdeturckheim) #26540
* stream
  * support passing generator functions into pipeline() (Robert Nagy) #31223
* tls
  * expose SSL\_export\_keying\_material (simon) #31814
* vm
  * implement vm.measureMemory() for per-context memory measurement (Joyee Cheung) #31824

PR-URL: #32027
codebytere added a commit that referenced this pull request Mar 3, 2020
@codebytere
2020-03-03 Version 13.10.0 (Current)
471590e
Notable changes:

* async_hooks
  * introduce async-context API (vdeturckheim) #26540
* stream
  * support passing generator functions into pipeline() (Robert Nagy) #31223
* tls
  * expose SSL\_export\_keying\_material (simon) #31814
* vm
  * implement vm.measureMemory() for per-context memory measurement (Joyee Cheung) #31824

PR-URL: #32027
codebytere added a commit that referenced this pull request Mar 3, 2020
@codebytere
2020-03-04 Version 13.10.0 (Current)
f6ffdc2
Notable changes:

* async_hooks
  * introduce async-context API (vdeturckheim) #26540
* stream
  * support passing generator functions into pipeline() (Robert Nagy) #31223
* tls
  * expose SSL\_export\_keying\_material (simon) #31814
* vm
  * implement vm.measureMemory() for per-context memory measurement (Joyee Cheung) #31824

PR-URL: #32027
codebytere added a commit that referenced this pull request Mar 4, 2020
@codebytere
2020-03-04 Version 13.10.0 (Current)
7546686
Notable changes:

* async_hooks
  * introduce async-context API (vdeturckheim) #26540
* stream
  * support passing generator functions into pipeline() (Robert Nagy) #31223
* tls
  * expose SSL\_export\_keying\_material (simon) #31814
* vm
  * implement vm.measureMemory() for per-context memory measurement (Joyee Cheung) #31824

PR-URL: #32027
@Sytten

Sytten commented Mar 6, 2020

Copy link
Copy Markdown

Any chance this will be backported to v12?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@puzpuzpuz

puzpuzpuz commented Mar 6, 2020

Copy link
Copy Markdown
Member

Any chance this will be backported to v12?

It would be great to have AsyncLocalStorage in v12, so I'm all for backporting it.

Note: executionAsyncResource (#30959) has to be backported first.

cc @Qard @vdeturckheim

vdeturckheim and Sytten reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Qard

Qard commented Mar 6, 2020

Copy link
Copy Markdown
Member

As far as I know, backporting executionAsyncResource to v12 should not be a problem, but I haven't tested it myself.

puzpuzpuz and Sytten reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Flarna

Flarna commented Mar 6, 2020

Copy link
Copy Markdown
Member

Care must be taken to include also the followup PRs fixing various bugs found afterwards.

puzpuzpuz, Qard, and Sytten reacted with thumbs up emoji
All reactions
  • 👍 3 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@puzpuzpuz

puzpuzpuz commented Mar 6, 2020

Copy link
Copy Markdown
Member

I've created a backport PR for executionAsyncResource: #32131. Hopefully, it includes all bugfixes.

wentout, Qard, and vdeturckheim reacted with heart emoji
All reactions
  • ❤️ 3 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Mar 16, 2020
test: add GC test for disabled AsyncLocalStorage #31995
Closed
async_hooks: fix context loss after nested calls to AsyncLocalStorage #32085
Closed
@codebytere

codebytere commented Mar 16, 2020

Copy link
Copy Markdown
Member

@puzpuzpuz thanks! this next release is a patch so it'll go out in the subsequent minor.

puzpuzpuz reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@puzpuzpuz puzpuzpuz mentioned this pull request Mar 17, 2020
[v12.x backport] async_hooks: add AsyncLocalStorage #32318
Closed
4 tasks done
@puzpuzpuz

puzpuzpuz commented Mar 17, 2020

Copy link
Copy Markdown
Member

v12 backport: #32318

wentout, Shahor, bajtos, GriffinSchneider, and vdeturckheim reacted with thumbs up emoji vdeturckheim and marcoreni reacted with hooray emoji wentout, codebytere, Shahor, GriffinSchneider, and vdeturckheim reacted with heart emoji
All reactions
  • 👍 5 reactions
  • 🎉 2 reactions
  • ❤️ 5 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@targos targos added the backported-to-v12.x label Mar 17, 2020
@legendecas legendecas mentioned this pull request Mar 20, 2020
Deprecate destroy hook of async_hooks #32386
Closed
@wentout wentout mentioned this pull request Apr 9, 2020
async_hooks performance/stability improvement plans nodejs/diagnostics#376
Closed
puzpuzpuz pushed a commit to puzpuzpuz/node that referenced this pull request Apr 14, 2020
@vdeturckheim @puzpuzpuz
async_hooks: introduce async-context API
0807da3
Adding AsyncLocalStorage class to async_hooks
 module.
This API provide a simple CLS-like set
of features.

Co-authored-by: Andrey Pechkurov <[email protected]>

PR-URL: nodejs#26540
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Stephen Belanger <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
targos pushed a commit to targos/node that referenced this pull request Apr 25, 2020
@vdeturckheim @targos
async_hooks: introduce async-context API
f95a2d3
Adding AsyncLocalStorage class to async_hooks
 module.
This API provide a simple CLS-like set
of features.

Co-authored-by: Andrey Pechkurov <[email protected]>

PR-URL: nodejs#26540
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Stephen Belanger <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
targos pushed a commit that referenced this pull request Apr 28, 2020
@vdeturckheim @targos
async_hooks: introduce async-context API
16e8b11
Adding AsyncLocalStorage class to async_hooks
 module.
This API provide a simple CLS-like set
of features.

Co-authored-by: Andrey Pechkurov <[email protected]>

PR-URL: #26540
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Stephen Belanger <[email protected]>
Reviewed-By: Gireesh Punathil <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
@targos targos mentioned this pull request May 2, 2020
v12.17.0 release proposal #33197
Merged
@legendecas legendecas mentioned this pull request May 27, 2020
Avoid name conflict with Web LocalStorage API tc39/proposal-async-context#1
Closed
@vdeturckheim vdeturckheim mentioned this pull request May 27, 2020
AsyncLocalStorage potential issue ? #33580
Closed
@indutny

indutny commented Jul 16, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Providing a reference to another issue here because most interested parties are subscribed to this issue.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Flarna Flarna Flarna left review comments
@legendecas legendecas legendecas left review comments
@cjihrig cjihrig cjihrig left review comments
@benjamingr benjamingr benjamingr left review comments
@gabrielschulhof gabrielschulhof gabrielschulhof left review comments
@addaleax addaleax addaleax left review comments
@Trott Trott Trott left review comments
@jasnell jasnell jasnell approved these changes
@Qard Qard Qard approved these changes
@mcollina mcollina mcollina approved these changes
@gireeshpunathil gireeshpunathil gireeshpunathil approved these changes
@misterdjules misterdjules Awaiting requested review from misterdjules
@bmeurer bmeurer Awaiting requested review from bmeurer
@BridgeAR BridgeAR Awaiting requested review from BridgeAR
+7 more reviewers
@wentout wentout wentout left review comments
@puzpuzpuz puzpuzpuz puzpuzpuz left review comments
@mhdawson mhdawson mhdawson left review comments
@devsnek devsnek devsnek left review comments
@bajtos bajtos bajtos left review comments
@targos targos targos approved these changes
@mscdex mscdex mscdex left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

async_hooks Issues and PRs related to the async hooks subsystem. build Issues and PRs related to Node.js builds or CI infrastructure. notable-change PRs with changes that should be highlighted in changelogs. semver-minor PRs that contain new features and should be released in the next minor version.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

20 participants

@vdeturckheim @nodejs-github-bot @addaleax @mcollina @jasnell @vsemozhetbyt @devsnek @targos @othiym23 @Flarna @mhdawson @kibertoad @wentout @puzpuzpuz @Shahor @sam-github @Trott @benjamingr @Qard @bajtos
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.