GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — streams: fix enqueue race condition on esm modules by RafaelGSS · Pull Request #40901 · nodejs/node

Conversation

@RafaelGSS

@RafaelGSS RafaelGSS commented Nov 21, 2021

Copy link
Copy Markdown
Member

Address: #39758

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bricss reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@nodejs-github-bot nodejs-github-bot added the needs-ci PRs that need a full CI run. label Nov 21, 2021
@RafaelGSS RafaelGSS mentioned this pull request Nov 21, 2021
readableStream.tee() doesn't work when using process.nextTick(...) #39758
Closed
szmarczak
szmarczak suggested changes Nov 21, 2021
View reviewed changes

@szmarczak szmarczak left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not the WHATWG streams. Those are implemented correctly according to the spec. It's just that finished uses process.nextTick instead.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@mcollina

mcollina commented Nov 21, 2021

Copy link
Copy Markdown
Member

cc @jasnell

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@RafaelGSS

RafaelGSS commented Nov 21, 2021

Copy link
Copy Markdown
Member Author

@szmarczak is that c4f0058 that you expect?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@RafaelGSS
RafaelGSS force-pushed the fix/readable-web-stream branch from 4e0348f to 303211b Compare November 22, 2021 16:50
@ljharb

ljharb commented Nov 22, 2021

Copy link
Copy Markdown
Member

cc @nodejs/modules

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

targos
targos previously requested changes Nov 22, 2021
View reviewed changes

@targos targos left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The added test doesn't fail before the fix.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bmeck

bmeck commented Nov 22, 2021

Copy link
Copy Markdown
Member

I don't think this is necessarily something Modules needs to do anything with, it looks like it is just because import is doing async work that it is showing up? It does look weird though and IDK if this is the right fix since it looks like the close() operation is happening to close both streams before the 2nd gets the data buffered? Not familiar w/ this streams impl but it doesn't seem to be specific to ESM itself.

ljharb and benjamingr reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@RafaelGSS

RafaelGSS commented Nov 22, 2021

Copy link
Copy Markdown
Member Author

The added test doesn't fail before the fix. -- @targos

You're right, I didn't notice it before the changes. I was using a .mjs to validate locally. Fixing it soon.

if this is the right fix since it looks like the close() operation is happening to close both streams before the 2nd gets the data buffered? -- @bmeck

I felt the same when looking at it, looks like a conditional flag before closing the stream is needed. But, I don't have enough knowledge in the streams/web to assume that.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

szmarczak
szmarczak reviewed Nov 22, 2021
View reviewed changes
Comment thread lib/internal/webstreams/readablestream.js Outdated

@szmarczak szmarczak Nov 22, 2021

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I meant that opposite. I think https://github.com/nodejs/node/blob/master/lib/internal/streams/end-of-stream.js should be queueMicrotask instead of process.nextTick but I'm not sure 🤔

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@RafaelGSS RafaelGSS Nov 23, 2021

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a simple test by just calling console.log on eos and it doesn't call, so not sure either if the end-of-stream is the best place. Looks like the kClose is suitable for that otherwise my second comment #40901 (comment) is valid.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@jasnell jasnell Dec 16, 2021

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a comment here explaining that the addition of the process.nextTick is not part of the spec and why it's needed with a link back to the original issue.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

RafaelGSS reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@RafaelGSS RafaelGSS Dec 17, 2021

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@RafaelGSS
RafaelGSS force-pushed the fix/readable-web-stream branch from 303211b to 37b200c Compare November 22, 2021 21:23
@RafaelGSS

RafaelGSS commented Nov 22, 2021

Copy link
Copy Markdown
Member Author

@targos Just created a new test with fails without this modification (.mjs)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jasnell

jasnell commented Nov 23, 2021

Copy link
Copy Markdown
Member

I'll have to stew over this change a bit. Changing the timing on the close using nextTick just doesn't feel right and changes the timing guarantees from what is spec'd. But, I need to think through about whether that's going to be a problem or not. I'm not giving this a thumbs down, just need to think about it more.

RafaelGSS reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@GeoffreyBooth

GeoffreyBooth commented Nov 23, 2021

Copy link
Copy Markdown
Member

changes the timing guarantees from what is spec’d

I’m also wondering if this is a breaking change.

Also how does equivalent code behave in browsers? This seems like it’s essentially a spec question, as in are we following the spec-defined behavior. If browsers behave differently then Node does now (and like how this PR changes Node to behave) that would be a strong argument for making the change, because it would imply that Node’s implementation is incorrect.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

simoneb
simoneb reviewed Nov 23, 2021
View reviewed changes
Comment thread test/parallel/test-whatwg-readablestream.mjs Resolved

Uh oh!

There was an error while loading. Please reload this page.

@RafaelGSS
RafaelGSS force-pushed the fix/readable-web-stream branch from 37b200c to 4e10155 Compare November 24, 2021 21:30
@ronag

ronag commented Dec 16, 2021

Copy link
Copy Markdown
Member

Please don't forget to label with streams and ping @nodejs/streams on these kind of PRs.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ronag ronag added stream Issues and PRs related to Node.js streams. web streams Issues and PRs related to the Web Streams API. labels Dec 16, 2021
@ronag

ronag commented Dec 16, 2021

Copy link
Copy Markdown
Member

@jasnell Have you considered the following example? #39758 (comment)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ronag
ronag approved these changes Dec 16, 2021
View reviewed changes
@ronag ronag added request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. and removed needs-ci PRs that need a full CI run. labels Dec 16, 2021
jasnell
jasnell approved these changes Dec 16, 2021
View reviewed changes

@jasnell jasnell left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still not 100% sure this is the right approach but LGTM for now with the addition of some comments in the code explaining why the change is made

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Dec 16, 2021
@nodejs-github-bot

nodejs-github-bot commented Dec 16, 2021

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/41516/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mcollina

mcollina commented Dec 16, 2021

Copy link
Copy Markdown
Member

@targos can you take a look again?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@targos
targos dismissed their stale review December 16, 2021 14:26

resolved

@RafaelGSS
RafaelGSS force-pushed the fix/readable-web-stream branch from 4e10155 to f365cdd Compare December 17, 2021 00:46
This was referenced Dec 18, 2021
CI Reliability 2021-12-18 nodejs/reliability#148
Open
CI Reliability 2021-12-19 nodejs/reliability#149
Open
CI Reliability 2021-12-20 nodejs/reliability#150
Open
@nodejs-github-bot

nodejs-github-bot commented Dec 20, 2021

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/41564/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot

nodejs-github-bot commented Dec 20, 2021

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/41565/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mcollina mcollina added the commit-queue PRs queued for automated landing through the Commit Queue. label Dec 20, 2021
@nodejs-github-bot nodejs-github-bot added commit-queue-failed PRs whose Commit Queue landing failed and need manual intervention before retrying. and removed commit-queue PRs queued for automated landing through the Commit Queue. labels Dec 20, 2021
@nodejs-github-bot

nodejs-github-bot commented Dec 20, 2021

Copy link
Copy Markdown
Collaborator
Commit Queue failed
- Loading data for nodejs/node/pull/40901
✔  Done loading data for nodejs/node/pull/40901
----------------------------------- PR info ------------------------------------
Title      streams: fix enqueue race condition on esm modules (#40901)
Author     Rafael Gonzaga  (@RafaelGSS)
Branch     RafaelGSS:fix/readable-web-stream -> nodejs:master
Labels     stream, web streams
Commits    1
 - stream: fix enqueue race condition on esm modules
Committers 1
 - RafaelGSS 
PR-URL: https://github.com/nodejs/node/pull/40901
Reviewed-By: Robert Nagy 
Reviewed-By: James M Snell 
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/40901
Reviewed-By: Robert Nagy 
Reviewed-By: James M Snell 
--------------------------------------------------------------------------------
   ⚠  Commits were pushed since the last review:
   ⚠  - stream: fix enqueue race condition on esm modules
   ℹ  This PR was created on Sun, 21 Nov 2021 02:45:45 GMT
   ✔  Approvals: 2
   ✔  - Robert Nagy (@ronag) (TSC): https://github.com/nodejs/node/pull/40901#pullrequestreview-834119884
   ✔  - James M Snell (@jasnell) (TSC): https://github.com/nodejs/node/pull/40901#pullrequestreview-834121218
   ✔  Last GitHub Actions successful
   ℹ  Last Full PR CI on 2021-12-20T19:16:43Z: https://ci.nodejs.org/job/node-test-pull-request/41565/
- Querying data for job/node-test-pull-request/41565/
   ✔  Last Jenkins CI successful
--------------------------------------------------------------------------------
   ✔  Aborted `git node land` session in /home/runner/work/node/node/.ncu
https://github.com/nodejs/node/actions/runs/1604180725
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@github-actions github-actions Bot mentioned this pull request Dec 21, 2021
CI Reliability 2021-12-21 nodejs/reliability#151
Open
22 tasks
mcollina
mcollina approved these changes Dec 21, 2021
View reviewed changes

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@mcollina mcollina added commit-queue PRs queued for automated landing through the Commit Queue. and removed commit-queue-failed PRs whose Commit Queue landing failed and need manual intervention before retrying. labels Dec 21, 2021
@nodejs-github-bot nodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Dec 21, 2021
@nodejs-github-bot
nodejs-github-bot merged commit 113133b into nodejs:master Dec 21, 2021
@nodejs-github-bot

nodejs-github-bot commented Dec 21, 2021

Copy link
Copy Markdown
Collaborator

Landed in 113133b

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mcollina mcollina added dont-land-on-v12.x labels Dec 21, 2021
This was referenced Dec 22, 2021
CI Reliability 2021-12-22 nodejs/reliability#152
Open
CI Reliability 2021-12-23 nodejs/reliability#153
Open
CI Reliability 2021-12-24 nodejs/reliability#154
Open
CI Reliability 2021-12-25 nodejs/reliability#155
Open
CI Reliability 2021-12-26 nodejs/reliability#156
Open
CI Reliability 2021-12-27 nodejs/reliability#157
Open
targos pushed a commit that referenced this pull request Jan 14, 2022
@RafaelGSS @targos
stream: fix enqueue race condition on esm modules
3dc6564
stream: use nextTick on close

PR-URL: #40901
Reviewed-By: Robert Nagy <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
@targos targos mentioned this pull request Jan 16, 2022
v17.4.0 release proposal #41557
Merged
danielleadams pushed a commit that referenced this pull request Jan 31, 2022
@RafaelGSS @danielleadams
stream: fix enqueue race condition on esm modules
9525b0d
stream: use nextTick on close

PR-URL: #40901
Reviewed-By: Robert Nagy <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
danielleadams pushed a commit that referenced this pull request Feb 1, 2022
@RafaelGSS @danielleadams
stream: fix enqueue race condition on esm modules
8cf507a
stream: use nextTick on close

PR-URL: #40901
Reviewed-By: Robert Nagy <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
@danielleadams danielleadams mentioned this pull request Feb 1, 2022
v16.14.0 proposal #41804
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcollina mcollina mcollina approved these changes
@jasnell jasnell jasnell approved these changes
@ronag ronag ronag approved these changes
+3 more reviewers
@simoneb simoneb simoneb left review comments
@szmarczak szmarczak szmarczak requested changes
@targos targos targos left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

stream Issues and PRs related to Node.js streams. web streams Issues and PRs related to the Web Streams API.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

11 participants

@RafaelGSS @mcollina @ljharb @bmeck @jasnell @GeoffreyBooth @ronag @nodejs-github-bot @simoneb @targos @szmarczak
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.