GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — stream: fixes async iterator destroyed error propagation by ronag · Pull Request #31314 · nodejs/node

Conversation

@ronag

@ronag ronag commented Jan 11, 2020

Copy link
Copy Markdown
Member

There was an edge case where if _destroy calls the error callback
later than one tick the iterator would complete early and not
propgate the error.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bricss reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@ronag

ronag commented Jan 11, 2020

Copy link
Copy Markdown
Member Author

ping @nodejs/streams bug?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ronag
ronag force-pushed the fix-async-iterator-delayed-error branch 3 times, most recently from 8613847 to 4eaa164 Compare January 11, 2020 12:44
@ronag
stream: fix async iterator destroyed error propagation
a09e59e
There was an edge case where if _destroy calls the error callback
later than one tick the iterator would complete early and not
propgate the error.
@ronag
ronag force-pushed the fix-async-iterator-delayed-error branch from 4eaa164 to a09e59e Compare January 11, 2020 12:54
mcollina
mcollina approved these changes Jan 11, 2020
View reviewed changes

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@ronag
fixup: lint error
5698700
BridgeAR
BridgeAR approved these changes Jan 11, 2020
View reviewed changes
Comment thread test/parallel/test-stream-readable-async-iterators.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot

This comment has been minimized.

Sign in to view
@BridgeAR BridgeAR added author ready PRs with CI started, the required approvals, and no outstanding review comments. stream Issues and PRs related to Node.js streams. labels Jan 11, 2020
@nodejs-github-bot

This comment has been minimized.

Sign in to view
@nodejs-github-bot

This comment has been minimized.

Sign in to view
@nodejs-github-bot

nodejs-github-bot commented Jan 12, 2020

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/28399/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ronag
fixup: ref compare error instead of message
4e7d5a8
@nodejs-github-bot

nodejs-github-bot commented Jan 12, 2020

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/28411/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Trott
Trott approved these changes Jan 13, 2020
View reviewed changes
JungMinu
JungMinu approved these changes Jan 13, 2020
View reviewed changes
jasnell
jasnell approved these changes Jan 13, 2020
View reviewed changes
Trott pushed a commit that referenced this pull request Jan 14, 2020
@ronag @Trott
stream: fix async iterator destroyed error propagation
d15b8ea
There was an edge case where if _destroy calls the error callback
later than one tick the iterator would complete early and not
propgate the error.

PR-URL: #31314
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Minwoo Jung <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@Trott

Trott commented Jan 14, 2020

Copy link
Copy Markdown
Member

Landed in d15b8ea

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Trott Trott closed this Jan 14, 2020
MylesBorins pushed a commit that referenced this pull request Jan 16, 2020
@ronag @MylesBorins
stream: fix async iterator destroyed error propagation
499c41d
There was an edge case where if _destroy calls the error callback
later than one tick the iterator would complete early and not
propgate the error.

PR-URL: #31314
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Minwoo Jung <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@codebytere codebytere mentioned this pull request Jan 16, 2020
v13.7.0 proposal #31382
Merged
@mcollina mcollina added dont-land-on-v10.x labels Jan 25, 2020
@mcollina

mcollina commented Jan 25, 2020

Copy link
Copy Markdown
Member

Please do not backport this PR. It's causing a regression on a very simple usage of the async iterator:

use strict'

const { Readable } = require('stream')

const r = new Readable({
  read () {}
})

r.destroy()

async function run () {

  for await (let chunk of r) {
  }
}

r.on('close', () => {
  console.log('close emitted')
  run().then(() => console.log('finished'))
})

I'll issue a revert and add a test for it.


This is not easy to fix, as we do not have a 'closeEmitted' or 'destroyCompleted' status, so we cannot know if the destroy callback finished. Note that .destroyed is set syncrhonously. The problem is that we call finished() waiting for the callback to fire - and it never will.
Fixing this is likely semver-major, hence the coming revert.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mcollina mcollina mentioned this pull request Jan 25, 2020
stream: fix async iterator return when destroyed earlier #31508
Closed
3 tasks done
mcollina added a commit to mcollina/node that referenced this pull request Jan 25, 2020
@mcollina
stream: add regression test for async iteration completion
638ba66
A test was missing for an async iterator created after the stream
had emitted 'close'. This was regressed by nodejs#31314.

See: nodejs#31314
@ronag ronag mentioned this pull request Jan 25, 2020
stream: finished should invoke callback for closed streams #31509
Closed
4 tasks done
Trott pushed a commit that referenced this pull request Jan 29, 2020
@mcollina @Trott
stream: add regression test for async iteration completion
24e81d7
A test was missing for an async iterator created after the stream
had emitted 'close'. This was regressed by #31314.

See: #31314

PR-URL: #31508
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anatoli Papirovski <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
@AliMirlou AliMirlou mentioned this pull request Jan 29, 2020
Not working on Node.js 13 ipfs-inactive/js-ipfs-http-client#1194
Closed
ronag added a commit to nxtedition/node that referenced this pull request Feb 8, 2020
@ronag
stream: fix async iterator destroyed error propagation
4bf0228
There was an edge case where if _destroy calls the error callback
later than one tick the iterator would complete early and not
propgate the error.

PR-URL: nodejs#31314
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Minwoo Jung <[email protected]>
Reviewed-By: James M Snell <[email protected]>
ronag added a commit that referenced this pull request Feb 14, 2020
@ronag
stream: fix async iterator destroyed error order
0875837
There was an edge case where if _destroy calls the error callback
later than one tick the iterator would complete early and not
propgate the error.

PR-URL: #31314
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Minwoo Jung <[email protected]>
Reviewed-By: James M Snell <[email protected]>

PR-URL: #31700
Reviewed-By: Anna Henningsen <[email protected]>
codebytere pushed a commit that referenced this pull request Feb 17, 2020
@mcollina @codebytere
stream: add regression test for async iteration completion
80e75ab
A test was missing for an async iterator created after the stream
had emitted 'close'. This was regressed by #31314.

See: #31314

PR-URL: #31508
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anatoli Papirovski <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
targos pushed a commit that referenced this pull request Apr 18, 2020
@mcollina @targos
stream: add regression test for async iteration completion
d07dd31
A test was missing for an async iterator created after the stream
had emitted 'close'. This was regressed by #31314.

See: #31314

PR-URL: #31508
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Anatoli Papirovski <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BridgeAR BridgeAR BridgeAR approved these changes
@mcollina mcollina mcollina approved these changes
@jasnell jasnell jasnell approved these changes
@Trott Trott Trott approved these changes
@JungMinu JungMinu JungMinu approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. stream Issues and PRs related to Node.js streams.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@ronag @nodejs-github-bot @Trott @mcollina @jasnell @JungMinu @BridgeAR
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.