GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — lib: fix blob.stream() causing hanging promises by debadree25 · Pull Request #48232 · nodejs/node

Conversation

@debadree25

@debadree25 debadree25 commented May 28, 2023

Copy link
Copy Markdown
Contributor

Running the following script on node v20.2.0

async function fn() {
  const b = new Blob([1, 2, 3, 4, 5, 5, 6, 6]).stream().getReader();

  const chunks = [];

  while (true) {
    const { done, value } = await b.read();
    chunks.push(value);
    console.log('read value', value);
    if (done) break;
  }

  console.log(chunks);
}

await fn();

would cause a promise hang and no output, In this PR have updated the pulling logic in blob.stream() to keep pulling until we reach the end, similar to how blob.arrayBuffer() is implemented.

Refs: #47993 (comment)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
bricss reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@debadree25
lib: fix blob.stream() causing hanging promises
24ef7d6 
Refs: nodejs#47993 (comment)
@nodejs-github-bot nodejs-github-bot added the needs-ci PRs that need a full CI run. label May 28, 2023
@debadree25 debadree25 added the web streams Issues and PRs related to the Web Streams API. label May 28, 2023
@debadree25

debadree25 commented May 28, 2023

Copy link
Copy Markdown
Contributor Author

cc @nodejs/whatwg-stream
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@debadree25 debadree25 mentioned this pull request May 28, 2023
byob reader support for Blob.stream() #47993
Closed
jasnell
jasnell approved these changes May 29, 2023
View reviewed changes
@debadree25 debadree25 added author ready PRs with CI started, the required approvals, and no outstanding review comments. request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. labels May 29, 2023
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label May 29, 2023
@nodejs-github-bot

nodejs-github-bot commented May 29, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/52001/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
mcollina
mcollina requested changes May 30, 2023
View reviewed changes

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my understanding of the code, this will read all the file without any backpressure kicking in. This is a recipe for memory leaks.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
benjamingr and debadree25 reacted with thumbs up emoji
All reactions
  • 👍 2 reactions
@debadree25

debadree25 commented May 30, 2023

Copy link
Copy Markdown
Contributor Author

Ah good point, trying to add checks for the same
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@debadree25

This comment was marked as outdated.

Sign in to view
@debadree25
fixup! add back pressure
3f321a1
@debadree25

debadree25 commented May 30, 2023

Copy link
Copy Markdown
Contributor Author

Hey @mcollina in the latest commit I have added some manual backpressure checking and a test too, does this look ok?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@debadree25
debadree25 requested a review from mcollina May 30, 2023 13:26
@debadree25 debadree25 removed the author ready PRs with CI started, the required approvals, and no outstanding review comments. label May 30, 2023
benjamingr
benjamingr approved these changes May 30, 2023
View reviewed changes
@debadree25 debadree25 added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label May 31, 2023
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label May 31, 2023
@nodejs-github-bot

nodejs-github-bot commented May 31, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/52013/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@nodejs-github-bot

nodejs-github-bot commented Jun 4, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/52081/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@github-actions github-actions Bot mentioned this pull request Jun 5, 2023
CI Reliability 2023-06-05 nodejs/reliability#586
Open
30 tasks
@nodejs-github-bot

nodejs-github-bot commented Jun 5, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/52097/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@github-actions github-actions Bot mentioned this pull request Jun 6, 2023
CI Reliability 2023-06-06 nodejs/reliability#587
Open
20 tasks
@nodejs-github-bot

nodejs-github-bot commented Jun 7, 2023

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/52140/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@debadree25

debadree25 commented Jun 7, 2023

Copy link
Copy Markdown
Contributor Author

ping @mcollina
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
mcollina
mcollina requested changes Jun 7, 2023
View reviewed changes

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add a test where the flow control mechanism kicks in? I would expect to see c.desiredSize turn negative.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread lib/internal/blob.js Resolved

Uh oh!

There was an error while loading. Please reload this page.
@debadree25 @mcollina
Update lib/internal/blob.js
fd56c64 
Co-authored-by: Matteo Collina <[email protected]>
@debadree25

debadree25 commented Jun 7, 2023

Copy link
Copy Markdown
Contributor Author

Can you please add a test where the flow control mechanism kicks in? I would expect to see c.desiredSize turn negative.

Sure adding tests for that
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@RafaelGSS RafaelGSS mentioned this pull request Jul 3, 2023
v20.4.0 proposal #48643
Merged
@bricss bricss mentioned this pull request Jul 6, 2023
http2: connection hangs forever ♾️ when blob stream is piped 🚰 through client to server #48685
Closed
@bricss bricss mentioned this pull request Jul 22, 2023
Awaiting a Promise between creating a stream and consuming a stream causes Node to crash with code 13 #48668
Closed
bellbind added a commit to bellbind/node that referenced this pull request Jul 26, 2023
@bellbind
fix lib/internal/blob.js for nodejs#48668 nodejs#48916
e8dcf79 
- also work example code in nodejs#48232
@bellbind bellbind mentioned this pull request Jul 26, 2023
lib: fix lib/internal/blob.js Blob.stream() for #48668 #48916 #48930
Closed
bellbind added a commit to bellbind/node that referenced this pull request Jul 26, 2023
@bellbind
lib: fix lib/internal/blob.js Blob.stream() for nodejs#48668 nodejs#4…
b7d9941 
…8916

Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: 8cc1438
@bellbind bellbind mentioned this pull request Jul 27, 2023
lib: fix to add resolve() before return at Blob.stream()'s source.pull() #48935
Closed
@bellbind bellbind mentioned this pull request Jul 27, 2023
lib: fix lacked resolve() in Blob.stream()'s source.pull() #48938
Closed
bellbind added a commit to bellbind/node that referenced this pull request Jul 28, 2023
@bellbind
lib: fix lacked resolve() in Blob.stream()'s source.pull()
0b8ab50 
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: nodejs@8cc1438
debadree25 pushed a commit to bellbind/node that referenced this pull request Aug 12, 2023
@bellbind @debadree25
lib: fix to add resolve() before return at Blob.stream()'s source.pull()
9f5022b 
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: nodejs@8cc1438
debadree25 pushed a commit that referenced this pull request Aug 12, 2023
@bellbind @debadree25
lib: fix to add resolve() before return at Blob.stream()'s source.pull()
3224527 
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: #48668
Fixes: #48916
Fixes: #48232
Refs: 8cc1438
PR-URL: #48935
Reviewed-By: Debadree Chatterjee <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Matthew Aitken <[email protected]>
Ceres6 pushed a commit to Ceres6/node that referenced this pull request Aug 14, 2023
@debadree25 @Ceres6
lib: fix blob.stream() causing hanging promises
3b7c4c9 
Refs: nodejs#47993 (comment)
PR-URL: nodejs#48232
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Ceres6 pushed a commit to Ceres6/node that referenced this pull request Aug 14, 2023
@bellbind @Ceres6
lib: fix to add resolve() before return at Blob.stream()'s source.pull()
c872bf2 
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: nodejs@8cc1438
PR-URL: nodejs#48935
Reviewed-By: Debadree Chatterjee <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Matthew Aitken <[email protected]>
Ceres6 pushed a commit to Ceres6/node that referenced this pull request Aug 14, 2023
@debadree25 @Ceres6
lib: fix blob.stream() causing hanging promises
fe1310b 
Refs: nodejs#47993 (comment)
PR-URL: nodejs#48232
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Ceres6 pushed a commit to Ceres6/node that referenced this pull request Aug 14, 2023
@bellbind @Ceres6
lib: fix to add resolve() before return at Blob.stream()'s source.pull()
3ead299 
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: nodejs@8cc1438
PR-URL: nodejs#48935
Reviewed-By: Debadree Chatterjee <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Matthew Aitken <[email protected]>
RafaelGSS pushed a commit that referenced this pull request Aug 15, 2023
@bellbind @RafaelGSS
lib: fix to add resolve() before return at Blob.stream()'s source.pull()
14e2b2a 
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: #48668
Fixes: #48916
Fixes: #48232
Refs: 8cc1438
PR-URL: #48935
Reviewed-By: Debadree Chatterjee <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Matthew Aitken <[email protected]>
RafaelGSS pushed a commit to RafaelGSS/node that referenced this pull request Aug 15, 2023
@bellbind @RafaelGSS
lib: fix to add resolve() before return at Blob.stream()'s source.pull()
8df3ba6 
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: nodejs#48668
Fixes: nodejs#48916
Fixes: nodejs#48232
Refs: nodejs@8cc1438
PR-URL: nodejs#48935
Reviewed-By: Debadree Chatterjee <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Matthew Aitken <[email protected]>
RafaelGSS pushed a commit that referenced this pull request Aug 16, 2023
@bellbind @RafaelGSS
lib: fix to add resolve() before return at Blob.stream()'s source.pull()
4d6880e 
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: #48668
Fixes: #48916
Fixes: #48232
Refs: 8cc1438
PR-URL: #48935
Reviewed-By: Debadree Chatterjee <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Matthew Aitken <[email protected]>
RafaelGSS pushed a commit that referenced this pull request Aug 17, 2023
@bellbind @RafaelGSS
lib: fix to add resolve() before return at Blob.stream()'s source.pull()
dca8678 
Add lacked calling resolve() for finish ReadableStream source.pull().

Fixes: #48668
Fixes: #48916
Fixes: #48232
Refs: 8cc1438
PR-URL: #48935
Reviewed-By: Debadree Chatterjee <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Matthew Aitken <[email protected]>
@ruyadorno ruyadorno added the backport-requested-v18.x label Sep 7, 2023
@ruyadorno

ruyadorno commented Sep 7, 2023

Copy link
Copy Markdown
Member

This does not land cleanly in v18.x-staging and will need manual backport in case you want this to land on v18.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@debadree25
debadree25 deleted the fix/blob-stream-hang branch September 7, 2023 06:27
@debadree25

debadree25 commented Sep 7, 2023

Copy link
Copy Markdown
Contributor Author

Hey @ruyadorno will open a backport pr for this also note that this PR did introduce a bug 😅 which was fixed by #48935 should i include both the commits in the backport PR?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcollina mcollina mcollina approved these changes
@jasnell jasnell jasnell approved these changes
@benjamingr benjamingr benjamingr approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. commit-queue-squash PRs the Commit Queue should land as one squashed commit. web streams Issues and PRs related to the Web Streams API.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@debadree25 @nodejs-github-bot @ruyadorno @mcollina @jasnell @benjamingr
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.