GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — stream: Expose DuplexPair API by awwright · Pull Request #34111 · nodejs/node

Conversation

@awwright

@awwright awwright commented Jun 29, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

The "Duplex Pair" pattern is a function that creates two symmetrical Duplex streams, such that what is written on one side will become readable on the other.

This pattern is used by the tls module in core, and in a great number of the tests. This PR merges the two implementations together into a single stream.DuplexPair implementation.

See #29986 for a discussion of this feature.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

haynesgt and worstpractice reacted with heart emoji bricss, himself65, and worstpractice reacted with eyes emoji
All reactions
  • ❤️ 2 reactions
  • 👀 3 reactions
@nodejs-github-bot nodejs-github-bot added the lib / src Issues and PRs involving general changes in the lib/ or src/ directories. label Jun 29, 2020
@awwright
awwright force-pushed the makeDuplexPair branch from b6946fd to 207c12d Compare June 29, 2020 00:09
@awwright awwright changed the title Expose DuplexPair API stream: Expose DuplexPair API Jun 29, 2020
@mscdex

mscdex commented Jun 29, 2020

Copy link
Copy Markdown
Contributor

Why should we be exposing this publicly? Is this a common use case? Is there a reference issue with more information?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@awwright
awwright force-pushed the makeDuplexPair branch from 207c12d to 3e98449 Compare June 29, 2020 00:43
@awwright

awwright commented Jun 29, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

@mscdex I think it's a much more familiar way of creating a Duplex stream for another party, instead of having to use the push/_write/_read/_destroy/_final API. I think there'd be a great benefit if more people moved in the direction of creating a stream using a symmetrical API... see #29986 for a discussion; @addaleax suggested filing a PR over there.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

awwright
awwright commented Jun 29, 2020
View reviewed changes
Comment thread test/parallel/test-http2-session-gc-while-write-scheduled.js Outdated

@awwright awwright Jun 29, 2020

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would anyone like to speculate why this test creates a duplex pair and then never uses the other side?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
mscdex
mscdex reviewed Jun 29, 2020
View reviewed changes
Comment thread lib/_stream_pair.js Outdated

@mscdex mscdex Jun 29, 2020

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we avoid adding this for new APIs? We may as well use a class, which would force new anyway.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

himself65 and addaleax reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

@awwright awwright Jun 29, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'll check it out.
Does Node.js do it this way anywhere else? I was trying to copy how the other stream classes work.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@addaleax addaleax Jun 29, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use class for almost everything since class has been introduced into the language. Unfortunately, we cannot convert this old-style pattern to ES6 classes without a breaking change.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@awwright awwright Jun 29, 2020

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax Would it be possible to get a deprecation notice for that, then? Or is there some documentation on what causes the breaking change? (My understanding was that class is just syntactic sugar.)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@addaleax addaleax Jun 29, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax Would it be possible to get a deprecation notice for that, then?

I don’t think this is something that we’re ever realistically going to be able to change, so there’s no real reason to deprecated it, I assume.

Or is there some documentation on what causes the breaking change? (My understanding was that class is just syntactic sugar.)

It’s the fact that classes have to be invoked with new, and will throw otherwise. That”s not the case in the current code here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@awwright awwright Jun 29, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax Thanks! I'm working on that now.

Sort of off-topic follow-up, is calling builtins without new discouraged too then? e.g. const now = Date();? I've been in discussions with a couple core members on the IRC channel, who've said more-or-less "new is obsolete these days" but I assume that was just a personal opinion around various programming styles.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@addaleax addaleax Jun 29, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@awwright Date is special, because Date() and new Date() do different things :)

But yes, for a function that implements a pattern like this (returns an instance of itself even when called without new), I would recommend using new because that allows transitioning to classes.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@awwright awwright Jun 29, 2020

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying this code:

class DuplexPair {
  [Symbol.iterator] = Array.prototype[Symbol.iterator];
  length = 2;
  constructor() {
    const clientSide = this[0] = new DuplexSide();
    const serverSide = this[1] = new DuplexSide();
    this.serverSide = clientSide[kOtherSide] = serverSide;
    this.clientSide = serverSide[kOtherSide] = clientSide;
  }
}

This passes the test-release suite, but the acorn dependency can't grok it ("SyntaxError: unexpected token (42:20)" which is at the first closing square bracket). Am I missing something here?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@addaleax addaleax Jun 29, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@awwright Yeah, we’ve run into trouble with acorn in the past for private properties as well, so I guess this falls into a similar category :/ Fwiw, I think if you do this, you could also use class DuplexPair extends Array, to automatically get length and the iterator property…

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

awwright reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@mscdex

mscdex commented Jun 29, 2020

Copy link
Copy Markdown
Contributor

If we're going to make this public, I think we should be using more generic property names instead of 'client' and 'server' since it's no longer specific to tls.

addaleax reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@awwright

awwright commented Jun 29, 2020

Copy link
Copy Markdown
Contributor Author

@mscdex I considered exposing socket1 and socket2, which is what tls uses (the test suite uses clientSide and serverSide). I settled on additionally exposing an array called pair so that you can pick any name you want:

const [ socket1, socket2 ] = new DuplexPair().pair;

This is what TLS now does, in the patch. Do you think that's sufficient?

Some other ideas:

  • make the while object iterable, so you can do e.g. const [ a, b ] = DuplexPair();
  • socket1 and socket2 (what tls uses)
  • upstream and downstream
  • inside and outside
  • a and b (for brevity when doing const { a: socket1, b: socket2 } = ...)

I think it'd be OK to expose multiple names so applications can pick one that looks most relevant.


To illustrate my thought process, I think it's important to have this be a class like Duplex or PassThrough is, because I'm playing with an idea for a SimplexPair class, which is nothing more than a PassThrough with the addition of SimplexPair#readableSide and SimplexPair#writableSide properties that are only Readable and Writable, respectively.

With this, you can do patterns that maintain encapsulation, like

function generate(){
   const stream = new SimplexPair;
   stream.write("foo");
   return stream.readableSide;
}
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@addaleax

addaleax commented Jun 29, 2020

Copy link
Copy Markdown
Member
  • socket1 and socket2 (what tls uses)

I like this because it matches port1 and port2 from new MessageChannel(). (Alternatively: stream1 and stream2.)

  • make the while object iterable, so you can do e.g. const [ a, b ] = DuplexPair();

Fwiw, that’s not exclusive with any naming choices. I like it. 👍

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mscdex

mscdex commented Jun 29, 2020

Copy link
Copy Markdown
Contributor

I'd prefer something even more generic than socket1 and socket2 because this interface doesn't utilize net in any way. Maybe something as simple as side1 and side2?

awwright reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@awwright
awwright force-pushed the makeDuplexPair branch from 3e98449 to d0feea3 Compare June 29, 2020 20:07
@awwright

awwright commented Jun 29, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Check my work on the iterable here:

function DuplexPair() {
  if (!(this instanceof DuplexPair))
    return new DuplexPair();
  const clientSide = this[0] = new DuplexSide();
  const serverSide = this[1] = new DuplexSide();
  this.length = 2;
  this[Symbol.iterator] = Array.prototype[Symbol.iterator];
  clientSide[kOtherSide] = serverSide;
  serverSide[kOtherSide] = clientSide;
}

afaict, this should behave the same as an Array, and still permit iterating over the other keys in a for..in loop.

I like side1 and side2 better than anything else, except I have this hesitation because the numbers don't line up with the array indices, and I'm slightly personally attached to serverSide/clientSide, since a lot of applications will fall into this paradigm.

Do we need the named properties at all, though?

If performance with the iterator function is a concern, you can still do

const { 0: side1, 1: side2 } = new DuplexPair;

How does this sound?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@awwright
awwright force-pushed the makeDuplexPair branch 6 times, most recently from 28d7814 to 892f4e0 Compare July 1, 2020 03:19
@awwright
awwright requested review from addaleax and mscdex July 1, 2020 19:27
awwright
awwright commented Jul 1, 2020
View reviewed changes
Comment thread lib/tls.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

addaleax
addaleax reviewed Jul 1, 2020
View reviewed changes
Comment thread lib/_stream_duplexpair.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/_stream_duplexpair.js Resolved

Uh oh!

There was an error while loading. Please reload this page.

@awwright
awwright force-pushed the makeDuplexPair branch from ee5cf57 to 0c116a9 Compare July 1, 2020 20:12
@awwright awwright closed this Jul 1, 2020
@awwright
awwright force-pushed the makeDuplexPair branch from 0c116a9 to 6213fce Compare July 1, 2020 20:41
@awwright awwright reopened this Jul 1, 2020
addaleax
addaleax approved these changes Jul 1, 2020
View reviewed changes
@addaleax

addaleax commented Jul 7, 2020

Copy link
Copy Markdown
Member

@nodejs/streams Any further thoughts on this?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@awwright
stream: expose DuplexPair API
594f2cd
@awwright
awwright force-pushed the makeDuplexPair branch from 26f9013 to 594f2cd Compare July 22, 2024 20:01
@mcollina mcollina added semver-minor PRs that contain new features and should be released in the next minor version. request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. labels Jul 23, 2024
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Jul 23, 2024
@nodejs-github-bot

nodejs-github-bot commented Jul 23, 2024

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/60551/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot

nodejs-github-bot commented Jul 25, 2024

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/60622/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nodejs-github-bot

nodejs-github-bot commented Jul 26, 2024

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/60640/

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

mcollina
mcollina approved these changes Jul 26, 2024
View reviewed changes

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@mcollina mcollina added the notable-change PRs with changes that should be highlighted in changelogs. label Jul 26, 2024
@github-actions

github-actions Bot commented Jul 26, 2024

Copy link
Copy Markdown
Contributor

The notable-change PRs with changes that should be highlighted in changelogs. label has been added by @mcollina.

Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. Otherwise, the commit will be placed in the Other Notable Changes section.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mcollina mcollina added the commit-queue PRs queued for automated landing through the Commit Queue. label Jul 26, 2024
@nodejs-github-bot nodejs-github-bot removed the commit-queue PRs queued for automated landing through the Commit Queue. label Jul 26, 2024
@nodejs-github-bot
nodejs-github-bot merged commit b32732b into nodejs:main Jul 26, 2024
@nodejs-github-bot

nodejs-github-bot commented Jul 26, 2024

Copy link
Copy Markdown
Collaborator

Landed in b32732b

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

targos pushed a commit that referenced this pull request Jul 28, 2024
@awwright @targos
stream: expose DuplexPair API
f5633b1
PR-URL: #34111
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
@RafaelGSS RafaelGSS mentioned this pull request Jul 30, 2024
v22.6.0 proposal #54123
Merged
RafaelGSS added a commit that referenced this pull request Jul 30, 2024
@RafaelGSS
2024-07-31, Version 22.6.0 (Current)
d77f596
Notable changes:

deps:
  * (SEMVER-MINOR) V8: backport 7857eb34db42 (Stephen Belanger) #53997
http:
  * (SEMVER-MINOR) add diagnostics channel `http.client.request.error` (Kohei Ueno) #54054
inspector:
  * (SEMVER-MINOR) add initial support for network inspection (Kohei Ueno) #53593
lib,src:
  * drop --experimental-network-imports (Rafael Gonzaga) #53822
meta:
  * add jake to collaborators (jakecastelli) #54004
module:
  * (SEMVER-MINOR) add --experimental-strip-types (Marco Ippolito) #53725
  * (SEMVER-MINOR) unflag detect-module (Geoffrey Booth) #53619
stream:
  * (SEMVER-MINOR) expose DuplexPair API (Austin Wright) #34111
test_runner:
  * (SEMVER-MINOR) fix support watch with run(), add globPatterns option (Matteo Collina) #53866
  * (SEMVER-MINOR) refactor snapshots to get file from context (Colin Ihrig) #53853
  * (SEMVER-MINOR) add context.filePath (Colin Ihrig) #53853

PR-URL: #54123
RafaelGSS pushed a commit that referenced this pull request Aug 5, 2024
@awwright @RafaelGSS
stream: expose DuplexPair API
15816bd
PR-URL: #34111
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
RafaelGSS added a commit that referenced this pull request Aug 5, 2024
@RafaelGSS
2024-08-06, Version 22.6.0 (Current)
b096e41
Notable changes:

deps:
  * (SEMVER-MINOR) V8: backport 7857eb34db42 (Stephen Belanger) #53997
http:
  * (SEMVER-MINOR) add diagnostics channel `http.client.request.error` (Kohei Ueno) #54054
inspector:
  * (SEMVER-MINOR) add initial support for network inspection (Kohei Ueno) #53593
lib,src:
  * drop --experimental-network-imports (Rafael Gonzaga) #53822
meta:
  * add jake to collaborators (jakecastelli) #54004
module:
  * (SEMVER-MINOR) add --experimental-strip-types (Marco Ippolito) #53725
stream:
  * (SEMVER-MINOR) expose DuplexPair API (Austin Wright) #34111
test_runner:
  * (SEMVER-MINOR) fix support watch with run(), add globPatterns option (Matteo Collina) #53866
  * (SEMVER-MINOR) refactor snapshots to get file from context (Colin Ihrig) #53853
  * (SEMVER-MINOR) add context.filePath (Colin Ihrig) #53853

PR-URL: #54123
RafaelGSS added a commit that referenced this pull request Aug 6, 2024
@RafaelGSS
2024-08-06, Version 22.6.0 (Current)
3660ad5
Notable changes:

deps:
  * (SEMVER-MINOR) V8: backport 7857eb34db42 (Stephen Belanger) #53997
http:
  * (SEMVER-MINOR) add diagnostics channel `http.client.request.error` (Kohei Ueno) #54054
inspector:
  * (SEMVER-MINOR) add initial support for network inspection (Kohei Ueno) #53593
lib,src:
  * drop --experimental-network-imports (Rafael Gonzaga) #53822
meta:
  * add jake to collaborators (jakecastelli) #54004
module:
  * (SEMVER-MINOR) add --experimental-strip-types (Marco Ippolito) #53725
stream:
  * (SEMVER-MINOR) expose DuplexPair API (Austin Wright) #34111
test_runner:
  * (SEMVER-MINOR) fix support watch with run(), add globPatterns option (Matteo Collina) #53866
  * (SEMVER-MINOR) refactor snapshots to get file from context (Colin Ihrig) #53853
  * (SEMVER-MINOR) add context.filePath (Colin Ihrig) #53853

PR-URL: #54123
@marco-ippolito marco-ippolito mentioned this pull request Aug 19, 2024
v20.17.0 proposal #54447
Merged
marco-ippolito pushed a commit that referenced this pull request Aug 19, 2024
@awwright @marco-ippolito
stream: expose DuplexPair API
7c21bb9
PR-URL: #34111
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcollina mcollina mcollina approved these changes
@jasnell jasnell jasnell approved these changes
@addaleax addaleax addaleax approved these changes
@ronag ronag Awaiting requested review from ronag
@Mifrill Mifrill Awaiting requested review from Mifrill
+2 more reviewers
@mscdex mscdex mscdex left review comments
@devsnek devsnek devsnek left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

notable-change PRs with changes that should be highlighted in changelogs. semver-minor PRs that contain new features and should be released in the next minor version. stream Issues and PRs related to Node.js streams.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

10 participants

@awwright @mscdex @addaleax @ronag @mcollina @jasnell @nodejs-github-bot @aduh95 @devsnek @Mifrill
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.