GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-40007: Make asyncio.transport.writelines on selector use sendmsg by tzickel · Pull Request #19062 · python/cpython

Conversation

@tzickel

@tzickel tzickel commented Mar 18, 2020 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

https://bugs.python.org/issue40007

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@tzickel
tzickel requested review from 1st1 and asvetlov as code owners March 18, 2020 20:58
@bedevere-bot bedevere-bot added the awaiting review label Mar 18, 2020
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Mar 18, 2020
@tzickel tzickel changed the title bpo-XXX: Make asyncio.transport.writelines on selector use sendmsg bpo-40007: Make asyncio.transport.writelines on selector use sendmsg Mar 18, 2020
@tzickel
tzickel force-pushed the asynciowritev branch from 548f8e3 to 1a399bc Compare March 18, 2020 21:01
aeros
aeros reviewed Mar 19, 2020
View reviewed changes

@aeros aeros left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @tzickel.

Assuming the general idea is approved of by Yury and/or Andrew, this PR will likely need to include a unit test that ensures it behaves as expected; it would be best included in Lib/test/test_asyncio/test_selector_events.py. See the existing transport tests in there for examples, such as the ones that start with test_write*.

The PR could also use a Misc/NEWS entry to briefly explain the changes.

In the meantime, I have a few suggestions/general comments:

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/asyncio/selector_events.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/selector_events.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/selector_events.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/selector_events.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Lib/asyncio/selector_events.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@aeros

aeros commented Mar 19, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Also, the current CI failures will have to be addressed of course. My comments above were just concerns that I noticed at a glance. I suspect that some of the failures may have been occurring as a result of the seld._buffer code typos that I mentioned, but I haven't looked over them yet.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tzickel
tzickel force-pushed the asynciowritev branch from 1a399bc to 7a2cb96 Compare March 19, 2020 06:46
@tzickel
bpo-40007: Make asyncio.transport.writelines on selector use sendmsg
8ea34d2
@tzickel
tzickel force-pushed the asynciowritev branch from 7a2cb96 to 8ea34d2 Compare March 19, 2020 08:45
@tzickel

tzickel commented Mar 19, 2020

Copy link
Copy Markdown
Contributor Author

@aeros Thanks for your comments, I've hopefully fixed the code / logic mistakes.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@aeros

aeros commented Mar 20, 2020

Copy link
Copy Markdown
Contributor

@tzickel

Thanks, but it's a small one-time overhead instead of a small overhead on each object initialization

I wasn't referring to repeating the getattr(socket.socket, "sendmsg", False) every time the object is created; that's not necessary. Here's an example of what I had in mind:

Current:

sendmsg = getattr(socket.socket, "sendmsg", False)

# [snip]
def writelines(self, lines):
    if not sendmsg:
        return self.write(b''.join(lines))
    # [snip]

Recommended:

# Used to determine if platform supports sendmsg to sockets
# [_NAME is our typical convention for internal globals, to differentiate it]
_SENDMSG = None

class _SelectorSocketTransport(_SelectorTransport):
    # [snip]

    def __init__(self, loop, sock, protocol, waiter=None,
                 extra=None, server=None):
       
       global _SENDMSG
       # [ensures the `getattr()` is only done once, but doesn't create extra
       # overhead when select_events is imported]
       if _SENDMSG is None:
           _SENDMSG = getattr(socket.socket, "sendmsg", False)
       
      self._sendmsg = _SENDMSG
      # [snip]

    def writelines(self, lines):
        if not self._sendmsg:
            return self.write(b''.join(lines))
        # [snip]

Does that make it more clear?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@aeros

aeros commented Mar 21, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Also, as a side note, there's typically no need to force push over old commits for PR branches in the CPython repo since we squash prior to merging. It can occasionally help if it the history becomes muddled over time, but it can also make changes made to the PR harder to follow for review purposes.

See https://discuss.python.org/t/pep-601-forbid-return-break-continue-breaking-out-of-finally/2239/42. It's also mentioned at the end of https://devguide.python.org/pullrequest/#submitting.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jakirkham jakirkham mentioned this pull request Feb 28, 2021
Using vectorized IO (scatter/gather) MagicStack/uvloop#404
Open
@asvetlov

asvetlov commented Mar 17, 2022

Copy link
Copy Markdown
Contributor

#31871 is the newer resurrection of the idea.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@asvetlov asvetlov closed this Mar 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@1st1 1st1 Awaiting requested review from 1st1 1st1 is a code owner
@asvetlov asvetlov Awaiting requested review from asvetlov asvetlov is a code owner
1 more reviewer
@aeros aeros aeros left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@tzickel @aeros @asvetlov @the-knights-who-say-ni @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.