GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GH-97696: Initial work on eager tasks by gvanrossum · Pull Request #98137 · python/cpython

Conversation

@gvanrossum

@gvanrossum gvanrossum commented Oct 10, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This defines an event loop method eager_task_factory() that can be made the task factory using loop.set_task_factory(loop.eager_task_factory). It will then return a Future if the coroutine completed (or failed) without ever suspending itself.

As a TEMPORARY demonstration (to be revisited), for now we call this factory from TaskGroup.create_task(), if the loop implementation defines it.

  • Issue: Add eager task creation API to asyncio #97696

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

itamaro reacted with heart emoji
All reactions
  • ❤️ 1 reaction
@gvanrossum
Initial work on eager tasks
5d6b3f1
This is PYthon only, and has no tests or docs.
@bedevere-bot bedevere-bot added the awaiting core review label Oct 10, 2022
@gvanrossum
Make it work with _CTask.
857312c
@gvanrossum gvanrossum mentioned this pull request Oct 10, 2022
Add eager task creation API to asyncio #97696
Closed
1st1
1st1 reviewed Oct 13, 2022
View reviewed changes
Comment thread Lib/asyncio/base_events.py

return task

def eager_task_factory(self, coro, *, name=None, context=None):

@1st1 1st1 Oct 13, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, this doesn't have to be a loop-level method? I imagined this to be a top-level function in the asyncio package.

One would install it like this:

async def main():
    loop = asyncio.get_running_loop()
    loop.set_task_factory(asyncio.eager_task_factory)
    
    # ... the rest of the main() function ...

asyncio.run(main())

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@gvanrossum gvanrossum Oct 14, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't try very hard to make that work yet... The problem is that the new factory has to call the old factory, but (at least in the Python version) the old factory defaults to None and then create_task() just constructs an instance of the Task class. But when there's a different loop I don't know how to do the equivalent thing -- can I just assume that get_task_factory() shouldn't return None if they have a different task class? (Is uvloop's factory ever None?)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@1st1 1st1 Oct 14, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that the new factory has to call the old factory

Why? I don't think this is a requirement. I remember we discussed it with the instagram people and the idea was to have two functions in asyncio:

# The custom eager factory API that instagram wants to use because
# they have a bunch of instrumentation on top of asyncio.Task:
def asyncio.create_eager_task_factory(custom_task_constructor):

    def factory(coro):
        try:
            result = coro.send(None)
        except StopIteration as si:
             f = Future()
             f.set_result(si.value)
             return f
        except Exception as ex:
             f = Future()
             f.set_exception(ex)
             return f
        else:
             return custom_task_constructor(coro, coro_result=result)

    return factory

# The default eager factory API that most users will use:
asyncio.eager_task_factory = create_eager_task_factory(asyncio.Task)

The implementation of loop.set_task_factory and loop.create_task methods would stay the same. We'd have to add the coro_result= keyword-only argument to asyncio.Task (both Python and C versions), but that I think is acceptable.

I just assume that get_task_factory() shouldn't return None if they have a different task class? (Is uvloop's factory ever None?)

I think None is just a marker for the "default factory is used" scenario. I don't think we were ever wanted to chain factories.


Overall I really want this to be implemented not on the event loop, the loop API surface is already very large.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

carljm reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@gvanrossum gvanrossum Oct 14, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, that's a nice solution. If I get back to this before anyone else I'll use that.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
itamaro added a commit to itamaro/cpython that referenced this pull request Feb 1, 2023
@itamaro
Copy pythonGH-98137, make some changes, add benchmarking script, tryi…
0ff6c93
…ng to get it working
itamaro added a commit to itamaro/cpython that referenced this pull request Feb 1, 2023
@itamaro
Copy pythonGH-98137, make some changes, add benchmarking script, tryi…
77f852c
…ng to get it working
@gvanrossum gvanrossum closed this Feb 1, 2023
itamaro added a commit to itamaro/cpython that referenced this pull request Feb 6, 2023
@itamaro
Copy pythonGH-98137, make some changes, add benchmarking script, tryi…
1ea845c
…ng to get it working
@itamaro itamaro mentioned this pull request Feb 6, 2023
gh-97696: DRAFT asyncio eager tasks factory prototype #101613
Closed
4 of 10 tasks
@gvanrossum
gvanrossum deleted the asyncio-eager-tasks branch August 17, 2023 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@1st1 1st1 1st1 left review comments
@asvetlov asvetlov Awaiting requested review from asvetlov asvetlov will be requested when the pull request is marked ready for review asvetlov is a code owner
@kumaraditya303 kumaraditya303 Awaiting requested review from kumaraditya303 kumaraditya303 will be requested when the pull request is marked ready for review kumaraditya303 is a code owner
@willingc willingc Awaiting requested review from willingc willingc will be requested when the pull request is marked ready for review willingc is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting core review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@gvanrossum @1st1 @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.