GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-32591: Add native coroutine origin tracking by njsmith · Pull Request #5250 · python/cpython

bpo-32591: Add native coroutine origin tracking #5250

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
1st1 merged 9 commits into python:master from njsmith:bpo-32591-coro-origin-tracking
Jan 21, 2018

Uh oh!

There was an error while loading. Please reload this page.
Merged

bpo-32591: Add native coroutine origin tracking #5250

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
091dc24
Add coro.cr_origin and sys.set_coroutine_origin_tracking_depth
njsmith Jan 20, 2018
d504810
Use coroutine origin information in the unawaited coroutine warning
njsmith Jan 20, 2018
5d8f591
Stop using set_coroutine_wrapper in asyncio debug mode
njsmith Jan 20, 2018
82d0c3e
In BaseEventLoop.set_debug, enable debugging in the correct thread
njsmith Jan 20, 2018
6c7f73a
Add NEWS blurb
njsmith Jan 20, 2018
7738cc4
Guess I should add myself to ACKS at some point
njsmith Jan 20, 2018
2157af3
Address most of Yury's comments
njsmith Jan 21, 2018
b0e52ec
Switch coroutine origin tracking to a get/set API
njsmith Jan 21, 2018
c0feb3b
Make cr_origin a tuple
njsmith Jan 21, 2018
File filter
Filter by extension
Clear filters
Conversations
Failed to load comments.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Diff view
Diff view
Prev Previous commit
Next Next commit
Use coroutine origin information in the unawaited coroutine warning
  • Loading branch information

Uh oh!

There was an error while loading. Please reload this page.

@njsmith
njsmith committed Jan 20, 2018
commit d504810f43ec842c8f34d489f9199cd73d7f6eba

There are no files selected for viewing

4 changes: 4 additions & 0 deletions 4 Include/warnings.h
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,10 @@ PyErr_WarnExplicitFormat(PyObject *category,
#define PyErr_Warn(category, msg) PyErr_WarnEx(category, msg, 1)
#endif

#ifndef Py_LIMITED_API
PyAPI_FUNC(void) _PyErr_WarnUnawaitedCoroutine(PyObject *coro);
Comment thread Outdated

@1st1 1st1 Jan 20, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strictly speaking we don't need PyAPI_FUNC here, as warnings.c|h will be linked with genobject.c and we are not going to use it in extensions.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
#endif

#ifdef __cplusplus
}
#endif
Expand Down
74 changes: 74 additions & 0 deletions 74 Lib/test/test_coroutines.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import types
import unittest
import warnings
import re
Comment thread Outdated

@1st1 1st1 Jan 20, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We sort modules alphabetically.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
from test import support


Expand Down Expand Up @@ -2086,6 +2087,79 @@ def nested():
finally:
sys.set_coroutine_origin_tracking_depth(orig_depth)

def test_origin_tracking_warning(self):
async def corofn():
pass

a1_filename, a1_lineno = self.here()
def a1():
return corofn() # comment in a1
a1_lineno += 2

a2_filename, a2_lineno = self.here()
def a2():
return a1() # comment in a2
a2_lineno += 2

def check(depth, msg):
sys.set_coroutine_origin_tracking_depth(depth)
with warnings.catch_warnings(record=True) as wlist:
a2()
support.gc_collect()
# This might be fragile if other warnings somehow get triggered
# inside our 'with' block... let's worry about that if/when it
# happens.
self.assertTrue(len(wlist) == 1)
self.assertIs(wlist[0].category, RuntimeWarning)
self.assertEqual(msg, str(wlist[0].message))

orig_depth = sys.set_coroutine_origin_tracking_depth(0)
try:
msg = check(0, f"coroutine '{corofn.__qualname__}' was never awaited")
check(1, "".join([
f"coroutine '{corofn.__qualname__}' was never awaited\n",
"Coroutine created at (most recent call last)\n",
f' File "{a1_filename}", line {a1_lineno}, in a1\n',
f' return corofn() # comment in a1',
]))
check(2, "".join([
f"coroutine '{corofn.__qualname__}' was never awaited\n",
"Coroutine created at (most recent call last)\n",
f' File "{a2_filename}", line {a2_lineno}, in a2\n',
f' return a1() # comment in a2\n',
f' File "{a1_filename}", line {a1_lineno}, in a1\n',
f' return corofn() # comment in a1',
]))

finally:
sys.set_coroutine_origin_tracking_depth(orig_depth)

def test_unawaited_warning_when_module_broken(self):
# Make sure we don't blow up too bad if
# warnings._warn_unawaited_coroutine is broken somehow (e.g. because
# of shutdown problems)
async def corofn():
pass

orig_wuc = warnings._warn_unawaited_coroutine
try:
warnings._warn_unawaited_coroutine = lambda coro: 1/0
with support.captured_stderr() as stream:
corofn()
support.gc_collect()
self.assertIn("Exception ignored in", stream.getvalue())
self.assertIn("ZeroDivisionError", stream.getvalue())
self.assertIn("was never awaited", stream.getvalue())

del warnings._warn_unawaited_coroutine
with support.captured_stderr() as stream:
corofn()
support.gc_collect()
self.assertIn("was never awaited", stream.getvalue())

finally:
warnings._warn_unawaited_coroutine = orig_wuc

@support.cpython_only
class CAPITest(unittest.TestCase):

Expand Down
17 changes: 17 additions & 0 deletions 17 Lib/warnings.py
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -488,6 +488,23 @@ def __exit__(self, *exc_info):
self._module._showwarnmsg_impl = self._showwarnmsg_impl


# Private utility function called by _PyErr_WarnUnawaitedCoroutine
def _warn_unawaited_coroutine(coro):
msg_lines = [
f"coroutine '{coro.__qualname__}' was never awaited\n"
]
if coro.cr_origin is not None:
import linecache, traceback
def extract():
for filename, lineno, funcname in reversed(coro.cr_origin):
line = linecache.getline(filename, lineno)
yield (filename, lineno, funcname, line)
msg_lines.append("Coroutine created at (most recent call last)\n")
msg_lines += traceback.format_list(list(extract()))
msg = "".join(msg_lines).rstrip("\n")
warn(msg, category=RuntimeWarning, stacklevel=2, source=coro)
Comment thread

@njsmith njsmith Jan 20, 2018

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small, easy to miss point here: passing source=coro here means that if the user happens to have tracemalloc enabled, its origin traceback will be included in the warning. Kinda neat! It does mean if you have both set_coroutine_origin_tracking_depth and tracemalloc enabled you'll get two at-least-partially-redundant tracebacks in the same warning. I don't know that I care that much about this but FYI.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@1st1 1st1 Jan 20, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a redundant warning is OK. tracemalloc is enabled quite rarely. I also think what you wrote in this comment should be reflected in a code comment in case someone debugs the double warnings issue :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions


# filters contains a sequence of filter 5-tuples
# The components of the 5-tuple are:
# - an action: error, ignore, always, default, module, or once
Expand Down
7 changes: 2 additions & 5 deletions 7 Objects/genobject.c
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,8 @@ _PyGen_Finalize(PyObject *self)
if (gen->gi_code != NULL &&
((PyCodeObject *)gen->gi_code)->co_flags & CO_COROUTINE &&
gen->gi_frame->f_lasti == -1) {
if (!error_value) {
PyErr_WarnFormat(PyExc_RuntimeWarning, 1,
"coroutine '%.50S' was never awaited",
gen->gi_qualname);
}
if (!error_value)
Comment thread Outdated

@1st1 1st1 Jan 20, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always use braces (per PEP 7):

if (!error_value) {
    _PyErr_WarnUnawaitedCoroutine((PyObject *)gen);
}

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
_PyErr_WarnUnawaitedCoroutine((PyObject *)gen);
}
else {
res = gen_close(gen, NULL);
Expand Down
46 changes: 46 additions & 0 deletions 46 Python/_warnings.c
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1153,6 +1153,52 @@ PyErr_WarnExplicitFormat(PyObject *category,
return ret;
}

void
_PyErr_WarnUnawaitedCoroutine(PyObject *coro)
{
_Py_IDENTIFIER(_warn_unawaited_coroutine);
PyObject *fn = NULL, *res = NULL;
int warned = 0;

/* First, we attempt to funnel the warning through
warnings._warn_unawaited_coroutine.

This could raise an exception, due to:
- a bug
- some kind of shutdown-related brokenness
- succeeding, but with an "error" warning filter installed, so the
warning is converted into a RuntimeWarning exception

In the first two cases, we want to print the error (so we know what it
is!), and then print a warning directly as a fallback. In the last
case, we want to print the error (since it's the warning!), but *not*
do a fallback. And after we print the error we can't check for what
type of error it was (because PyErr_WriteUnraisable clears it), so we
need a flag to keep track.

Since this is called from __del__ context, it's careful to never raise
an exception.
*/
fn = get_warnings_attr(&PyId__warn_unawaited_coroutine, 1);
if (fn) {
res = PyObject_CallFunctionObjArgs(fn, coro, NULL);
Comment thread Outdated

@1st1 1st1 Jan 20, 2018

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just add Py_DECREF(fn) after this line.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
if (res || PyErr_ExceptionMatches(PyExc_RuntimeWarning))
warned = 1;
}
Py_XDECREF(fn);
Py_XDECREF(res);

if (PyErr_Occurred())
PyErr_WriteUnraisable(coro);
if (!warned) {
PyErr_WarnFormat(PyExc_RuntimeWarning, 1,
"coroutine '%.50S' was never awaited",
((PyCoroObject *)coro)->cr_qualname);
/* Maybe *that* got converted into an exception */
if (PyErr_Occurred())
PyErr_WriteUnraisable(coro);
}
}

PyDoc_STRVAR(warn_explicit_doc,
"Low-level inferface to warnings functionality.");
Expand Down
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.