GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-40334: Produce better error messages on invalid targets by lysnikolaou · Pull Request #20106 · python/cpython

Conversation

@lysnikolaou

@lysnikolaou lysnikolaou commented May 15, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

The following error messages get produced:

  • cannot delete ... for invalid del targets
  • cannot assign to ... for invalid targets in for and with
    statements

Additionally a few cuts were added in various places before the
invocation of the invalid_* rule, in order to speed things
up.

https://bugs.python.org/issue40334

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@lysnikolaou
bpo-40334: Produce better error messages on invalid targets
0dfaa22
The following error messages get produced:
- `cannot delete ...` for invalid `del` targets
- `... is an illegal 'for' target` for invalid targets in for
  statements
- `... is an illegal 'with' target` for invalid targets in
  with statements

Additionally a few `cut`s were added in various places before the
invocation of the `invalid_*` rule, in order to speed things
up.
@lysnikolaou
lysnikolaou requested a review from pablogsal as a code owner May 15, 2020 13:03
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label May 15, 2020
@bedevere-bot bedevere-bot added the awaiting review label May 15, 2020
lysnikolaou and others added 2 commits May 15, 2020 16:33
@lysnikolaou
Fix test_exceptions
d4a9dd6
@blurb-it
📜🤖 Added by blurb_it.
72a1339
pablogsal
pablogsal reviewed May 15, 2020
View reviewed changes
Comment thread Lib/test/test_syntax.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@lysnikolaou
Produce the same error messages with the old parser for with and for …
1b6cf0b
…targets
@lysnikolaou lysnikolaou added the skip news label May 15, 2020
@lysnikolaou

lysnikolaou commented May 15, 2020

Copy link
Copy Markdown
Member Author

Since the error messages will not change I removed the NEWS blurb and added the skip news label. Also updated the initial PR description.

pablogsal reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lysnikolaou
lysnikolaou requested a review from pablogsal May 15, 2020 18:29
@lysnikolaou
Add invalid_group rule
1ad3819
@lysnikolaou
lysnikolaou requested a review from gvanrossum May 15, 2020 18:36
pablogsal
pablogsal reviewed May 15, 2020
View reviewed changes
Comment thread Grammar/python.gram Resolved

Uh oh!

There was an error while loading. Please reload this page.

pablogsal
pablogsal reviewed May 15, 2020
View reviewed changes
Comment thread Grammar/python.gram Resolved

Uh oh!

There was an error while loading. Please reload this page.

pablogsal
pablogsal reviewed May 15, 2020
View reviewed changes
Comment thread Parser/pegen/pegen.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

gvanrossum
gvanrossum reviewed May 15, 2020
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Grammar/python.gram Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Grammar/python.gram Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Parser/pegen/pegen.c Outdated

expr_ty
_PyPegen_get_invalid_target(expr_ty e)
_PyPegen_get_invalid_target(expr_ty e, int del_targets)

@gvanrossum gvanrossum May 15, 2020

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please adopt a naming convention that makes it clear that this is effectively a bool? I was a bit baffled by this for a while. A venerable convention is to use names like is_foo or has_bar.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

lysnikolaou reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

@lysnikolaou lysnikolaou May 16, 2020

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure, if is_del is okay, because the questions "What's del, the target, which target?" comes to mind. What do you think?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
lysnikolaou and others added 3 commits May 16, 2020 12:55
@lysnikolaou @pablogsal
Update Grammar/python.gram
6f30315
Co-authored-by: Pablo Galindo <[email protected]>
@lysnikolaou @pablogsal
Update Parser/pegen/pegen.c
f750aba
Co-authored-by: Pablo Galindo <[email protected]>
@lysnikolaou
Address Guido's feedback
132178f
Correctly identify invalid targets, when there are multiple context
managers in a with statement, avoid SEGFAULT in invalid for targets
and improve paremeter naming in _PyPegen_get_invalid_target.
@lysnikolaou
lysnikolaou requested a review from gvanrossum May 16, 2020 11:58
@lysnikolaou
Inline _PyPegen_get_invalid_for_target
17cc4bc
pablogsal
pablogsal reviewed May 17, 2020
View reviewed changes
Comment thread Parser/pegen/pegen.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

lysnikolaou and others added 2 commits May 17, 2020 05:23
@lysnikolaou
Revert inlining
def0f3d
@pablogsal
Merge remote-tracking branch 'upstream/master' into targets-errors
ee3524b
@pablogsal

pablogsal commented May 17, 2020

Copy link
Copy Markdown
Member

I have updated the PR with the latest changes from #20003

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gvanrossum
gvanrossum reviewed May 17, 2020
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm... I poked another hole here. :-(

>>> for i, (f, g()) in 10: pass
Assertion failed: (e != NULL && e->kind == Compare_kind), function _PyPegen_get_invalid_for_target, file Parser/pegen/pegen.c, line 2110.
Abort trap: 6

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@gvanrossum

gvanrossum commented May 17, 2020

Copy link
Copy Markdown
Member

Simpler example that gets the same error:

>>> for f, g() in 10: pass

And this too:

>>> for f(), g in 10: pass

I recommend extreme caution here. If I can poke holes this quickly, I worry that we may be causing a regression if we rush this in before beta1. We obviously don't have the right testing strategy here.

There's nothing that says we have to get this level of detail right in beta1.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lysnikolaou
Fix searching for invalid for targets, by visiting Compare nodes in _…
4959a12
…PyPegen_get_invalid_target
@lysnikolaou

lysnikolaou commented May 17, 2020

Copy link
Copy Markdown
Member Author

Simpler example that gets the same error:

>>> for f, g() in 10: pass

And this too:

>>> for f(), g in 10: pass

I recommend extreme caution here. If I can poke holes this quickly, I worry that we may be causing a regression if we rush this in before beta1. We obviously don't have the right testing strategy here.

There's nothing that says we have to get this level of detail right in beta1.

I pushed a version that I think is pretty close, but I'm not 100% certain. I agree, that there's no rush to land this and that we should test a bit more.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@isidentical

isidentical commented Jun 10, 2020

Copy link
Copy Markdown
Member

Could you explain this a bit more? I don't get it.

What I meant is that; since this PR already generates 'can't use starred expression here' error messages for some cases, it isn't fully compatible with the old parser. del *a, b was the example I tested both on the old parser and pegen (with this PR applied), and they give different results.

For starred expressions, the message differs from the old parser;

(new parser)
>>> del *a, b
SyntaxError: cannot delete starred
(old parser)
>>> del *a, b
SyntaxError: can't use starred expression here
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lysnikolaou

lysnikolaou commented Jun 10, 2020

Copy link
Copy Markdown
Member Author

Could you explain this a bit more? I don't get it.

What I meant is that; since this PR already generates 'can't use starred expression here' error messages for some cases, it isn't fully compatible with the old parser. del *a, b was the example I tested both on the old parser and pegen (with this PR applied), and they give different results.

For starred expressions, the message differs from the old parser;

(new parser)
>>> del *a, b
SyntaxError: cannot delete starred
(old parser)
>>> del *a, b
SyntaxError: can't use starred expression here

I think I understand now.

Indeed the old parser used to produce can't use starred expression here and the error message has changed since then. But this PR does not change it, it actually just adopts the changed error message which was introduced in #19911.

Also, I can remember @pablogsal mentioning somewhere that changing just the message is okay in terms of backwards-compatibility.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lysnikolaou
Merge branch 'master' into targets-errors
f0f934d
@isidentical

isidentical commented Jun 10, 2020

Copy link
Copy Markdown
Member

Indeed the old parser used to produce can't use starred expression here and the error message has changed since then. But this PR does not change it, it actually just adopts the changed error message which was introduced in #19911.

I see, didn't notice that. Sorry for distraction.

Also, I can remember @pablogsal mentioning somewhere that changing just the message is okay in terms of backwards-compatibility.

So, is this error will be kept (cannot delete starred) or would it be cool to have a patch that customizes errors for starred context (can't use starred expression here)?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lysnikolaou

lysnikolaou commented Jun 10, 2020

Copy link
Copy Markdown
Member Author

I like the cannot delete starred one better. When I see it, I immediately know without even looking at the line of the error that it's got to do with a del statement. On the contrary, can't use starred expression here does not give enough context, I feel. But really, I don't feel very strongly either way. What do the others think?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gvanrossum

gvanrossum commented Jun 10, 2020

Copy link
Copy Markdown
Member

Same here.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lysnikolaou
Merge branch 'master' into targets-errors
ede107e
pablogsal
pablogsal reviewed Jun 18, 2020
View reviewed changes
Comment thread Parser/pegen.h Outdated
FOR_TARGETS
} TARGETS_TYPE;
expr_ty _PyPegen_get_invalid_target(expr_ty e, TARGETS_TYPE targets_type);
#define GET_INVALID_TARGET(e) _PyPegen_get_invalid_target(e, STAR_TARGETS)

@pablogsal pablogsal Jun 18, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should surround _PyPegen_get_invalid_target in a CHECK to make sure this does not return NULL (and not crash in that case).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

lysnikolaou reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
pablogsal
pablogsal approved these changes Jun 18, 2020
View reviewed changes

@pablogsal pablogsal left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reviewing it again I have played with it a bit more (before the CHECK commit) and I was not able to break it in any way so I think we can land it now so more people interact with it and it this way we could detect any non-trivial problems if they exist.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting review labels Jun 18, 2020
@lysnikolaou
Add CHECK* checks to GET_INVALID_*TARGET macros
55b5327
@lysnikolaou
lysnikolaou force-pushed the targets-errors branch from a00b611 to 55b5327 Compare June 18, 2020 22:41
@lysnikolaou

lysnikolaou commented Jun 18, 2020

Copy link
Copy Markdown
Member Author

FYI there is an unrelated CI failure on macOS in test_asyncio.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pablogsal
pablogsal merged commit 01ece63 into python:master Jun 18, 2020
@miss-islington

miss-islington commented Jun 18, 2020

Copy link
Copy Markdown
Contributor

Thanks @lysnikolaou for the PR, and @pablogsal for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting merge label Jun 18, 2020
@miss-islington

miss-islington commented Jun 18, 2020

Copy link
Copy Markdown
Contributor

Sorry, @lysnikolaou and @pablogsal, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 01ece63d42b830df106948db0aefa6c1ba24416a 3.9

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington miss-islington assigned pablogsal Jun 18, 2020
@pablogsal

pablogsal commented Jun 18, 2020

Copy link
Copy Markdown
Member

@lysnikolaou Could you do the manual backport?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lysnikolaou

lysnikolaou commented Jun 18, 2020

Copy link
Copy Markdown
Member Author

@lysnikolaou Could you do the manual backport?

Of course, on it.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@lysnikolaou
lysnikolaou deleted the targets-errors branch June 18, 2020 23:12
lysnikolaou added a commit to lysnikolaou/cpython that referenced this pull request Jun 18, 2020
@lysnikolaou @pablogsal
bpo-40334: Produce better error messages on invalid targets (pythonGH…
cc9106b
…-20106)

The following error messages get produced:
- `cannot delete ...` for invalid `del` targets
- `... is an illegal 'for' target` for invalid targets in for
  statements
- `... is an illegal 'with' target` for invalid targets in
  with statements

Additionally, a few `cut`s were added in various places before the
invocation of the `invalid_*` rule, in order to speed things
up.

Co-authored-by: Pablo Galindo <[email protected]>
(cherry picked from commit 01ece63)
@bedevere-bot

bedevere-bot commented Jun 18, 2020

Copy link
Copy Markdown

GH-20973 is a backport of this pull request to the 3.9 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.9 label Jun 18, 2020
pablogsal added a commit that referenced this pull request Jun 19, 2020
@lysnikolaou @pablogsal
[3.9] bpo-40334: Produce better error messages on invalid targets (GH…
a5442b2
…-20106) (GH-20973)

* bpo-40334: Produce better error messages on invalid targets (GH-20106)

The following error messages get produced:
- `cannot delete ...` for invalid `del` targets
- `... is an illegal 'for' target` for invalid targets in for
  statements
- `... is an illegal 'with' target` for invalid targets in
  with statements

Additionally, a few `cut`s were added in various places before the
invocation of the `invalid_*` rule, in order to speed things
up.

Co-authored-by: Pablo Galindo <[email protected]>
(cherry picked from commit 01ece63)
@stestagg

stestagg commented Jun 20, 2020

Copy link
Copy Markdown
Contributor

FYI, appears to have caused a regression: https://bugs.python.org/issue41060.
This causes with a as b to crash the interpreter

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

fasih pushed a commit to fasih/cpython that referenced this pull request Jun 29, 2020
@lysnikolaou @pablogsal @fasihahmad
bpo-40334: Produce better error messages on invalid targets (pythonGH…
6740d17
…-20106)

The following error messages get produced:
- `cannot delete ...` for invalid `del` targets
- `... is an illegal 'for' target` for invalid targets in for
  statements
- `... is an illegal 'with' target` for invalid targets in
  with statements

Additionally, a few `cut`s were added in various places before the
invocation of the `invalid_*` rule, in order to speed things
up.

Co-authored-by: Pablo Galindo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pablogsal pablogsal pablogsal approved these changes
@gvanrossum gvanrossum Awaiting requested review from gvanrossum

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@pablogsal pablogsal

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

8 participants

@lysnikolaou @pablogsal @gvanrossum @isidentical @miss-islington @bedevere-bot @stestagg @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.