GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-104050: Annotate more Argument Clinic DSLParser state methods by erlend-aasland · Pull Request #106376 · python/cpython

Conversation

@erlend-aasland

@erlend-aasland erlend-aasland commented Jul 3, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Annotate the following methods:

  • state_parameter()
  • state_parameter_docstring_start()
  • Issue: Add type annotations to clinic.py #104050

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@erlend-aasland
Annotate Argument Clinic DSLParser.state_parameter()
fa43e47
@erlend-aasland
erlend-aasland requested a review from AlexWaygood July 3, 2023 21:09
@erlend-aasland erlend-aasland changed the title Annotate Argument Clinic DSLParser.state_parameter() gh-104050: Annotate Argument Clinic DSLParser.state_parameter() Jul 3, 2023
@bedevere-bot bedevere-bot mentioned this pull request Jul 3, 2023
Add type annotations to clinic.py #104050
Closed
@erlend-aasland

erlend-aasland commented Jul 3, 2023

Copy link
Copy Markdown
Contributor Author

@AlexWaygood, can you help me with the last bits on this PR?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood

AlexWaygood commented Jul 3, 2023

Copy link
Copy Markdown
Member

@AlexWaygood, can you help me with the last bits on this PR?

Taking a look now :)

erlend-aasland reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood

AlexWaygood commented Jul 3, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This diff fixes the first two mypy errors:

--- a/Tools/clinic/clinic.py
+++ b/Tools/clinic/clinic.py
@@ -4846,7 +4846,9 @@ def state_parameter(self, line: str | None) -> None:
         if not module:
             fail("Function " + self.function.name + " has an invalid parameter declaration:\n\t" + line)

-        function_args = module.body[0].args
+        function = module.body[0]
+        assert isinstance(function, ast.FunctionDef)
+        function_args = function.args

         if len(function_args.args) > 1:
             fail("Function " + self.function.name + " has an invalid parameter declaration (comma?):\n\t" + line)
@@ -4931,7 +4933,9 @@ def bad_node(self, node):
                 if bad:
                     fail("Unsupported expression as default value: " + repr(default))

-                expr = module.body[0].value
+                assignment = module.body[0]
+                assert isinstance(assignment, ast.Assign)
+                expr = assignment.value
                 # mild hack: explicitly support NULL as a default value
                 c_default: str | None
                 if isinstance(expr, ast.Name) and expr.id == 'NULL':

In both cases, mypy is flagging that, in principle, an ast.Module node can have any kind of ast.stmt nodes in its .body. Not all ast.stmt instances have a .args attribute or a .value attribute (only instances of certain subclasses of ast.stmt do), so we need to do some kind of type narrowing here to help mypy out. In both cases, we can be confident that these assertions are safe. For the first one, the source code that we're parsing is generated here:

cpython/Tools/clinic/clinic.py

Lines 4849 to 4850 in 71b4044

ast_input = f"def x({base}): pass"
module = ast.parse(ast_input)

For the second one, the source code that we're parsing is generated here:

cpython/Tools/clinic/clinic.py

Lines 4902 to 4904 in 71b4044

ast_input = f"x = {default}"
try:
module = ast.parse(ast_input)

So we can be 100% confident that in the first case, module.body[0] will always be an instance of ast.FunctionDef, and in the second case, module.body[0] will always be an instance of ast.Assign.


This diff fixes the last mypy error:

--- a/Tools/clinic/clinic.py
+++ b/Tools/clinic/clinic.py
@@ -4970,7 +4974,7 @@ def bad_node(self, node):
                 else:
                     value = ast.literal_eval(expr)
                     py_default = repr(value)
-                    if isinstance(value, (bool, None.__class__)):
+                    if isinstance(value, (bool, NoneType)):
                         c_default = "Py_" + py_default
                     elif isinstance(value, str):
                         c_default = c_repr(value)

Type checkers do a lot of special casing around None due to its status as a singleton; they understand isinstance() checks against types.NoneType, but get bamboozled by more unusual idioms like isinstance(x, None.__class__) :)

erlend-aasland reacted with hooray emoji
All reactions
  • 🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood
Merge branch 'main' into clinic/annotate-state-parameter
2bd134f
@AlexWaygood AlexWaygood added the skip news label Jul 3, 2023
@erlend-aasland
erlend-aasland marked this pull request as ready for review July 3, 2023 21:42
@bedevere-bot bedevere-bot added the awaiting core review label Jul 3, 2023
@erlend-aasland

erlend-aasland commented Jul 3, 2023

Copy link
Copy Markdown
Contributor Author

Of course, the assert isinstance trick 📝

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland

erlend-aasland commented Jul 3, 2023

Copy link
Copy Markdown
Contributor Author

Thanks!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@AlexWaygood @erlend-aasland
Fix the remaining stuff
73c1dc2
@erlend-aasland erlend-aasland changed the title gh-104050: Annotate Argument Clinic DSLParser.state_parameter() gh-104050: Annotate more Argument Clinic DSLParser state methods Jul 3, 2023
@AlexWaygood

AlexWaygood commented Jul 3, 2023

Copy link
Copy Markdown
Member

No problem at all! 🚀

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

AlexWaygood
AlexWaygood approved these changes Jul 3, 2023
View reviewed changes

@AlexWaygood AlexWaygood left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Off-topic: this function is way too large and complicated, and desperately in need of modernisation as well. I had a go a while back (main...AlexWaygood:cpython:clinic-asts), but some tests were failing... that branch is also quite out of date now :-) And test coverage was poor, IIRC.

But maybe I'll get back to it at some point...

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Jul 3, 2023
@erlend-aasland
erlend-aasland enabled auto-merge (squash) July 3, 2023 21:50
@erlend-aasland
erlend-aasland merged commit b24479d into python:main Jul 3, 2023
@bedevere-bot bedevere-bot removed the awaiting merge label Jul 3, 2023
@erlend-aasland
erlend-aasland deleted the clinic/annotate-state-parameter branch July 3, 2023 22:11
carljm added a commit to carljm/cpython that referenced this pull request Jul 4, 2023
@carljm
Merge branch 'main' into cachedpropdoublecheck
7a7fd43
* main:
  pythongh-106368: Increase Argument Clinic test coverage (python#106389)
  pythongh-106320: Fix _PyImport_GetModuleAttr() declaration (python#106386)
  pythongh-106368: Harden Argument Clinic parser tests (python#106384)
  pythongh-106320: Remove private _PyImport C API functions (python#106383)
  pythongh-86085: Remove _PyCodec_Forget() declaration (python#106377)
  pythongh-106320: Remove more private _PyUnicode C API functions (python#106382)
  pythongh-104050: Annotate more Argument Clinic DSLParser state methods (python#106376)
  pythongh-106368: Clean up Argument Clinic tests (python#106373)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AlexWaygood AlexWaygood AlexWaygood approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@erlend-aasland @AlexWaygood @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.