GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-64490: Fix bugs in argument clinic varargs processing by colorfulappl · Pull Request #32092 · python/cpython

Conversation

@colorfulappl

@colorfulappl colorfulappl commented Mar 24, 2022 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Fix 3 bugs introduced in #18609.

Bug reason lists in each commit's commit message.

https://bugs.python.org/issue20291

  • Issue: Argument Clinic should understand *args and **kwargs parameters #64490

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
colorfulappl added 3 commits March 24, 2022 15:21
@colorfulappl
Fix a bug which allows more than one varargs
6793f38
Python allows at most *one* vararg in one function.
Remove the improper varargs check which allows function definition like
```
    *vararg1: object
    *vararg2: object
```
in argument clinic.
@colorfulappl
Fix an array index out-of-bound bug
6203962
Variable `vararg` indicates the index of vararg in parameter list.
While copying kwargs to `buf`, the index `i` should not add `vararg`, which leads to an out-of-bound bug.

When there are positional args, vararg and keyword args in a function definition, in which case `vararg` > 1, this bug can be triggered.

e.g.
```
    pos: object
    *args: object
    kw: object
```
@colorfulappl
Fix a kwarg parse bug
03be6f9
The calculation of noptargs is incorrect when there is a vararg.
This bug prevents parsed arguments passing to XXXX_impl function.

e.g.
Define function
```
    *args: object
    kw: object
```
and pass kw=1 to the function, the `kw` parameter won't receive the pass in value.
@bedevere-bot bedevere-bot added the awaiting review label Mar 24, 2022
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Mar 24, 2022
@colorfulappl colorfulappl mentioned this pull request Mar 24, 2022
gh-90370: Avoid temporary varargs tuple creation in argument passing #30312
Closed
@isidentical
isidentical self-requested a review March 24, 2022 13:12
@serhiy-storchaka serhiy-storchaka added the topic-argument-clinic label Apr 30, 2022
@ezio-melotti ezio-melotti removed the CLA signed label Jul 13, 2022
erlend-aasland
erlend-aasland reviewed Jul 20, 2022
View reviewed changes
Comment thread Tools/clinic/clinic.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland

erlend-aasland commented Jul 20, 2022

Copy link
Copy Markdown
Contributor

Could you also add a NEWS entry?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland erlend-aasland self-assigned this Jul 20, 2022
@erlend-aasland
erlend-aasland requested a review from serhiy-storchaka July 20, 2022 20:57
colorfulappl added 3 commits August 11, 2022 15:42
@colorfulappl
Merge branch 'main' into unpack_keywords_bugfix
cefe5e9
# Conflicts:
#	Lib/test/clinic.test
@colorfulappl
Regenerate clinic.test
45bde03
@colorfulappl
Replace pre_buffer to a more descriptive name nargs
bfc5241
@bedevere-bot

bedevere-bot commented Aug 11, 2022

Copy link
Copy Markdown

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland

erlend-aasland commented Aug 11, 2022

Copy link
Copy Markdown
Contributor

Thanks; still missing the NEWS entry, though.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@blurb-it
📜🤖 Added by blurb_it.
0b1523c
@colorfulappl

colorfulappl commented Aug 11, 2022

Copy link
Copy Markdown
Contributor Author

NEWS added :)

erlend-aasland reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@colorfulappl
Replace inline literals with double backticks in news
411ad37
@erlend-aasland erlend-aasland changed the title bpo-20291: Fix bugs in argument clinic varargs processing gh-64490: Fix bugs in argument clinic varargs processing Aug 11, 2022
@colorfulappl
Rename news
c6a5171
@erlend-aasland

erlend-aasland commented Aug 11, 2022

Copy link
Copy Markdown
Contributor

NEWS added :)

Thanks; that's some entry! :) Could you narrow it down to a couple of concise sentences that describe the actual changes only?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland erlend-aasland mentioned this pull request Aug 11, 2022
gh-90928: Statically Initialize the Keywords Tuple in Clinic-Generated Code #95860
Merged
@erlend-aasland

erlend-aasland commented Aug 11, 2022

Copy link
Copy Markdown
Contributor

Also, could you please add tests for each case?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

colorfulappl added 2 commits August 12, 2022 17:03
@colorfulappl
Merge branch 'main' into unpack_keywords_bugfix
42f9273
# Conflicts:
#	Lib/test/clinic.test
@colorfulappl
Simplify news
7fdd501
@colorfulappl

colorfulappl commented Aug 12, 2022

Copy link
Copy Markdown
Contributor Author

Also, could you please add tests for each case?

Do you mean put the test cases into test_linic.py?

To see if the OOB bug is fixed in commit 6203962 , I should write some code snippet calling function _PyArg_UnpackKeywordsWithVararg, compile it and ensure that it does not crash.

I'm not sure how can I do this without touching the Python vm or library,
though I have wrote a proof-of-concept by adding a built-in function and test it manually:
https://github.com/colorfulappl/cpython/tree/unpack_keywords_oob_bug_poc .

As for commit 6793f38, I'm looking at test_linic.py and will have a try later.

erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@colorfulappl
Regenerate clinic.test
0aa4be4
@erlend-aasland

erlend-aasland commented Nov 22, 2022

Copy link
Copy Markdown
Contributor

Is it still waiting for further changes?

Since gh-96178 has recently been merged, we finally have a test framework for functional clinic tests, so I'd say this PR should add some tests before we are ready to land.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

colorfulappl added 3 commits November 23, 2022 12:00
@colorfulappl
Merge branch 'main' into unpack_keywords_bugfix
0afce9d
@colorfulappl
Add Argument Clinic functional test cases
87314ff
@colorfulappl
Fix bug
a48971d
@colorfulappl

colorfulappl commented Nov 24, 2022

Copy link
Copy Markdown
Contributor Author

I added some test cases and did another change in a48971d, so this PR may be necessary to be reviewed again. @erlend-aasland @isidentical @kumaraditya303

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@colorfulappl
Do not generate unused noptargs
e6a4e13
@kumaraditya303
kumaraditya303 self-requested a review November 24, 2022 06:43
@colorfulappl

colorfulappl commented Nov 24, 2022

Copy link
Copy Markdown
Contributor Author

Another change in e6a4e13:
Argument Clinic does not generate noptargs when there is a vararg and no optional argument.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

colorfulappl added 3 commits November 24, 2022 14:50
@colorfulappl
Update news
8592676
@colorfulappl
Update Lib/test/clinic.test
4c682bf
@colorfulappl
Delete unnecessary function doc
c3d6b73
@colorfulappl
colorfulappl force-pushed the unpack_keywords_bugfix branch from 6087c2e to c3d6b73 Compare November 24, 2022 11:29
erlend-aasland
erlend-aasland approved these changes Nov 24, 2022
View reviewed changes

@erlend-aasland erlend-aasland left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! Kumar, Batuhan, do you have further remarks?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
kumaraditya303
kumaraditya303 approved these changes Nov 24, 2022
View reviewed changes

@kumaraditya303 kumaraditya303 left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@erlend-aasland

erlend-aasland commented Nov 24, 2022

Copy link
Copy Markdown
Contributor

@colorfulappl, please fix the merge conflicts

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

colorfulappl added 2 commits November 24, 2022 22:10
@colorfulappl
Merge branch 'main' into unpack_keywords_bugfix
8b26eb8
# Conflicts:
#	Lib/test/test_clinic.py
#	Modules/_testclinic.c
#	Modules/clinic/_testclinic.c.h
@colorfulappl
Rerun make clinic
3f88c54
@erlend-aasland
erlend-aasland merged commit 0da7283 into python:main Nov 24, 2022
@miss-islington

miss-islington commented Nov 24, 2022

Copy link
Copy Markdown
Contributor

Thanks @colorfulappl for the PR, and @erlend-aasland for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10, 3.11.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting merge label Nov 24, 2022
@miss-islington

miss-islington commented Nov 24, 2022

Copy link
Copy Markdown
Contributor

Sorry, @colorfulappl and @erlend-aasland, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 0da728387c99fe6c127b070f2d250dc5bdd62ee5 3.11

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington

miss-islington commented Nov 24, 2022

Copy link
Copy Markdown
Contributor

Sorry @colorfulappl and @erlend-aasland, I had trouble checking out the 3.10 backport branch.
Please retry by removing and re-adding the "needs backport to 3.10" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker 0da728387c99fe6c127b070f2d250dc5bdd62ee5 3.10

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@erlend-aasland

erlend-aasland commented Nov 24, 2022

Copy link
Copy Markdown
Contributor

@kumaraditya303 I'm inclined to backport the functional tests for AC to 3.11 and 3.10. What do you think? We've backported tests to increase coverage in the maintained branches before; it would help backporting the AC fixes (and by the way, we should backport the AC bug fixes that we landed earlier today)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

colorfulappl added a commit to colorfulappl/cpython that referenced this pull request Dec 20, 2022
@colorfulappl
[3.11] pythongh-64490: Fix bugs in argument clinic varargs processing (…
c420e8f
…pythonGH-32092)

(cherry picked from commit 0da7283)
@bedevere-bot

bedevere-bot commented Dec 20, 2022

Copy link
Copy Markdown

GH-100368 is a backport of this pull request to the 3.11 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Dec 20, 2022
@colorfulappl

colorfulappl commented Dec 20, 2022

Copy link
Copy Markdown
Contributor Author

I have backported this to 3.11 (#100368).
And since 3.10 Argument Clinic does not support varargs parsing, there is no need to backport this to 3.10, IMHO.

erlend-aasland reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

erlend-aasland pushed a commit that referenced this pull request Dec 28, 2022
@colorfulappl
[3.11] gh-64490: Fix bugs in argument clinic varargs processing (GH-3…
a3dbd4c
…2092) (#100368)

(cherry picked from commit 0da7283)
@serhiy-storchaka serhiy-storchaka removed the needs backport to 3.10 label Aug 11, 2024
@serhiy-storchaka

serhiy-storchaka commented Aug 11, 2024

Copy link
Copy Markdown
Member

It is too later for 3.10.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@isidentical isidentical isidentical approved these changes
@erlend-aasland erlend-aasland erlend-aasland approved these changes
@kumaraditya303 kumaraditya303 kumaraditya303 approved these changes
@serhiy-storchaka serhiy-storchaka Awaiting requested review from serhiy-storchaka

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@erlend-aasland erlend-aasland

Labels

topic-argument-clinic type-bug An unexpected behavior, bug, or error

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

10 participants

@colorfulappl @erlend-aasland @bedevere-bot @kumaraditya303 @smontanaro @miss-islington @serhiy-storchaka @isidentical @the-knights-who-say-ni @ezio-melotti
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.