GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-87390: Fix starred tuple equality and pickling by mrahtz · Pull Request #92249 · python/cpython

Conversation

@mrahtz

@mrahtz mrahtz commented May 3, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Hilariously, before this PR, it turned out that *tuple[int] was equal to tuple[int]. Fixing it, I realised that pickling was also broken: it didn't preserve gaobject.starred. This PR fixes both.

I've tested for refleaks with python3 -m test -v test_genericalias -R 3:3, and it came back clean, so I think we're good.

@Fidget-Spinner I guess you're the one most familiar with this code? :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@mrahtz
Fix starred tuple equality and pickling
335747d
@mrahtz
mrahtz force-pushed the unpacked-tuple-equality branch from b808ffd to 335747d Compare May 3, 2022 18:30
@bedevere-bot bedevere-bot added the awaiting review label May 3, 2022
@mrahtz
Update test comment
5d384b9
@mrahtz mrahtz mentioned this pull request May 3, 2022
gh-91162: Disallow tuple[T][*anything] #92255
Closed
@JelleZijlstra

JelleZijlstra commented May 3, 2022

Copy link
Copy Markdown
Member

I think this is worth a NEWS entry; it's been a while since we put the rest of this code into main. I'll review the PR now.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
JelleZijlstra
JelleZijlstra reviewed May 3, 2022
View reviewed changes

@JelleZijlstra JelleZijlstra left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I'm a little unhappy about making unpacked a public argument to the constructor, but I'm not sure how else we'd get pickling to work.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Objects/genericaliasobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Objects/genericaliasobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
mrahtz and others added 3 commits May 3, 2022 21:33
@mrahtz @JelleZijlstra
PyTuple_Size → PyTuple_GET_SIZE
4143451 
Co-authored-by: Jelle Zijlstra <[email protected]>
@mrahtz
Check type of unpackedness argument to GenericAlias
f33b28c
@blurb-it
📜🤖 Added by blurb_it.
57e18e8
@mrahtz

mrahtz commented May 3, 2022

Copy link
Copy Markdown
Contributor Author

I think this is worth a NEWS entry; it's been a while since we put the rest of this code into main.

I've added a NEWS entry for this specific change, but just to double check - were you thinking of this specific change, or the rest of the PEP 646-related changes we made to genericaliasobject.c recently? (I had to check, but we did have something in the NEWS for 3.11.0a7 saying that we "Allow unpacking types.GenericAlias objects".)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@JelleZijlstra

JelleZijlstra commented May 3, 2022

Copy link
Copy Markdown
Member

I think this is worth a NEWS entry; it's been a while since we put the rest of this code into main.

I've added a NEWS entry for this specific change, but just to double check - were you thinking of this specific change, or the rest of the PEP 646-related changes we made to genericaliasobject.c recently? (I had to check, but we did have something in the NEWS for 3.11.0a7 saying that we "Allow unpacking types.GenericAlias objects".)

Just for this change (the PR title would be good as a NEWS entry too).
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
JelleZijlstra
JelleZijlstra reviewed May 3, 2022
View reviewed changes
Comment thread Objects/genericaliasobject.c
Comment on lines +745 to +746
"must be a bool (since it's a flag controlling "
"whether the alias is unpacked)");

@JelleZijlstra JelleZijlstra May 3, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"must be a bool (since it's a flag controlling "
"whether the alias is unpacked)");
"must be a bool");

We don't need the explanation.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@JelleZijlstra JelleZijlstra self-assigned this May 3, 2022
Fidget-Spinner
Fidget-Spinner reviewed May 4, 2022
View reviewed changes

@Fidget-Spinner Fidget-Spinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! We're pretty close.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Objects/genericaliasobject.c
Comment on lines +610 to +616
PyObject *starred = PyBool_FromLong(alias->starred);
PyObject *value = Py_BuildValue("O(OOO)", Py_TYPE(alias),
alias->origin, alias->args, starred);
// Avoid double increment of reference count on Py_True/Py_False - once from
// PyBool_FromLong, and once from Py_BuildValue.
Py_CLEAR(starred);
return value;

@Fidget-Spinner Fidget-Spinner May 4, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about:

Suggested change
PyObject *starred = PyBool_FromLong(alias->starred);
PyObject *value = Py_BuildValue("O(OOO)", Py_TYPE(alias),
alias->origin, alias->args, starred);
// Avoid double increment of reference count on Py_True/Py_False - once from
// PyBool_FromLong, and once from Py_BuildValue.
Py_CLEAR(starred);
return value;
PyObject *starred = alias->starred ? Py_True : Py_False;
return Py_BuildValue("O(OOO)", Py_TYPE(alias),
alias->origin, alias->args, starred);

BTW, if you don't want to use the code above, you can avoid the double incref by using the format string O(OON). N doesn't incref. https://docs.python.org/3/c-api/arg.html#c.Py_BuildValue

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Objects/genericaliasobject.c
return NULL;
}
if (!_PyArg_CheckPositional("GenericAlias", PyTuple_GET_SIZE(args), 2, 2)) {
if (!_PyArg_CheckPositional("GenericAlias", PyTuple_GET_SIZE(args), 2, 3)) {

@Fidget-Spinner Fidget-Spinner May 4, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the creation signature in the docs https://docs.python.org/3/library/types.html#types.GenericAlias.

Also, this doesn't break backwards compatibility right? IIUC, the two-argument form will still work.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Objects/genericaliasobject.c
Comment on lines 734 to +736
PyObject *origin = PyTuple_GET_ITEM(args, 0);
PyObject *arguments = PyTuple_GET_ITEM(args, 1);

@Fidget-Spinner Fidget-Spinner May 4, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code has gotten complex enough that I recommend using https://docs.python.org/3/c-api/arg.html#c.PyArg_ParseTuple or https://docs.python.org/3/c-api/arg.html#c.PyArg_UnpackTuple instead to parse our arguments for us.

The format value list is on that page. But off the top of my head, it should be OO|O:GenericAlias. Then you keep your bool checking/error raising code below.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@kumaraditya303 kumaraditya303 May 4, 2022

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Argument clinic can also be used here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
JelleZijlstra reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread Objects/genericaliasobject.c
"whether the alias is unpacked)");
return NULL;
}
starred = PyLong_AsLong(py_starred);

@Fidget-Spinner Fidget-Spinner May 4, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised you don't need to cast to (bool) here (or maybe it's implicit, I don't remember how C treats this).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@Fidget-Spinner

Fidget-Spinner commented May 4, 2022

Copy link
Copy Markdown
Member

Azure pipelines patcheck fail is unrelated, I will fix it on main.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@Fidget-Spinner

Fidget-Spinner commented May 4, 2022

Copy link
Copy Markdown
Member

Azure pipelines patcheck fail is unrelated, I will fix it on main.

Nevermind, it seems that 65f88a6 fixed the whitespace issues, so this just needs updating from main.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@AlexWaygood AlexWaygood added the topic-typing label May 4, 2022
serhiy-storchaka
serhiy-storchaka reviewed May 5, 2022
View reviewed changes

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not like an idea about extending GenericAlias() with the third parameter. It is a public API.

I'll provide an alternate PR.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@serhiy-storchaka serhiy-storchaka mentioned this pull request May 5, 2022
gh-87390: Fix starred tuple equality and pickling (alt) #92337
Merged
@serhiy-storchaka

serhiy-storchaka commented May 5, 2022

Copy link
Copy Markdown
Member

See #92337.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@carljm

carljm commented May 7, 2022

Copy link
Copy Markdown
Member

Looks like this can be closed since the alternative was already merged.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@carljm carljm closed this May 7, 2022
@mrahtz
mrahtz deleted the unpacked-tuple-equality branch May 7, 2022 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JelleZijlstra JelleZijlstra JelleZijlstra left review comments
@serhiy-storchaka serhiy-storchaka serhiy-storchaka left review comments
@Fidget-Spinner Fidget-Spinner Fidget-Spinner left review comments
@kumaraditya303 kumaraditya303 kumaraditya303 left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@JelleZijlstra JelleZijlstra

Labels

awaiting review topic-typing

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

8 participants

@mrahtz @JelleZijlstra @Fidget-Spinner @serhiy-storchaka @carljm @kumaraditya303 @bedevere-bot @AlexWaygood
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.