GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-131798: JIT: Propagate the result in `_BINARY_OP_SUBSCR_TUPLE_INT` by tomasr8 · Pull Request #133003 · python/cpython

Conversation

@tomasr8

@tomasr8 tomasr8 commented Apr 26, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

More context: #132851 (comment)

This propagates the information about tuple elements after _BINARY_OP_SUBSCR_TUPLE_INT when the RHS is a constant.
For example in

foo = (1, 2)
x = foo[0]  # _BINARY_OP_SUBSCR_TUPLE_INT

we can now deduce that x is 1.

  • Issue: Better uop coverage in the JIT optimizer #131798

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
tomasr8 added 2 commits April 26, 2025 14:01
@tomasr8
Make _BINARY_OP_SUBSCR_TUPLE_INT smarter
c3cd21c
@tomasr8
Add news entry
c7b7b48
@tomasr8
tomasr8 requested a review from Fidget-Spinner as a code owner April 26, 2025 12:12
@bedevere-app bedevere-app Bot added the awaiting review label Apr 26, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Apr 26, 2025
Better uop coverage in the JIT optimizer #131798
Open
tomasr8
tomasr8 commented Apr 26, 2025
View reviewed changes
Comment thread Python/optimizer_bytecodes.c Outdated
Comment on lines +378 to +379
assert(index >= 0);
assert(index < sym_tuple_length(left));

@tomasr8 tomasr8 Apr 26, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure these asserts are actually needed. The instruction will deopt when this is not true so perhaps we can remove them?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@tomasr8 tomasr8 Apr 26, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, we are actually hitting the assertion:

Python/optimizer_cases.c.h:630: optimize_uops: Assertion `index < sym_tuple_length(left)' failed.

I guess I need to move the DEOPT_IF checks into a separate guard for it to work?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@Fidget-Spinner Fidget-Spinner Apr 26, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check out the code for sym_tuple_length. It can return -1 if the length is not known. So for example if you propagate a PyTuple_Type, but not the length, it will fail.

You need to check that the length is not -1.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

tomasr8 reacted with heart emoji
All reactions
  • ❤️ 1 reaction

@tomasr8 tomasr8 Apr 26, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, thanks! I somehow didn't realize you can know something is a tuple but not know its length..

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Fidget-Spinner
Fidget-Spinner reviewed Apr 26, 2025
View reviewed changes
Comment thread Python/optimizer_bytecodes.c Outdated
long index = PyLong_AsLong(sym_get_const(ctx, right));
assert(index >= 0);
assert(index < sym_tuple_length(left));
res = sym_tuple_getitem(ctx, left, index);

@Fidget-Spinner Fidget-Spinner Apr 26, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit unfortunate. I would wish we could automatically evaluate this, but it seems tuples are a special category so we can't.

#132733

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@tomasr8 tomasr8 Apr 26, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't fully digested your PR yet, but maybe there is some way to extend it to support tuples as well?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
tomasr8 added 2 commits April 26, 2025 20:11
@tomasr8
Fix a case when the tuple length is not known
dddd56f
@tomasr8
Keep the same names for stack variables
00b00cb
Fidget-Spinner
Fidget-Spinner approved these changes Apr 26, 2025
View reviewed changes
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Apr 26, 2025
@Fidget-Spinner
Fidget-Spinner merged commit 5e96e4f into python:main Apr 26, 2025
@bedevere-app bedevere-app Bot removed the awaiting merge label Apr 26, 2025
@Fidget-Spinner

Fidget-Spinner commented Apr 26, 2025

Copy link
Copy Markdown
Member

Thanks!

tomasr8 reacted with rocket emoji
All reactions
  • 🚀 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tomasr8
tomasr8 deleted the jit-subscr-tuple branch April 26, 2025 19:15
brandtbucher
brandtbucher reviewed May 6, 2025
View reviewed changes
Comment thread Python/optimizer_bytecodes.c
Comment on lines +375 to +391
if (sym_is_const(ctx, sub_st)) {
assert(PyLong_CheckExact(sym_get_const(ctx, sub_st)));
long index = PyLong_AsLong(sym_get_const(ctx, sub_st));
assert(index >= 0);
int tuple_length = sym_tuple_length(tuple_st);
if (tuple_length == -1) {
// Unknown length
res = sym_new_not_null(ctx);
}
else {
assert(index < tuple_length);
res = sym_tuple_getitem(ctx, tuple_st, index);
}
}
else {
res = sym_new_not_null(ctx);
}

@brandtbucher brandtbucher May 6, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tomasr8, this can be improved to handle abstract tuples that may not be constant (and it cleans things up, since it includes a length check):

Suggested change
if (sym_is_const(ctx, sub_st)) {
assert(PyLong_CheckExact(sym_get_const(ctx, sub_st)));
long index = PyLong_AsLong(sym_get_const(ctx, sub_st));
assert(index >= 0);
int tuple_length = sym_tuple_length(tuple_st);
if (tuple_length == -1) {
// Unknown length
res = sym_new_not_null(ctx);
}
else {
assert(index < tuple_length);
res = sym_tuple_getitem(ctx, tuple_st, index);
}
}
else {
res = sym_new_not_null(ctx);
}
long index = PyLong_AsLong(sym_get_const(ctx,
sub_st));
// ...PyLong_AsLong can fail, need to check for error here...
res = sym_tuple_getitem(ctx, tuple_st, index)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@tomasr8 tomasr8 May 8, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be improved to handle abstract tuples that may not be constant

@brandtbucher, I think this already handles abstract tuples? Assuming that abstract means those that have JIT_SYM_TUPLE_TAG rather than JIT_SYM_KNOWN_VALUE_TAG? For tuples that just have JIT_SYM_KNOWN_CLASS_TAG we don't know the length so there's nothing we can do I believe.

But yeah, we can simplify the code since _Py_uop_sym_tuple_getitem can handle unknown length. The only difference is that it sets the type to unknown rather than non null as we do here. Is that a problem? Shouldn't _Py_uop_sym_tuple_getitem always return at least JIT_SYM_NON_NULL_TAG?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@brandtbucher brandtbucher May 8, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your current code is gated on sym_is_const, which abstract tuples fail. The rest of your understanding is correct, though.

Is that a problem? Shouldn't _Py_uop_sym_tuple_getitem always return at least JIT_SYM_NON_NULL_TAG?

Sounds like you found your next PR. :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@brandtbucher brandtbucher May 8, 2025

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I completely misunderstood, sorry. I thought you were checking whether the tuple was const, not the index. Disregard!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@tomasr8 tomasr8 May 8, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no worries :) I think we can still simplify the code a bit though!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
tomasr8
tomasr8 commented May 10, 2025
View reviewed changes
Comment thread Python/optimizer_bytecodes.c
Comment on lines +380 to +383
if (tuple_length == -1) {
// Unknown length
res = sym_new_not_null(ctx);
}

@tomasr8 tomasr8 May 10, 2025

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brandtbucher
Following the previous discussion, I'm wondering if it is correct to return sym_new_not_null here rather than sym_new_unknown?

We have an abstract tuple and we don't know its length so it is impossible to be sure that the index is not out of bounds for the tuple. Given that, I think it is incorrect to return sym_new_not_null here as regular Python code could raise an IndexError. Should we change this to sym_new_unknown?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@eendebakpt eendebakpt mentioned this pull request May 17, 2026
Optimize through symbolic tuples in tier 2 optimizer #119477
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@brandtbucher brandtbucher brandtbucher left review comments
@Fidget-Spinner Fidget-Spinner Fidget-Spinner approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@tomasr8 @Fidget-Spinner @brandtbucher
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.