GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — gh-100762: Fix optimization in gen_close by iritkatriel · Pull Request #111069 · python/cpython

Conversation

iritkatriel commented Oct 19, 2023 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Member

In f02fa64 lines 383-389 were added, with returns that do not decref yf. I think this is ok because yf is always NULL in these cases. Adding the assertions.

Also, the optimization for the case of exception_handler_depth== 1 is not working because op.code is checked instead of op.arg.

Issue: Generator finalization is slower in 3.11 vs 3.10 #100762

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions


          pythongh-100762: fix refleak in gen_close

084c91a

iritkatriel requested a review from markshannon as a code owner

October 19, 2023 13:36

bedevere-app Bot mentioned this pull request

Oct 19, 2023

Generator finalization is slower in 3.11 vs 3.10 #100762

Closed

bedevere-app Bot added the awaiting core review label

Oct 19, 2023

iritkatriel added type-bug

An unexpected behavior, bug, or error

interpreter-core

(Objects, Python, Grammar, and Parser dirs)

3.12

only security fixes

needs backport to 3.12 only security fixes and removed 3.12

only security fixes

type-bug

An unexpected behavior, bug, or error

needs backport to 3.12 only security fixes labels

Oct 19, 2023

iritkatriel changed the title ~~gh-100762: fix refleak in gen_close~~ gh-100762: assert that yf is NULL in gen_close in the cases where it is not DECREFed

Oct 19, 2023

iritkatriel changed the title ~~gh-100762: assert that yf is NULL in gen_close in the cases where it is not DECREFed~~ gh-100762: Fix optimization in gen_close and assert that yf is NULL in the cases where it is not DECREFed

Oct 19, 2023

iritkatriel added 2 commits

October 19, 2023 16:44


          assert instead

eafd6b1


          op.code --> op.arg

d3e4f64

iritkatriel commented

Oct 19, 2023

View reviewed changes

Comment thread

Objects/genobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.


          whitespace

9a83cf8

iritkatriel added the skip news label

Oct 19, 2023

iritkatriel mentioned this pull request

Oct 19, 2023

gh-109094: replace frame->prev_instr by frame->instr_ptr #109095

Merged

iritkatriel closed this

Oct 19, 2023

iritkatriel reopened this

Oct 20, 2023

iritkatriel mentioned this pull request

Oct 20, 2023

broken optimization in gen_close() #111107

Closed

iritkatriel marked this pull request as draft

October 20, 2023 12:48

bedevere-app Bot removed the awaiting core review label

Oct 20, 2023


          set frame state when returning early

90d6409

markshannon commented Oct 24, 2023

Copy link

Copy Markdown

Member

This looks good.
The failure is in a new test, that didn't exist when the unintentional change from oparg to opcode was made. https://github.com/python/cpython/pull/101912/files#diff-2a0c449b68605ebd0872fd232e60ce7e838a77782a6d2e364764f99514fb508aR376

Just change the test.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

markshannon commented Oct 24, 2023

Copy link

Copy Markdown

Member

One minor efficiency improvement that could be made is to not call _PyGen_yf(gen) until after the checks for FRAME_CREATED and FRAME_COMPLETED.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.


          apply review comments

86b325d

iritkatriel marked this pull request as ready for review

October 24, 2023 11:11

bedevere-app Bot added the awaiting core review label

Oct 24, 2023

iritkatriel added needs backport to 3.11

only security fixes

needs backport to 3.12 only security fixes labels

Oct 24, 2023

iritkatriel changed the title ~~gh-100762: Fix optimization in gen_close and assert that yf is NULL in the cases where it is not DECREFed~~ gh-100762: Fix optimization in gen_close

Oct 24, 2023

iritkatriel added needs backport to 3.12 only security fixes and removed needs backport to 3.11

only security fixes

needs backport to 3.12 only security fixes labels

Oct 24, 2023

markshannon approved these changes

Oct 25, 2023

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting core review labels

Oct 25, 2023

iritkatriel merged commit 0db2517 into python:main

Oct 25, 2023

bedevere-app Bot removed the awaiting merge label

Oct 25, 2023

markshannon mentioned this pull request

Jan 5, 2024

Differing behavior for sys.setrace between 3.12 and 3.13 #113728

Closed

aisk pushed a commit to aisk/cpython that referenced this pull request

Feb 11, 2024


          pythongh-100762: Fix optimization in gen_close (python#111069)

3baff26

cdce8p commented Feb 22, 2024

Copy link

Copy Markdown

Contributor

Would it be possible / make sense to cherry-pick this change onto 3.12?

For pylint / astroid we're seeing ValueError: generator already executing errors since 3.12.0b1. We haven't been able to create a standalone reproducer - which is also the reason I haven't opened a bug report yet. However testing either 3.13.0a2 or 3.12 with this change cherry-picked on top does seem to resolve it as best as I can tell.

pylint-dev/pylint#9138

matejsp reacted with thumbs up emoji

All reactions

👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

cdce8p mentioned this pull request

Feb 22, 2024

Python 3.12 - ValueError: generator already executing pylint-dev/pylint#9138

Closed

iritkatriel commented Feb 22, 2024

Copy link

Copy Markdown

Member Author

Would it be possible / make sense to cherry-pick this change onto 3.12?

I think so. I'll make a PR.

cdce8p reacted with thumbs up emoji

All reactions

👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

iritkatriel added the needs backport to 3.12 only security fixes label

Feb 22, 2024

miss-islington-app Bot commented Feb 22, 2024

Copy link

Copy Markdown

Thanks @iritkatriel for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request

Feb 22, 2024


          pythongh-100762: Fix optimization in gen_close (pythonGH-111069)

fdbb567

(cherry picked from commit 0db2517)

Co-authored-by: Irit Katriel <[email protected]>

bedevere-app Bot commented Feb 22, 2024

Copy link

Copy Markdown

GH-115818 is a backport of this pull request to the 3.12 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-app Bot removed the needs backport to 3.12 only security fixes label

Feb 22, 2024

iritkatriel added a commit that referenced this pull request

Feb 22, 2024


          [3.12] gh-100762: Fix optimization in gen_close (GH-111069) (#115818)

eb4774d

gh-100762: Fix optimization in gen_close  (GH-111069)
(cherry picked from commit 0db2517)

Co-authored-by: Irit Katriel <[email protected]>

iritkatriel added a commit to iritkatriel/cpython that referenced this pull request

Jun 13, 2024


          Revert "[3.12] pythongh-100762: Fix optimization in gen_close (python…

47c7a30

…GH-111069) (python#115818)"

This reverts commit eb4774d.

iritkatriel mentioned this pull request

Jun 13, 2024

[3.12] gh-119897: Revert buggy optimization which was removed in 3.13 #120467

Merged

Glyphack pushed a commit to Glyphack/cpython that referenced this pull request

Sep 2, 2024


          pythongh-100762: Fix optimization in gen_close (python#111069)

bba1850

gaogaotiantian mentioned this pull request

Jun 12, 2025

3.14.0b2 yield gives TypeError: _pystart_callback expected 2 arguments, got 3 #135429

Closed

markshannon mentioned this pull request

Oct 21, 2025

sys.monitoring PY_UNWIND not being delivered when a generator is closed/garbage collected #140373

Closed

Reviewers

markshannon

markshannon approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

interpreter-core

(Objects, Python, Grammar, and Parser dirs)

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

iritkatriel commented Oct 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

markshannon commented Oct 24, 2023

Uh oh!

markshannon commented Oct 24, 2023

Uh oh!

cdce8p commented Feb 22, 2024

Uh oh!

iritkatriel commented Feb 22, 2024

Uh oh!

miss-islington-app Bot commented Feb 22, 2024

Uh oh!

bedevere-app Bot commented Feb 22, 2024

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

iritkatriel commented Oct 19, 2023 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.