GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Incorrect optimization in JIT build · Issue #121012 · python/cpython

Incorrect optimization in JIT build #121012

New issue
Copy link
New issue
Copy link
Closed
Closed
Incorrect optimization in JIT build#121012
Copy link
Assignees
markshannon
Labels
type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Description

@Eclips4
Eclips4
opened on Jun 25, 2024
Issue body actions

Bug report

Bug description:

Reproducer:

def foo():
    a = [1, 2, 3]
    exhit = iter(a)
    for _ in exhit:
        pass
    a.append("this should'be in exhit")
    print(f"got {list(exhit)}, should be []")

foo()
foo()
foo()
foo()
foo()
foo()

Output:

got [], should be []
got [], should be []
got [], should be []
got [], should be []
got [], should be []
got ["this should'be in exhit"], should be []

Obviously, the last line is incorrect.

Output with a PYTHON_LLTRACE=2 env:

got [], should be []
got [], should be []
got [], should be []
got [], should be []
got [], should be []
Optimizing foo (/home/eclips4/programming-languages/cpython/example.py:1) at byte offset 42
   1 ADD_TO_TRACE: _START_EXECUTOR (0, target=21, operand=0x7f4646e59832)
21: JUMP_BACKWARD(5)
   2 ADD_TO_TRACE: _CHECK_VALIDITY_AND_SET_IP (0, target=21, operand=0x7f4646e59832)
   3 ADD_TO_TRACE: _TIER2_RESUME_CHECK (0, target=21, operand=0)
18: FOR_ITER_LIST(3)
   4 ADD_TO_TRACE: _CHECK_VALIDITY_AND_SET_IP (0, target=18, operand=0x7f4646e5982c)
   5 ADD_TO_TRACE: _ITER_CHECK_LIST (3, target=18, operand=0)
   6 ADD_TO_TRACE: _GUARD_NOT_EXHAUSTED_LIST (3, target=18, operand=0)
   7 ADD_TO_TRACE: _ITER_NEXT_LIST (3, target=18, operand=0)
20: STORE_FAST(2)
   8 ADD_TO_TRACE: _CHECK_VALIDITY_AND_SET_IP (0, target=20, operand=0x7f4646e59830)
   9 ADD_TO_TRACE: _STORE_FAST (2, target=20, operand=0)
21: JUMP_BACKWARD(5)
  10 ADD_TO_TRACE: _CHECK_VALIDITY_AND_SET_IP (0, target=21, operand=0x7f4646e59832)
  11 ADD_TO_TRACE: _JUMP_TO_TOP (0, target=0, operand=0)
Created a proto-trace for foo (/home/eclips4/programming-languages/cpython/example.py:1) at byte offset 36 -- length 11
Optimized trace (length 10):
   0 OPTIMIZED: _START_EXECUTOR (0, jump_target=7, operand=0x7f4646e59e80)
   1 OPTIMIZED: _TIER2_RESUME_CHECK (0, jump_target=7, operand=0)
   2 OPTIMIZED: _ITER_CHECK_LIST (3, jump_target=8, operand=0)
   3 OPTIMIZED: _GUARD_NOT_EXHAUSTED_LIST (3, jump_target=9, operand=0)
   4 OPTIMIZED: _ITER_NEXT_LIST (3, target=18, operand=0)
   5 OPTIMIZED: _STORE_FAST_2 (2, target=20, operand=0)
   6 OPTIMIZED: _JUMP_TO_TOP (0, target=0, operand=0)
   7 OPTIMIZED: _DEOPT (0, target=21, operand=0)
   8 OPTIMIZED: _EXIT_TRACE (0, exit_index=0, operand=0)
   9 OPTIMIZED: _EXIT_TRACE (0, exit_index=1, operand=0)
got ["this should'be in exhit"], should be []

It's definitely related to this part of code:

cpython/Python/optimizer.c

Lines 1027 to 1033 in e4a97a7

if (is_for_iter_test[opcode]) {
/* Target the POP_TOP immediately after the END_FOR,
* leaving only the iterator on the stack. */
int extended_arg = inst->oparg > 255;
int32_t next_inst = target + 1 + INLINE_CACHE_ENTRIES_FOR_ITER + extended_arg;
jump_target = next_inst + inst->oparg + 1;
}

I guess the culprit is there. If we remove the _GUARD_NOT_EXHAUSTED_LIST from is_for_iter_test, the problem will go away (although it can still be reproduced in other ways using other (range, tuple) iterators).

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux, macOS

Linked PRs

  • GH-121012: Set index to -1 when list iterators become exhausted in tier 2 #121483
  • [3.13] GH-121012: Set index to -1 when list iterators become exhausted in tier 2 (GH-121483) #121494
Reacted by Aleksandr, lubaskin, Kirill.Stepankov and Dmitriy

Activity

  1. Eclips4
    added
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    topic-JIT
    on Jun 25, 2024
  2. Eclips4
    assigned
    markshannon
    on Jun 25, 2024
  3. brandtbucher commented on Jul 1, 2024

    @brandtbucher
    brandtbucher
    on Jul 1, 2024
    Member
    More actions

    This is a tier two thing, not a JIT thing. Let's reserve the topic-JIT label for issues that reproduce with --enable-experimental-jit (but not --enable-experimental-jit=interpreter).

    Reacted by Kirill Podoprigora
  4. brandtbucher
    removed
    topic-JIT
    on Jul 1, 2024
  5. markshannon
    added a commit that references this issue on Jul 8, 2024

    GH-121012: Set index to -1 when list iterators become exhausted in ti…

    8ad6067
  6. miss-islington
    added a commit that references this issue on Jul 8, 2024

    pythonGH-121012: Set index to -1 when list iterators become exhausted…

    68c2489
  7. markshannon
    added a commit that references this issue on Jul 8, 2024

    [3.13] GH-121012: Set index to -1 when list iterators become exhauste…

    2f8919e
  8. estyxx
    added a commit that references this issue on Jul 17, 2024

    pythonGH-121012: Set index to -1 when list iterators become exhausted…

    4266da2
  9. Eclips4
    closed this as completedon Jul 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

  • @markshannon
    markshannon

Labels

type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    You are visitor 01402520 — GeezerVison does not store the pages you look up.
    Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.