Description

markshannon

opened on Jun 30, 2022

Issue body actions

The frame_setlineno function works in in stages:

• Determine a set of possible bytecode offsets as targets from the line number.
• Compute the stack state for these targets and the current position
• Determine a best target. That is, the first one that has a compatible stack.
• Pop values form the stack and jump.

The first steps is faulty (I think, I haven't demonstrated this) as it might be possible to jump to an instruction involved in frame creation. This should be easy to fix using the new _co_firsttraceable field.

The second step has (at least) three flaws:

• It does not account for NULLs on the stack, making it possible to jump from a stack with NULLs to one that cannot handle NULLs.
• It does not skip over caches, so could produce incorrect stacks by misinterpreting cache entries as normal instructions.
• It is out of date. For example it thinks that PUSH_EXC_INFO pushes three values. It only pushes one.

Setting the line number of a frame is only possible in the debugger, so this isn't as terrible as might appear, but it definitely needs fixing.

Linked PRs

• gh-94438: Add additional cases to mark_stacks with tests #111237
• [3.12] GH-94438: Restore ability to jump over None tests (GH-111237) #111243
• [3.11] GH-94438: Restore ability to jump over None tests (GH-111237) #111338
• GH-94438: Fix RuntimeWarning for jump tests in test_sys_settrace #111341
• [3.12] GH-94438: Fix RuntimeWarning for jump tests in test_sys_settrace (GH-111341) #111369

Activity

1. markshannon

   added

   type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

   deferred-blocker

   on Jun 30, 2022
2. markshannon

   assigned

   iritkatriel

   and

   markshannon

   on Jun 30, 2022
3. ezio-melotti

   added this to Release and Deferred blockers 🚫on Jun 30, 2022
4. markshannon

   added

   3.11only security fixesonly security fixes

   3.12only security fixesonly security fixes

   on Jun 30, 2022
5. miss-islington

   added a commit that references this issue on Jul 1, 2022

   pythonGH-94438: Account for NULLs on evaluation stack when jumping li…

   edfdd6e
6. markshannon

   added 2 commits that reference this issue on Jul 1, 2022

   GH-94438: Account for NULLs on evaluation stack when jumping lines. (G…

   be80db1

   [3.11] GH-94438: Backport GH-94444 (#94486)

   02b30a8
7. pablogsal

   added

   release-blocker

   and removed

   deferred-blocker

   on Jul 4, 2022
8. pablogsal commented on Jul 4, 2022

   

   pablogsal

   on Jul 4, 2022

   Member

   More actions

   Moving this back to release blocker because apparently, this could end in many changes.

   I am missing some context here on what this is affecting so I changed it from deferred blocker to release blocker if we think we can delay this to 3.12, please, say so :)
9. iritkatriel

   added a commit that references this issue on Jul 5, 2022

   pythongh-94438: in frameobject's mark_stacks switch, the PUSH_EXC_INF…

   9efd9e0
10. iritkatriel commented on Jul 5, 2022

    

    iritkatriel

    on Jul 5, 2022

    Member

    More actions

    It is out of date. For example it thinks that PUSH_EXC_INFO pushes three values. It only pushes one.

    After failing to write a test that will crash on this, I analysed the code and realised that the "exception handling opcode" cases of the switch are no longer reachable - there is nothing that will initialise their stack[] entries, so they get skipped in the continue; before the switch.

    I created a PR to use my favourite macro in these cases: #94582

    We could backport it, but we don't have to.
11. 31 remaining items

12. gvanrossum commented on Feb 27, 2023

    

    gvanrossum

    on Feb 27, 2023

    Member

    More actions

    Shall we nevertheless close this, and open a more specific issue?
13. markshannon commented on Feb 27, 2023

    

    markshannon

    on Feb 27, 2023

    MemberAuthor

    More actions

    Yes.
    Most, if not all, of the flaws I listed have been fixed.
14. gvanrossum

    closed this as completedon Feb 27, 2023
15. github-project-automation

    moved this from In Progress to Done in Release and Deferred blockers 🚫on Feb 27, 2023
16. brandtbucher

    added a commit that references this issue on Oct 24, 2023

    GH-94438: Restore ability to jump over None tests (GH-111237)

    6640f1d
17. miss-islington

    added a commit that references this issue on Oct 24, 2023

    pythonGH-94438: Restore ability to jump over None tests (pythonGH-111237

    2c77dfb
18. brandtbucher

    added 3 commits that reference this issue on Oct 24, 2023

    [3.12] GH-94438: Restore ability to jump over None tests (GH-111243)

    0bbdfe6

    [3.11] GH-94438: Restore ability to jump over None tests (GH-111338)

    12c7e50

    GH-94438: Fix RuntimeWarning for jump tests in test_sys_settrace (GH-…

    a254120
19. miss-islington

    added a commit that references this issue on Oct 26, 2023

    pythonGH-94438: Fix RuntimeWarning for jump tests in test_sys_settrace (

    097beb2
20. brandtbucher

    added a commit that references this issue on Oct 26, 2023

    [3.12] GH-94438: Fix RuntimeWarning for jump tests in test_sys_settra…

    a70b2cd
21. aisk

    added 2 commits that reference this issue on Feb 11, 2024

    pythonGH-94438: Restore ability to jump over None tests (pythonGH-111237

    d9715b6

    pythonGH-94438: Fix RuntimeWarning for jump tests in test_sys_settrace (

    143e950
22. Glyphack

    added 2 commits that reference this issue on Sep 2, 2024

    pythonGH-94438: Restore ability to jump over None tests (pythonGH-111237

    a721258

    pythonGH-94438: Fix RuntimeWarning for jump tests in test_sys_settrace (

    f037497

Issue actions