Description

nedbat

opened on Jun 3, 2021

Issue body actions

BPO	44298
Nosy	@nedbat, @markshannon, @pablogsal
PRs	bpo-44298: Fix line numbers for early exits in with statements. #26513 [3.10] bpo-44298: Backport #26513 to 3.10 #26516

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/markshannon'
closed_at = <Date 2021-06-03.19:59:00.106>
created_at = <Date 2021-06-03.10:05:02.392>
labels = ['interpreter-core', 'type-bug', '3.10', 'release-blocker']
title = '3.10.0b2 traces with-exit before the break that caused the exit'
updated_at = <Date 2021-06-03.21:41:17.713>
user = 'https://github.com/nedbat'

bugs.python.org fields:

activity = <Date 2021-06-03.21:41:17.713>
actor = 'nedbat'
assignee = 'Mark.Shannon'
closed = True
closed_date = <Date 2021-06-03.19:59:00.106>
closer = 'pablogsal'
components = ['Interpreter Core']
creation = <Date 2021-06-03.10:05:02.392>
creator = 'nedbat'
dependencies = []
files = []
hgrepos = []
issue_num = 44298
keywords = ['patch', '3.10regression']
message_count = 8.0
messages = ['394990', '394991', '394994', '394995', '395007', '395033', '395038', '395052']
nosy_count = 3.0
nosy_names = ['nedbat', 'Mark.Shannon', 'pablogsal']
pr_nums = ['26513', '26516']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44298'
versions = ['Python 3.10']

Activity

1. nedbat commented on Jun 3, 2021

   

   nedbat

   on Jun 3, 2021

   MemberAuthor

   More actions

   Python 3.10 now traces back to with statements when exiting the with block. When the exit is a break statement, the with exit is visited before the break statement is. This seems confusing.

   --- 8< -----------------------------------------

   import linecache, sys
   
   def trace(frame, event, arg):
       # The weird globals here is to avoid a NameError on shutdown...
       if frame.f_code.co_filename == globals().get("__file__"):
           lineno = frame.f_lineno
           print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, lineno).rstrip()))
       return trace
   
   def doit():
       for i in range(2):
           with open("test", "w") as f:
               a = 13
               b = 14
               break
       c = 16
   
   print(sys.version)
   sys.settrace(trace)
   doit()

   --- 8< -----------------------------------------

   3.10.0b2 (default, Jun 3 2021, 05:27:13) [Clang 12.0.0 (clang-1200.0.32.29)]
   call 10: def doit():
   line 11: for i in range(2):
   line 12: with open("test", "w") as f:
   line 13: a = 13
   line 14: b = 14
   line 12: with open("test", "w") as f:
   line 15: break
   line 16: c = 16
   retu 16: c = 16

   Shouldn't we get a trace for line 15 (break), then line 12 (with-exit), then line 15 again, then line 16?
2. nedbat

   added

   3.10 (EOL)end of lifeend of life

   on Jun 3, 2021
3. nedbat

   assigned

   markshannon

   on Jun 3, 2021
4. nedbat

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Jun 3, 2021
5. nedbat commented on Jun 3, 2021

   

   nedbat

   on Jun 3, 2021

   MemberAuthor

   More actions

   (I'm not sure whether to create other issues for further details)

   I'm also seeing a return in a with will trace withexit/return for a plain "return" statement, but return/withexit/return for a return with a value ("return 17").

   I would expect that a statement causing an exit from a with block would always be traced before the with-exit.
6. markshannon commented on Jun 3, 2021

   

   markshannon

   on Jun 3, 2021

   Member

   More actions

   For context, this behavior was introduced in https://bugs.python.org/issue43933
7. markshannon commented on Jun 3, 2021

   

   markshannon

   on Jun 3, 2021

   Member

   More actions

   Why this occurs:

   with cm: 
       A
       break

   translates to something like:

   ex = cm.__exit__; cm.__enter__()  # with cm
       A
       ex(...)
       goto loop_end   # break

   So, the break is traced after the exit call.

   However, this doesn't seem consistent with try-finally statements which trace any break/continue/return before the finally block.
8. markshannon

   added

   release-blocker

   on Jun 3, 2021
9. markshannon commented on Jun 3, 2021

   

   markshannon

   on Jun 3, 2021

   Member

   More actions

   New changeset 937cebc by Mark Shannon in branch 'main':
   bpo-44298: Fix line numbers for early exits in with statements. (GH-26513)
   937cebc
10. pablogsal commented on Jun 3, 2021

    

    pablogsal

    on Jun 3, 2021

    Member

    More actions

    New changeset cea0585 by Mark Shannon in branch '3.10':
    [3.10] bpo-44298: Backport bpo-26513 to 3.10 (bpo-26516)
    cea0585
11. pablogsal commented on Jun 3, 2021

    

    pablogsal

    on Jun 3, 2021

    Member

    More actions

    Ned, can you confirm that this works for you? I am closing the issue, but if something is missing, please, reopen it and we will look into it :)
12. pablogsal

    closed this as completedon Jun 3, 2021
13. nedbat commented on Jun 3, 2021

    

    nedbat

    on Jun 3, 2021

    MemberAuthor

    More actions

    Thanks for the quick turnaround, this works!
14. ezio-melotti

    transferred this issue fromon Apr 10, 2022
15. gpshead commented on Jun 20, 2022

    

    gpshead

    on Jun 20, 2022

    Member

    More actions

    this appears to have caused a regression in line numbers for return statements. #93975

Issue actions