Description

Eclips4

opened on Jun 3, 2023

Issue body actions

Tried on current main:
Fails on test_capi/test_misc/TestOptimizerAPI.test_counter_optimizer

./python -m test -v test_capi -m test_misc -m TestOptimizerAPI --forever
... many lines
======================================================================
FAIL: test_counter_optimizer (test.test_capi.test_misc.TestOptimizerAPI.test_counter_optimizer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\KIRILL-1\CLionProjects\cpython\Lib\test\test_capi\test_misc.py", line 1929, in test_counter_optimizer
    self.assertEqual(opt.get_count(), 1000)
AssertionError: 0 != 1000

----------------------------------------------------------------------
Ran 168 tests in 6.590s

FAILED (failures=1, skipped=5)
test test_capi failed
test_capi failed (1 failure)

== Tests result: FAILURE ==

1 test OK.

1 test failed:
    test_capi

Total duration: 13.2 sec
Tests result: FAILURE

Activity

1. Eclips4

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Jun 3, 2023
2. Eclips4 commented on Jun 3, 2023

   

   Eclips4

   on Jun 3, 2023

   MemberAuthor

   More actions

   Seems to be introduced in #105100
   cc @markshannon
3. AlexWaygood commented on Jun 19, 2023

   

   AlexWaygood

   on Jun 19, 2023

   Member

   More actions

   Fixed by #105683

   Reacted by Erlend E. Aasland and Kirill Podoprigora
4. AlexWaygood

   closed this as completedon Jun 19, 2023
5. sunmy2019 commented on Jun 23, 2023

   

   sunmy2019

   on Jun 23, 2023

   Member

   More actions

   I am testing heads/main: 6a80664 and the issue happens again.

   ./python -m test -R 3:3 -v test_capi -m test_long_loop -m test_counter_optimizer
   
   0:00:00 load avg: 0.62 Run tests sequentially
   0:00:00 load avg: 0.62 [1/1] test_capi
   beginning 6 repetitions
   123456
   test_counter_optimizer (test.test_capi.test_misc.TestOptimizerAPI.test_counter_optimizer) ... ok
   test_long_loop (test.test_capi.test_misc.TestOptimizerAPI.test_long_loop)
   Check that we aren't confused by EXTENDED_ARG ... ok
   
   ----------------------------------------------------------------------
   Ran 2 tests in 0.001s
   
   OK
   .test_counter_optimizer (test.test_capi.test_misc.TestOptimizerAPI.test_counter_optimizer) ... FAIL
   test_long_loop (test.test_capi.test_misc.TestOptimizerAPI.test_long_loop)
   Check that we aren't confused by EXTENDED_ARG ... FAIL
   
   ======================================================================
   FAIL: test_counter_optimizer (test.test_capi.test_misc.TestOptimizerAPI.test_counter_optimizer)
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/home/xxx/cpython/Lib/test/test_capi/test_misc.py", line 2386, in test_counter_optimizer
       self.assertEqual(opt.get_count(), 1000)
   AssertionError: 0 != 1000
   
   ======================================================================
   FAIL: test_long_loop (test.test_capi.test_misc.TestOptimizerAPI.test_long_loop)
   Check that we aren't confused by EXTENDED_ARG
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/home/xxx/cpython/Lib/test/test_capi/test_misc.py", line 2408, in test_long_loop
       self.assertEqual(opt.get_count(), 10)
   AssertionError: 0 != 10
   
   ----------------------------------------------------------------------
   Ran 2 tests in 0.002s
   
   FAILED (failures=2)
   test test_capi failed
   test_capi failed (2 failures)
   
   == Tests result: FAILURE ==
   
   1 test failed:
       test_capi
   
   Total duration: 76 ms
   Tests result: FAILURE
   
6. sunmy2019

   reopened this on Jun 23, 2023
7. markshannon commented on Jun 23, 2023

   

   markshannon

   on Jun 23, 2023

   Member

   More actions

   Fixed by #105683

   Unfortunately not.

   Reacted by Alex Waygood
8. gvanrossum commented on Jun 27, 2023

   

   gvanrossum

   on Jun 27, 2023

   Member

   More actions

   Just got hit by this (it makes the refleak buildbots fail in #105924).

   I've got a hunch the problem is in deopt_code() (in codeobject.c), but I can't quite nail it. I put a printf() there when opcode == ENTER_EXECUTOR, and in the loop in the test which executes five times, it prints only four times?! Then the second time the whole test is re-run, it prints never at all (but the test fails after the first time through the loop). It appears the first iteration (out of five in the test -- for repeat in range(5)) doesn't cause any executor to be cleared when clear_executors() sets func.__code__ = func.__code__.replace(). But the first iteration clearly has an executor, because the assert passes (the first time the whole test is run).

   Not sure where this really points, but the logic in deopt_code() is subtle (it gets the opcode from _Py_GetBaseOpcode(code, i), then patches instructions[i]). I get lost when I try to understand _Py_GetBaseOpcode().
9. gvanrossum commented on Jun 28, 2023

   

   gvanrossum

   on Jun 28, 2023

   Member

   More actions

   This was now fixed by @vstinner in #106171. His fix is to create a whole new function object (by using exec). This is needed because the original code object is kept alive by the parent code object -- and the only way to clear a code object's co_executors array is to cause the code object to be finalized.

   (I tried putting the assignment to func.__code__ in front of the yield, but it didn't seem to work.)

   Arguably we ought to have an explicit API for this, but that can wait.

   Reacted by Kirill Podoprigora
10. gvanrossum

    closed this as completedon Jun 28, 2023

Issue actions