Description

brandtbucher

opened on Nov 8, 2024

Issue body actions

We should rip the _PyOptimizer API out, and just keep all of the code that does that actual optimizing.

The API itself is unstable, constantly changing, poorly-defined, and undocumented. There's a bunch of infrastructure required just to test the API (not the actual optimizations we perform), and it introduces indirection and artificial boundaries into some pretty performance-sensitive stuff. Nobody is using it that we're aware of, nobody we've talked to is planning on using it, and frankly we don't want anyone to start using it. So let's remove it.

Linked PRs

• gh-126599: Refactor the _PyOptimizer APIs #126853
• GH-126599: Remove the PyOptimizer API #129194

Reacted by Kirill Podoprigora, Mikhail Efimov and Xuanteng Huang

Activity

1. brandtbucher

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   3.14bugs and security fixesbugs and security fixes

   topic-JIT

   on Nov 8, 2024
2. xuantengh commented on Nov 9, 2024

   

   xuantengh

   on Nov 9, 2024

   Contributor

   More actions

   Hi can I take and try this issue?
3. rruuaanng commented on Nov 9, 2024

   

   rruuaanng

   on Nov 9, 2024

   Contributor

   More actions

   Don't rush, please wait for the discussion result :)

   Reacted by Xuanteng Huang
4. brandtbucher commented on Nov 10, 2024

   

   brandtbucher

   on Nov 10, 2024

   MemberAuthor

   More actions

   This has already been discussed quite a bit offline with the relevant parties. I think we can probably move ahead with it.

   It's a pretty big project, but help is welcome if you really feel confident tackling it. It's not just deleting things, but also refactoring the code to remove the current API boundary between the interpreter and the optimizer. I'm happy to help out if you get stuck or need any advice, too.

   Reacted by RUANG (James Roy)
5. rruuaanng commented on Nov 11, 2024

   

   rruuaanng

   on Nov 11, 2024

   Contributor

   More actions

   @xuantengh Would you like to challenge?

   If you don't have the confidence to give it to me, I will finish it when I have time.
6. xuantengh commented on Nov 11, 2024

   

   xuantengh

   on Nov 11, 2024

   Contributor

   More actions

   Would you like to challenge?

   Hi I would like to give it a try.

   Reacted by RUANG (James Roy)
7. xuantengh commented on Nov 11, 2024

   

   xuantengh

   on Nov 11, 2024

   Contributor

   More actions

   Hi @brandtbucher, after some preliminary investigation, this issue aims to remove the following 3 internal C APIs:

   • _PyOptimizer_NewCounter
   • _PyOptimizer_NewUOpOptimizer
   • _PyOptimizer_Optimize

   where the first two are only used for tests (_testinternalcapi.c), and it's trival to rip them. For the last, it's used in instruction JUMP_BACKWARD as well as tier2 OPs _EXIT_TRACE and _DYNAMIC_EXIT. So we need to refactor the code to let these instruction/OPs directly conduct the optimizations?

   Please correct me if something wrong or missing.
8. rruuaanng commented on Nov 14, 2024

   

   rruuaanng

   on Nov 14, 2024

   Contributor

   More actions

   @xuantengh Maybe you should submit a PR :)
9. brandtbucher

   added a commit that references this issue on Jan 16, 2025

   GH-126599: Remove the "counter" optimizer/executor (GH-126853)

   b44ff6d
10. brandtbucher commented on Jan 16, 2025

    

    brandtbucher

    on Jan 16, 2025

    MemberAuthor

    More actions

    Thanks for getting the ball rolling with the counter optimizer/executor removal! I'll go ahead and rip out the rest.

    Reacted by Xuanteng Huang
11. brandtbucher

    self-assigned this

    on Jan 16, 2025
12. brandtbucher

    added a commit that references this issue on Jan 29, 2025

    GH-126599: Remove the PyOptimizer API (GH-129194)

    828b276
13. brandtbucher

    closed this as completedon Feb 12, 2025

Issue actions