Description

iritkatriel

opened on Jun 7, 2023

Issue body actions

We would ideally have bytecodes.c as the single source of truth about opcodes. So opcode.py and the code generated from it should be replaced by alternatives from the cases_generator, if we can.

Linked PRs

• gh-105481: add flags to each instr in the opcode metadata table, to replace opcode.hasarg/hasname/hasconst #105482
• gh-105481: add pseudo-instructions to the cases DSL #105506
• gh-105481: Fix types and bug for pseudos #105788
• gh-105481: add HAS_JUMP flag to opcode metadata #105791
• gh-105481: remove HAS_ARG, HAS_CONST, IS_JUMP_OPCODE, IS_PSEUDO_OPCODE and replace by their new versions #105865
• gh-105481: generate _specializations and _specialized_instructions from bytecodes.c #105913
• gh-105481: refactor instr flag related code into a new InstructionFlags class #105950
• gh-105481: move Python/opcode_metadata.h to Include/internal/pycore_opcode_metadata.h #106673
• gh-105481: expose opcode metadata via the _opcode module #106688
• gh-105481: Generate the opcode lists in dis from data extracted from bytecodes.c #106758
• gh-105481: do not auto-generate pycore_intrinsics.h #106913
• gh-105481: add OPCODE_IS_INSTRUMENTED (generated from bytecodes.c) to replace MIN_INSTRUMENTED_OPCODE (defined in opcode.py) #107276
• gh-105481: remove dependency of _inline_cache_entries on opname #107339
• gh-105481: the ENABLE_SPECIALIZATION flag does not need to be generated by the build script, or exposed in opcode.py #107534
• gh-105481: combine regen-opcode-targets with regen-opcode to avoid calculating the specialized opcodes in two places #107540
• gh-105481: simplify definition of pseudo ops in Lib/opcode.py #107561
• GH-105481: Mark more files as generated #107598
• gh-105481: split opcode_ids.h out of opcode.h so that it can be generated separately #107866
• gh-105481: reduce repetition in opcode metadata generation code #107942
• gh-105481: generate op IDs from bytecode.c instead of hard coding them in opcode.py #107971
• gh-105481: generate _PyOpcode_Deopt and _PyOpcode_OpName in generate_cases instead of generate_opcode_h #108025
• gh-105481: fix out of date comment #108079
• gh-105481: opcode.h is no longer generated during the build #108080
• gh-105481: remove regen-opcode. Generated _PyOpcode_Caches in regen-cases. #108367

Activity

1. iritkatriel

   added

   type-featureA feature request or enhancementA feature request or enhancement

   on Jun 7, 2023
2. iritkatriel

   added a commit that references this issue on Jun 7, 2023

   pythongh-105481: add flags to each instr in the opcode metadata table…

   f0898ab
3. iritkatriel

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   on Jun 7, 2023
4. iritkatriel

   added 5 commits that reference this issue on Jun 8, 2023

   pythongh-105481: add pseudo-instructions to the cases DSL

   b68004f

   gh-105481: add pseudo-instructions to the bytecodes DSL (#105506)

   58f5227

   gh-105481: add flags to each instr in the opcode metadata table, to r…

   be2779c

   pythongh-105481: add HAS_JUMP flag to opcode metadata

   e43ae9c

   pythongh-105481: add HAS_JUMP flag to opcode metadata

   54aad59
5. gvanrossum

   added a commit that references this issue on Jun 14, 2023

   gh-105481: Fix types and a bug for pseudos (#105788)

   d50930a
6. iritkatriel

   added a commit that references this issue on Jun 14, 2023

   gh-105481: add HAS_JUMP flag to opcode metadata (#105791)

   d1b0297
7. iritkatriel commented on Jun 15, 2023

   

   iritkatriel

   on Jun 15, 2023

   MemberAuthor

   More actions

   @gvanrossum The old macros HAS_ARG and HAS_CONST are exposed in Include/opcode.h (and have been for a long time). Does this mean that before we can remove them, we need to expose the opcode_metadata.h file there as well? We should also mark all of this as unstable API (how do we do that?)
8. carljm

   added a commit that references this issue on Jun 15, 2023

   Merge branch 'main' into modulelocals

   166aca9
9. gvanrossum commented on Jun 16, 2023

   

   gvanrossum

   on Jun 16, 2023

   Member

   More actions

   Neither HAS_ARG nor HAS_CONST is mentioned in the docs at all. I am of the opinion that in this case this implies they are not public, and anyone using them should not be surprised if they disappear. (It would be more of an issue if they turned into lies.)

   opcode.h is mentioned only once in the docs, as the source of truth for dis.py. (It is also mentioned twice in Misc/NEWS.d/, but that's really just a changelog.)

   I do think we ought to provide similar functionality to debuggers.

   Info about declaring unstable APIs comes from PEP 649, which links to https://devguide.python.org/developer-workflow/c-api/#c-api. It looks like you must use the PyUnstable_ prefix (e.g. PyUnstable_BytecodeHasArg), and it ought to be in a .h file under Include/cpython/.
10. iritkatriel commented on Jun 16, 2023

    

    iritkatriel

    on Jun 16, 2023

    MemberAuthor

    More actions

    I do think we ought to provide similar functionality to debuggers.

    Do you mean to expose the stuff in opcode_metadata.h file through some unstable c api?
11. gvanrossum commented on Jun 16, 2023

    

    gvanrossum

    on Jun 16, 2023

    Member

    More actions

    Yeah, to the extent that it’s useful, in function form (so the data structure is not public).

    Or we can wait until someone asks.
12. 32 remaining items
13. iritkatriel

    added 8 commits that reference this issue on Aug 14, 2023

    pythongh-105481: allocate op IDs in cases generator. Generate _PyOpco…

    0811c6d

    pythongh-105481: generate _PyOpcode_Deopt and _PyOpcode_OpName in gen…

    2b57422

    gh-105481: generate op IDs from bytecode.c instead of hard coding the…

    665a439

    pythongh-105481: fix out of date comment

    81b0a46

    pythongh-105481: opcode.h is no longer generated during the build

    d9373ff

    gh-105481: fix out of date comment (#108079)

    2da67a5

    gh-105481: opcode.h is no longer generated during the build (#108080)

    0b243c2

    pythongh-105481: remove regen-opcode. Generated _PyOpcode_Caches in r…

    fb23d1b
14. iritkatriel

    closed this as completedin #108367on Aug 23, 2023
15. iritkatriel

    added a commit that references this issue on Aug 23, 2023

    gh-105481: remove regen-opcode. Generated _PyOpcode_Caches in regen-c…

    72119d1
16. vstinner commented on Nov 8, 2023

    

    vstinner

    on Nov 8, 2023

    Member

    More actions

    @encukou wrote "Unstable API needs documentation and tests."

    I reopen this issue and close gh-107149.
17. vstinner

    reopened this on Nov 8, 2023
18. iritkatriel commented on Nov 13, 2023

    

    iritkatriel

    on Nov 13, 2023

    MemberAuthor

    More actions

    I'm not sure it was the right move to rename _PyUnstable_GetUnaryIntrinsicName to PyUnstable_GetUnaryIntrinsicName, etc.

    I created them as private because I did not intend to document them as a public API (we need a reason to do that). I put Unstable in the name because it's one of those things that can change all the time. If we don't want the _PyUnstable_ prefix then let's remove the Unstable and leave the _ until we have a reason/decision to make these part of the public C API.
19. iritkatriel commented on Nov 13, 2023

    

    iritkatriel

    on Nov 13, 2023

    MemberAuthor

    More actions

    I'm moving this discussion back to #107149, and closing this, because this PR was about a large change, and #107149 is about the naming.
20. iritkatriel

    closed this as completedon Nov 13, 2023

Issue actions