GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-105481: add flags to each instr in the opcode metadata table, to replace opcode.hasarg/hasname/hasconst by iritkatriel · Pull Request #105482 · python/cpython

Conversation

@iritkatriel

@iritkatriel iritkatriel commented Jun 7, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This is not yet working for pseudo instructions. Should we add them to bytecodes.c in some form?

Also, I needed to add a silly assert to YIELD_VALUE (which is irregular) to make it look to the code generator like it uses oparg. The bytecode doesn't use the oparg but oparg is set to the exception stack depth, so the opcode needs to be considered as HAS_ARG because there are assertions that oparg is 0 for instructions without args.

  • Issue: Generate opcode metadata from bytecodes.c instead of opcode.py #105481

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@iritkatriel
pythongh-105481: add flags to each instr in the opcode metadata table…
f0898ab
…, to replace opcode.hasarg/hasname/hasconst
@iritkatriel
iritkatriel requested a review from markshannon as a code owner June 7, 2023 22:40
@bedevere-bot bedevere-bot added the awaiting core review label Jun 7, 2023
@bedevere-bot bedevere-bot mentioned this pull request Jun 7, 2023
Generate opcode metadata from bytecodes.c instead of opcode.py #105481
Closed
@iritkatriel
iritkatriel requested a review from gvanrossum June 7, 2023 22:44
@gvanrossum

gvanrossum commented Jun 7, 2023

Copy link
Copy Markdown
Member

This is not yet working for pseudo instructions. Should we add them to bytecodes.c in some form?

I think so (unless all the flags are always false for all pseudo-ops, which I doubt is the case). In fact I think we should probably propose a solution for moving the canonical definition of the numeric opcode values before we move on this.

(Other than that, this approach seems fine.)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iritkatriel

iritkatriel commented Jun 7, 2023

Copy link
Copy Markdown
Member Author

Also I don’t know how to identify the jump opcodes in the generator. JUMPBY is used for caches so it’s no help. Maybe we could have a SKIP_CACHE macro and the a jumpby is really a jump?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

gvanrossum
gvanrossum reviewed Jun 7, 2023
View reviewed changes
Comment thread Python/bytecodes.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Python/opcode_metadata.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Python/compile.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Tools/cases_generator/generate_cases.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Tools/cases_generator/generate_cases.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Tools/cases_generator/generate_cases.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@iritkatriel
iritkatriel marked this pull request as draft June 8, 2023 09:58
@bedevere-bot bedevere-bot removed the awaiting core review label Jun 8, 2023
iritkatriel added 2 commits June 8, 2023 12:45
@iritkatriel
emit symbolic flags
35430de
@iritkatriel
add macros CO_CONSTS, CO_NAMES
fdce9f4
gvanrossum
gvanrossum reviewed Jun 10, 2023
View reviewed changes

@gvanrossum gvanrossum left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I don’t know how to identify the jump opcodes in the generator. JUMPBY is used for caches so it’s no help. Maybe we could have a SKIP_CACHE macro and the a jumpby is really a jump?

Yeah, disambiguating these through their macro names sounds fine.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Tools/cases_generator/generate_cases.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Tools/cases_generator/generate_cases.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Tools/cases_generator/generate_cases.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Python/ceval_macros.h Resolved

Uh oh!

There was an error while loading. Please reload this page.

This was referenced Jun 11, 2023
gh-105481: add pseudo-instructions to the cases DSL #105506
Merged
Creation of superblocks faster-cpython/ideas#558
Open
@iritkatriel
code review comments
14fa4c6
@gvanrossum

gvanrossum commented Jun 13, 2023

Copy link
Copy Markdown
Member

LG. Next steps?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

iritkatriel added 2 commits June 13, 2023 17:21
@iritkatriel
Merge remote-tracking branch 'upstream/main' into metadata
bf9df5f
@iritkatriel
enable asseertions for pseudo-ops as well
e93f976
@iritkatriel

iritkatriel commented Jun 13, 2023

Copy link
Copy Markdown
Member Author

LG. Next steps?

next is to rebase and enable the assertions for pseudo ops. If that worksI'll check the diff, maybe this PR is done then.

gvanrossum reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iritkatriel
remove delim arg
fe06e04
@iritkatriel

iritkatriel commented Jun 13, 2023

Copy link
Copy Markdown
Member Author

Maybe we should make the parser or generator complain if you use frame->f_code->co_names or frame->f_code->co_consts in a bytecode definition?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gvanrossum

gvanrossum commented Jun 13, 2023

Copy link
Copy Markdown
Member

Maybe we should make the parser or generator complain if you use frame->f_code->co_names or frame->f_code->co_consts in a bytecode definition?

How would it know? Maybe the original idea of looking for that sequence of tokens isn't so bad. Just don't fold it into variable_used -- make it a separate helper function.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iritkatriel

iritkatriel commented Jun 13, 2023

Copy link
Copy Markdown
Member Author

Maybe we should make the parser or generator complain if you use frame->f_code->co_names or frame->f_code->co_consts in a bytecode definition?

How would it know? Maybe the original idea of looking for that sequence of tokens isn't so bad. Just don't fold it into variable_used -- make it a separate helper function.

I didn't like that frame->f_code->co_names means something different then code = frame->f_code; code->co_names.
We could make it illegal to access frame->f_code from the code, and add #defines for everything you might need from there.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@gvanrossum

gvanrossum commented Jun 13, 2023

Copy link
Copy Markdown
Member

I grepped and I see about a dozen places where frame->f_code is used to access something other than co_consts or co_names. So you'd need to review those.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

iritkatriel added 2 commits June 13, 2023 20:43
@iritkatriel
forbid reserved words 'co_consts', 'co_names'
0d3dc8e
@iritkatriel
change order of args to PseudoInstruction
33dfd3e
@iritkatriel
iritkatriel marked this pull request as ready for review June 13, 2023 19:53
@bedevere-bot bedevere-bot added the awaiting core review label Jun 13, 2023
@iritkatriel
Merge branch 'main' into metadata
33abc4a
gvanrossum
gvanrossum approved these changes Jun 13, 2023
View reviewed changes

@gvanrossum gvanrossum left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Undraft, add skip news label, go!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Jun 13, 2023
@iritkatriel iritkatriel added the skip news label Jun 13, 2023
@iritkatriel
iritkatriel merged commit be2779c into python:main Jun 13, 2023
@bedevere-bot bedevere-bot removed the awaiting merge label Jun 13, 2023
iritkatriel
iritkatriel commented Jun 13, 2023
View reviewed changes
Comment thread Python/opcode_metadata.h
[SEND] = { true, INSTR_FMT_IBC, HAS_ARG_FLAG },
[SEND_GEN] = { true, INSTR_FMT_IBC, HAS_ARG_FLAG },
[INSTRUMENTED_YIELD_VALUE] = { true, INSTR_FMT_IB, HAS_ARG_FLAG },
[YIELD_VALUE] = { true, INSTR_FMT_IB, HAS_ARG_FLAG },

@iritkatriel iritkatriel Jun 13, 2023

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed that this PR changed the format of YIELD_VALUE/INSTRUMENTED_YIELD_VALUE from INSTR_FMT_IX to INSTR_FMT_IB. This is because we added the assertion that makes the generator identify it has HAS_ARG. I guess the new value is correct?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@gvanrossum gvanrossum Jun 13, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In opcode.py it is classified as having an arg (by being >= HAVE_ARGUMENT) so I think it's correct. The oparg is used elsewhere to indicate the stack depth.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gvanrossum gvanrossum gvanrossum approved these changes
@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@iritkatriel @gvanrossum @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.