GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GH-98831: Refactor and fix cases generator by gvanrossum · Pull Request #99526 · python/cpython

Conversation

@gvanrossum

@gvanrossum gvanrossum commented Nov 16, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This refactor also fixes code generation for cache effects that aren't unused and removes check_overlaps(). It shows it all off for the BINARY_SUBSCR family. (I had an earlier PR (GH-99408) for this, but it got a bit stale and I decided to delete the history.)

  • Issue: Generate the interpreter #98831

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
gvanrossum added 2 commits November 15, 2022 20:29
@gvanrossum
Thoroughly refactor cases generator
9cfb501
@gvanrossum
Complete cache effects for BINARY_SUBSCR family
b5163cf
@bedevere-bot bedevere-bot added the awaiting core review label Nov 16, 2022
@bedevere-bot bedevere-bot mentioned this pull request Nov 16, 2022
Generate the interpreter #98831
Closed
@gvanrossum gvanrossum added the skip news label Nov 16, 2022
@gvanrossum
Merge remote-tracking branch 'origin/main' into cases-refactor
25d34cb
@gvanrossum
gvanrossum marked this pull request as ready for review November 16, 2022 23:28
@gvanrossum
gvanrossum requested a review from brandtbucher November 16, 2022 23:28
@gvanrossum

gvanrossum commented Nov 16, 2022

Copy link
Copy Markdown
Member Author

I have a newer version of the same code in GH-99495 (which adds macros and ops) but that's not fully baked and I figured I'd release this in stages so it's easier on the reviewer. If you'd rather review the tooling in even larger chunks let me know. (I think the changes to the instruction definitions are pretty mild in this one.)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

brandtbucher
brandtbucher approved these changes Nov 17, 2022
View reviewed changes

@brandtbucher brandtbucher left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. A few comments:

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Python/bytecodes.c
PyObject *container = SECOND();
_PyBinarySubscrCache *cache = (_PyBinarySubscrCache *)next_instr;
uint32_t type_version = read_u32(cache->type_version);
inst(BINARY_SUBSCR_GETITEM, (container, sub, unused/1, type_version/2, func_version/1 -- unused)) {

@brandtbucher brandtbucher Nov 17, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super cool to see the cache reads in action!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Python/bytecodes.c Outdated
}
PyObject *arg = TOP();
PyObject *res = _PyCFunction_TrampolineCall(cfunc, PyCFunction_GET_SELF(callable), arg);
PyObject *res = cfunc(PyCFunction_GET_SELF(callable), arg);

@brandtbucher brandtbucher Nov 17, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to change this (also the two instances below)?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Tools/cases_generator/parser.py
return self.header.inputs

@property
def outputs(self) -> list[StackEffect]:

@brandtbucher brandtbucher Nov 17, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def outputs(self) -> list[StackEffect]:
def outputs(self) -> list[OutputEffect]:

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Tools/cases_generator/generate_cases.py
Comment on lines +73 to +75
if ceffect.name != "unused":
# TODO: if name is 'descr' use PyObject *descr = read_obj(...)
bits = ceffect.size * 16

@brandtbucher brandtbucher Nov 17, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had suggestions on these from the other PR, not sure if you saw (I like _ more than unused, and both "unused" and 16 might benefit from constant names in this file).

Unless you just disagreed, in which case, feel free to ignore this. :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Tools/cases_generator/generate_cases.py
# Write output stack effect assignments
input_names = [seffect.name for seffect in self.input_effects]
for i, output in enumerate(reversed(self.output_effects), 1):
if output.name not in input_names and output.name != "unused":

@brandtbucher brandtbucher Nov 17, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For output.name not in input_names, don't we need to check that they occur at the same (stack-adjusted) position, and write them out if not?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Tools/cases_generator/generate_cases.py
Comment on lines +179 to +183
while tkn := psr.next(raw=True):
if tkn.text == BEGIN_MARKER:
break
else:
raise psr.make_syntax_error(f"Unexpected token")
return instrs, supers, families
raise psr.make_syntax_error(f"Couldn't find {BEGIN_MARKER!r} in {psr.filename}")

@brandtbucher brandtbucher Nov 17, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wild, I've never seen while/else before...

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Tools/cases_generator/generate_cases.py
Comment on lines +222 to +226
print(
f"Unknown instruction {target!r} predicted in {instr.name!r}",
file=sys.stderr,
)
self.errors += 1

@brandtbucher brandtbucher Nov 17, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You do this move in lots of places... maybe add a method?

def error(self, e: str) -> None:
    print(e, file=sys.stderr)
    self.errors += 1

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Tools/cases_generator/generate_cases.py
Comment on lines +257 to +258
if len(members) < 2:
continue

@brandtbucher brandtbucher Nov 17, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be an error, yeah?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Nov 17, 2022
gvanrossum added 2 commits November 17, 2022 16:08
@gvanrossum
Merge remote-tracking branch 'origin/main' into cases-refactor
4720c91
@gvanrossum
Fix three accidental changes
74da54c
gvanrossum added a commit to gvanrossum/cpython that referenced this pull request Nov 18, 2022
@gvanrossum
Code review from pythonGH-99526
598e71a
@gvanrossum
gvanrossum merged commit 4f5e1cb into python:main Nov 18, 2022
@bedevere-bot bedevere-bot removed the awaiting merge label Nov 18, 2022
@gvanrossum
gvanrossum deleted the cases-refactor branch November 18, 2022 01:06
gvanrossum added a commit to gvanrossum/cpython that referenced this pull request Nov 18, 2022
@gvanrossum
Code review from pythonGH-99526
d84a5c3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@brandtbucher brandtbucher brandtbucher approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@gvanrossum @brandtbucher @bedevere-bot
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.