GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GH-113464: Generate a more efficient JIT by brandtbucher · Pull Request #118512 · python/cpython

Conversation

@brandtbucher

@brandtbucher brandtbucher commented May 2, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This breaks up the JIT into smaller functions, reduces a lot of branching in hot inner loops, and generally makes the C code cleaner (and probably faster).

Currently, we generate declarative structures at build time that we then loop over in order to emit the desired machine code at runtime. For example, the _STORE_FAST stencil looks like this:

Details 
static const unsigned char _STORE_FAST_code_body[61] = {
    0x50, 0x48, 0x8b, 0x45, 0xf8, 0x48, 0x83, 0xc5,
    0xf8, 0x0f, 0xb7, 0x0d, 0x00, 0x00, 0x00, 0x00,
    0x49, 0x8b, 0x7c, 0xcd, 0x48, 0x49, 0x89, 0x44,
    0xcd, 0x48, 0x48, 0x85, 0xff, 0x74, 0x0f, 0x48,
    0x8b, 0x07, 0x85, 0xc0, 0x78, 0x08, 0x48, 0xff,
    0xc8, 0x48, 0x89, 0x07, 0x74, 0x07, 0x58, 0xff,
    0x25, 0x00, 0x00, 0x00, 0x00, 0xff, 0x15, 0x00,
    0x00, 0x00, 0x00, 0x58,
};
static const Hole _STORE_FAST_code_holes[4] = {
    {0xc, HoleKind_R_X86_64_GOTPCREL, HoleValue_DATA, NULL, -0x4},
    {0x31, HoleKind_R_X86_64_GOTPCRELX, HoleValue_DATA, NULL, 0x4},
    {0x37, HoleKind_R_X86_64_GOTPCRELX, HoleValue_DATA, NULL, 0xc},
};
static const unsigned char _STORE_FAST_data_body[25] = {
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
};
static const Hole _STORE_FAST_data_holes[4] = {
    {0x0, HoleKind_R_X86_64_64, HoleValue_OPARG, NULL, 0x0},
    {0x8, HoleKind_R_X86_64_64, HoleValue_CONTINUE, NULL, 0x0},
    {0x10, HoleKind_R_X86_64_64, HoleValue_ZERO, &_Py_Dealloc, 0x0},
};

This very general approach means that we have a lot of complex logic in our hot inner loop to decode instructions and set up values for patching that may not even be needed. It's also very branchy, since we're essentially "interpreting" the array of holes for each instruction.

With this PR, jit_stencils.h instead contains the following function:

Details 
void
emit__STORE_FAST(
    unsigned char *code, unsigned char *data, _PyExecutorObject *executor,
    const _PyUOpInstruction *instruction, uintptr_t instruction_starts[])
{
    const unsigned char code_body[60] = {
        0x50, 0x48, 0x8b, 0x45, 0xf8, 0x48, 0x83, 0xc5,
        0xf8, 0x0f, 0xb7, 0x0d, 0x00, 0x00, 0x00, 0x00,
        0x49, 0x8b, 0x7c, 0xcd, 0x48, 0x49, 0x89, 0x44,
        0xcd, 0x48, 0x48, 0x85, 0xff, 0x74, 0x0f, 0x48,
        0x8b, 0x07, 0x85, 0xc0, 0x78, 0x08, 0x48, 0xff,
        0xc8, 0x48, 0x89, 0x07, 0x74, 0x07, 0x58, 0xff,
        0x25, 0x00, 0x00, 0x00, 0x00, 0xff, 0x15, 0x00,
        0x00, 0x00, 0x00, 0x58,
    };
    const unsigned char data_body[24] = {
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    };
    memcpy(data, data_body, sizeof(data_body));
    patch_64(data + 0x0, instruction->oparg);
    patch_64(data + 0x8, (uintptr_t)code + sizeof(code_body));
    patch_64(data + 0x10, (uintptr_t)&_Py_Dealloc);
    memcpy(code, code_body, sizeof(code_body));
    patch_32r(code + 0xc, (uintptr_t)data + -0x4);
    patch_x86_64_32rx(code + 0x31, (uintptr_t)data + 0x4);
    patch_x86_64_32rx(code + 0x37, (uintptr_t)data + 0xc);
}

This function is called directly to emit the machine code for every _STORE_FAST instruction, and hardcodes the logic for all of the necessary copies and patches. The result is one indirect call, no unnecessary branching, and (in my opinion) cleaner code, since a lot of the tricky logic is now hidden away in generated files.

I know this is right before feature freeze, but I'd really like to get this in 3.13 since it will make backporting any fixes much easier. It doesn't change the actual jitted code in any way.

Note to reviewers: the diff is a bit messy, so it may make more sense to compare the before-vs-after files side-by-side instead.

  • Issue: JIT Compilation #113464

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
stonebig reacted with heart emoji
All reactions
  • ❤️ 1 reaction
brandtbucher and others added 16 commits May 1, 2024 15:15
@brandtbucher
Replace stencils with dedicated writer functions
2901caf
@brandtbucher
Generate patching logic
f30fa64
@brandtbucher
Cleanup
23e211c
@brandtbucher
uint64_t -> uintptr_t
431fbed
@brandtbucher
uintptr_t -> uint64_t
3e6b25c
@brandtbucher
Linting
82030c8
@brandtbucher
Restore AArch64 pair folding
236af82
@brandtbucher
Cleanup
3b7e693
@brandtbucher
Add missing relocations
fbb97fc
@brandtbucher
Fix AArch64 folds
c40bb34
@brandtbucher
Dedent
bd570b5
@brandtbucher
Use a single array of structs
b2fd9d2
@brandtbucher
Move C initializer formation to StencilGroup
7aa12a2
@brandtbucher
Add comment on why data is first
9ec64ac
@brandtbucher
Silence warnings
eb0826f
@brandtbucher
Add missing space
a04d7f8
@brandtbucher brandtbucher added performance Performance or resource usage interpreter-core (Objects, Python, Grammar, and Parser dirs) build The build process and cross-build labels May 2, 2024
@brandtbucher brandtbucher self-assigned this May 2, 2024
@bedevere-app bedevere-app Bot added the awaiting core review label May 2, 2024
@bedevere-app bedevere-app Bot mentioned this pull request May 2, 2024
JIT Compilation #113464
Closed
@brandtbucher
brandtbucher requested a review from markshannon May 2, 2024 15:51
@brandtbucher

brandtbucher commented May 2, 2024

Copy link
Copy Markdown
Member Author

@savannahostrowski, I'd love to get your review of this if you have a few cycles.
savannahostrowski reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@brandtbucher brandtbucher added the skip news label May 2, 2024
savannahostrowski
savannahostrowski reviewed May 2, 2024
View reviewed changes

@savannahostrowski savannahostrowski left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of comments and questions but after sitting and reading through this code a bunch over the last week or two, I'm excited about how much more readable this will get with this change! 💆‍♀️

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
stonebig reacted with heart emoji
All reactions
  • ❤️ 1 reaction
Comment thread Python/jit.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Python/jit.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Tools/jit/_stencils.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Tools/jit/_writer.py
"""Yield a JIT compiler line-by-line as a C header file."""
yield from _dump_header()
for opname, group in groups.items():
for opname, group in sorted(groups.items()):

@savannahostrowski savannahostrowski May 2, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason that this needs to be sorted?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@brandtbucher brandtbucher May 3, 2024

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, I just like it that way (if you couldn't tell by now). ;)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
savannahostrowski reacted with laugh emoji
All reactions
  • 😄 1 reaction
Comment thread Tools/jit/_stencils.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Tools/jit/_stencils.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
brandtbucher added 2 commits May 3, 2024 14:30
@brandtbucher
Clarify which patch functions are relaxing (and what that means)
b919fcc
@brandtbucher
Exaplain why GOT is commented out
46adf09
savannahostrowski
savannahostrowski approved these changes May 3, 2024
View reviewed changes

@savannahostrowski savannahostrowski left a comment •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding in the comment about the naming conventions - I think that helps! Otherwise, this looks pretty solid to me (barring some Windows CI failures). Lots of moving things into function but it's a whole lot more readable! 🎉

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@brandtbucher

brandtbucher commented May 3, 2024

Copy link
Copy Markdown
Member Author

Windows JIT CI fixed in GH-118564.
stonebig reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@brandtbucher
brandtbucher merged commit 1b7e5e6 into python:main May 3, 2024
@bedevere-app bedevere-app Bot removed the awaiting core review label May 3, 2024
SonicField pushed a commit to SonicField/cpython that referenced this pull request May 8, 2024
@brandtbucher @SonicField
pythonGH-113464: Generate a more efficient JIT (pythonGH-118512)
1493daa
@brandtbucher brandtbucher added the topic-JIT label May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@savannahostrowski savannahostrowski savannahostrowski approved these changes
@markshannon markshannon Awaiting requested review from markshannon

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@brandtbucher brandtbucher

Labels

build The build process and cross-build interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage skip news topic-JIT

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@brandtbucher @savannahostrowski
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.