GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-142913: Add test case for interpreter generator w/ overridden opcodes by DinoV · Pull Request #142911 · python/cpython

Conversation

@DinoV

@DinoV DinoV commented Dec 18, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

The interpreter generator has long supported giving it multiple bytecodes files where the latter files override opcodes defined in the earlier opcodes. This enables PEP 523 hook users to generate an interpreter loop using the existing opcodes and their own set of customizations. But there've been no test cases that validate this actually works.

This adds such a test case and fixes things up so this practically works. static functions used directly in the interpreter loop are moved into ceval.h. APIs which are used but not exported are marked as exported.

The generated test case is updated as mark of mark regen-cases so it'll stay in sync as the bytecodes change.

  • Issue: Add test coverage for interpreter loop generator overridden insts #142913

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@DinoV DinoV added the skip news label Dec 18, 2025
@DinoV
DinoV force-pushed the eval_loop_test branch from e25709a to 40d5d70 Compare December 18, 2025 00:22
@DinoV DinoV changed the title Add test case for interpreter generator w/ overridden opcodes gh-142913: Add test case for interpreter generator w/ overridden opcodes Dec 18, 2025
@bedevere-app bedevere-app Bot mentioned this pull request Dec 18, 2025
Add test coverage for interpreter loop generator overridden insts #142913
Closed
@DinoV
DinoV force-pushed the eval_loop_test branch 7 times, most recently from 85c6f7e to 506d4b6 Compare December 18, 2025 01:38
@DinoV DinoV linked an issue Dec 18, 2025 that may be closed by this pull request
Add test coverage for interpreter loop generator overridden insts #142913
Closed
@DinoV
DinoV force-pushed the eval_loop_test branch 3 times, most recently from 80a2701 to 8fec3b0 Compare December 18, 2025 17:37
@DinoV
DinoV marked this pull request as ready for review December 18, 2025 18:33
@DinoV
DinoV requested review from a team, AA-Turner, emmatyping, ericsnowcurrently, erlend-aasland and markshannon as code owners December 18, 2025 18:33
@bedevere-app bedevere-app Bot added the awaiting core review label Dec 18, 2025
@DinoV DinoV mentioned this pull request Jan 7, 2026
gh-143531: Use macro to check if PEP 523 is hooked #143532
Merged
@DinoV
DinoV force-pushed the eval_loop_test branch 2 times, most recently from abb5095 to c5caac6 Compare January 13, 2026 20:17
@DinoV
DinoV requested a review from mpage January 14, 2026 17:07
mpage
mpage reviewed Jan 14, 2026
View reviewed changes
Comment thread Modules/_testinternalcapi.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Modules/_testinternalcapi.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Modules/_testinternalcapi.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Modules/_testinternalcapi.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
DinoV added 2 commits January 14, 2026 18:08
@DinoV
Add test case for interpreter generator w/ overridden opcodes
85ad71f
@DinoV
Add missing decrefs
b1fa3a9
@DinoV
Rebase & regen
893afef
@DinoV
DinoV force-pushed the eval_loop_test branch from 54dee3c to 893afef Compare January 15, 2026 02:26
mpage
mpage approved these changes Jan 15, 2026
View reviewed changes

@mpage mpage left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Modules/_testinternalcapi/interpreter.c
#define Py_BUILD_CORE_MODULE
#endif

#include "../../Python/ceval.h"

@mpage mpage Jan 15, 2026

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Third-party interpreter loops will still need to borrow the contents of Python/ceval.h and Python/ceval_macros.h, right? At some point it'd be nice if we could avoid borrowing these as well.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@DinoV DinoV Jan 15, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think depends on what you mean by "borrow". They'll need to include them (which should require a little extra work as the files aren't in Include). For ceval.h we could potentially do better but that's in-avoidable for ceval_macros.h as they are in fact (mostly) macros.

But this does put them in a place where rather than needing to do any actual copying they can build and re-use those source files. A re-generated interpreter could have some minor drift between patch versions of Python but because the bytecodes.c + ceval.h are all cohesive they should just either be missing or have extra bug fixes vs the patch version they're running on.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@mpage mpage Jan 16, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm an extension author how would I set up my build so that I can include these files? I was assuming that since they aren't in Include I would probably need to copy them into my own source tree...

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Jan 15, 2026
@DinoV
DinoV merged commit 4d5a676 into python:main Jan 16, 2026
64 checks passed

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the awaiting merge label Jan 16, 2026
@DinoV DinoV mentioned this pull request Jan 16, 2026
gh-142913: Update generated code that conflicted with other landed change #143932
Merged
@Fidget-Spinner

Fidget-Spinner commented Jan 16, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This broke the JIT builds/tests it seems with ./configure --enable-experimental-jit=interpreter --with-pydebug. Could you please look into it? Thank you!
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@Fidget-Spinner Fidget-Spinner mentioned this pull request Jan 16, 2026
gh-130415: Narrowing to constants in branches involving is comparisons with a constant #143895
Merged
encukou added a commit to encukou/cpython that referenced this pull request Jan 21, 2026
@encukou
pythongh-142913: Revert adding test_replaced_interpreter
deb66d5 
This partially reverts commit 4d5a676
(pythonGH-142911)

This test fails when re-run in --huntrleaks mode.
@encukou encukou mentioned this pull request Jan 21, 2026
gh-142913: Revert adding test_replaced_interpreter #144110
Merged
meta-codesync Bot pushed a commit to facebookincubator/cinderx that referenced this pull request Jan 22, 2026
@DinoV @meta-codesync
Fix the CinderX 3.15 build
92da407 
Summary:
Fixes up the CinderX build after the latest import. This brought in python/cpython#142911 which move somethings into ceval.h so that we can directly include it rather than borrowing those functions. We bring back `cinderx-only-headers` and now use them on 3.10 and 3.15+.

There's also a fix related to python/cpython#143569 where we need to update the interpreter loop.

Also python/cpython@bb25f72 seems to have refactored `_PyDict_LookupIndex` to use `_PyDict_LookupIndexAndValue`.

There's some more work after this to further reduce our usage of `ceval_macros.h` but this just gets things working again.

Reviewed By: itamaro

Differential Revision: D91146024

fbshipit-source-id: 6da3ef18fe5fe5f79e29a71cdbe98491bbb45a9d
encukou added a commit that referenced this pull request Jan 22, 2026
@encukou
gh-142913: Revert adding test_replaced_interpreter (GH-144110)
3e66171 
This partially reverts commit 4d5a676
(GH-142911)

The removed test fails when re-run in --huntrleaks mode.
@cdce8p cdce8p mentioned this pull request Jan 24, 2026
gh-142913: Add generated test files to gitattributes #144209
Merged
@jjhelmus jjhelmus mentioned this pull request Feb 11, 2026
CPython 3.15.0a5 -> 3.15.0a6 astral-sh/python-build-standalone#981
Merged
thunder-coding pushed a commit to thunder-coding/cpython that referenced this pull request Feb 15, 2026
@DinoV @thunder-coding
pythongh-142913: Add test case for interpreter generator w/ overridde…
0efcab0 
…n opcodes (python#142911)

Add test case for interpreter generator w/ overridden opcodes
thunder-coding pushed a commit to thunder-coding/cpython that referenced this pull request Feb 15, 2026
@encukou @thunder-coding
pythongh-142913: Revert adding test_replaced_interpreter (pythonGH-14…
707d3e6 
…4110)

This partially reverts commit 4d5a676
(pythonGH-142911)

The removed test fails when re-run in --huntrleaks mode.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mpage mpage mpage approved these changes
@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner
@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner
@erlend-aasland erlend-aasland Awaiting requested review from erlend-aasland erlend-aasland is a code owner
@AA-Turner AA-Turner Awaiting requested review from AA-Turner AA-Turner is a code owner
@emmatyping emmatyping Awaiting requested review from emmatyping emmatyping is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add test coverage for interpreter loop generator overridden insts

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@DinoV @Fidget-Spinner @mpage
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.