GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-87092: expose the compiler's codegen to python for unit tests by iritkatriel · Pull Request #99111 · python/cpython

Conversation

@iritkatriel

@iritkatriel iritkatriel commented Nov 4, 2022 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

With this we can run just codegen from python, and get the un-optimized instruction sequence for an AST.

  • Issue: Split compiler into code-gen, optimizer and assembler. #87092

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
iritkatriel added 2 commits November 4, 2022 16:50
@iritkatriel
extract compiler_codegen from compiler_mod
d5f14fc
@iritkatriel
add unit tests for codegen
89b8357
@iritkatriel
iritkatriel requested a review from markshannon as a code owner November 4, 2022 22:03
@bedevere-bot bedevere-bot mentioned this pull request Nov 4, 2022
Split compiler into code-gen, optimizer and assembler. #87092
Closed
@bedevere-bot bedevere-bot added the awaiting core review label Nov 4, 2022
@iritkatriel iritkatriel self-assigned this Nov 4, 2022
@iritkatriel iritkatriel added tests Tests in the Lib/test dir skip news interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Nov 5, 2022
iritkatriel and others added 3 commits November 5, 2022 10:11
@iritkatriel
fix whitespace
ddac06a
@iritkatriel
fix refleak
b07b9c6
@iritkatriel
Merge branch 'main' into expose_codegen
a2a1c17
@iritkatriel iritkatriel added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 5, 2022
@bedevere-bot

bedevere-bot commented Nov 5, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @iritkatriel for commit a2a1c17 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 5, 2022
@iritkatriel
Merge remote-tracking branch 'upstream/main' into expose_codegen
3c1c647
@iritkatriel iritkatriel added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 8, 2022
@bedevere-bot

bedevere-bot commented Nov 8, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @iritkatriel for commit 3c1c647 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 8, 2022
iritkatriel added 4 commits November 8, 2022 20:57
@iritkatriel
fix flags bug
9bafd3a
@iritkatriel
Merge remote-tracking branch 'upstream/main' into expose_codegen
ae1e146
@iritkatriel
regenerate files
ce45de8
@iritkatriel
fix flags in _PyCompile_CodeGen
d98f150
@iritkatriel iritkatriel added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 8, 2022
@bedevere-bot

bedevere-bot commented Nov 8, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @iritkatriel for commit d98f150 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 8, 2022
@iritkatriel
PyCompilerFlags stored on struct compiler by value
00aea64
@iritkatriel iritkatriel added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 9, 2022
@bedevere-bot

bedevere-bot commented Nov 9, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @iritkatriel for commit 00aea64 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 9, 2022
iritkatriel added 3 commits November 9, 2022 10:51
@iritkatriel
Merge remote-tracking branch 'upstream/main' into expose_codegen
50c0cc2
@iritkatriel
Merge remote-tracking branch 'upstream/main' into expose_codegen
61edee2
@iritkatriel
regen
ab3f67c
@iritkatriel iritkatriel added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 10, 2022
@bedevere-bot

bedevere-bot commented Nov 10, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @iritkatriel for commit ab3f67c 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 10, 2022
markshannon
markshannon reviewed Nov 11, 2022
View reviewed changes

@markshannon markshannon left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nice improvement.
I have a few questions and comments.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Include/internal/pycore_compile.h

/* Access compiler internals for unit testing */

PyAPI_FUNC(PyObject*) _PyCompile_CodeGen(

@markshannon markshannon Nov 11, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this (or the other functions) need to be an API function?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@iritkatriel iritkatriel Nov 11, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can _testinternalcapi import them otherwise?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@markshannon markshannon Nov 11, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so, that why it has "internal" in the name.
It defines Py_BUILD_CORE_BUILTIN

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@iritkatriel iritkatriel Nov 11, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't seem to work:

[ERROR] _testinternalcapi failed to import: dlopen(/Users/iritkatriel/src/cpython-654/build/lib.macosx-13.0-x86_64-3.12-pydebug/_testinternalcapi.cpython-312d-darwin.so, 0x0002): symbol not found in flat namespace '__PyCompile_CodeGen'

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@iritkatriel iritkatriel Nov 14, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll merge this PR now, it's getting into merge conflicts all the time. We can remove these from the API later if it's possible.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Lib/test/support/bytecode_helper.py Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Python/compile.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
Comment thread Python/compile.c Resolved

Uh oh!

There was an error while loading. Please reload this page.
iritkatriel added 3 commits November 11, 2022 13:34
@iritkatriel
compareInstructions --> assertInstructionsMatch
4453129
@iritkatriel
new_compiler
ad1ef54
@iritkatriel
revert change to cfg_to_instructions
4f51307
@iritkatriel
iritkatriel requested a review from brandtbucher as a code owner November 11, 2022 14:40
iritkatriel added 2 commits November 11, 2022 15:37
@iritkatriel
static
3657eca
@iritkatriel
Merge remote-tracking branch 'upstream/main' into expose_codegen
9cf20ef
@iritkatriel
iritkatriel merged commit a3ac923 into python:main Nov 14, 2022
@bedevere-bot bedevere-bot removed the awaiting core review label Nov 14, 2022
@thatbirdguythatuknownot thatbirdguythatuknownot mentioned this pull request Jun 19, 2023
syntactical __future__ import barry_as_FLUFL does not work in the REPL anymore #105908
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@markshannon markshannon markshannon left review comments
@brandtbucher brandtbucher Awaiting requested review from brandtbucher

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@iritkatriel iritkatriel

Labels

interpreter-core (Objects, Python, Grammar, and Parser dirs) skip news tests Tests in the Lib/test dir

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@iritkatriel @bedevere-bot @markshannon
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.