GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-114809: Support fat builds on macOS with the experimental JIT by ronaldoussoren · Pull Request #115742 · python/cpython

Conversation

@ronaldoussoren

@ronaldoussoren ronaldoussoren commented Feb 20, 2024 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Updates to the build system to support fat builds (--enable-universalsdk) on macOS with the experimental JIT.

Current status: "it compiles, therefore it works"

  • Issue: JIT & macOS fat builds #114809

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@ronaldoussoren
First draft of supporting fat builds on macOS with the experimental JIT
efd5863
@bedevere-app bedevere-app Bot mentioned this pull request Feb 20, 2024
JIT & macOS fat builds #114809
Closed
ronaldoussoren
ronaldoussoren commented Feb 20, 2024
View reviewed changes
Comment thread Tools/jit/_targets.py
o = tempdir / f"{opname}.o"
args = [
f"--target={self.triple}",
"-isysroot", "/Users/ronald/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk",

@ronaldoussoren ronaldoussoren Feb 20, 2024

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be removed in the final version, and likely needs a different issue and patch: I installed llvm using a download on llvm's GitHub, and use Xcode to build python (not Command Line Tools). Without this addition "clang-16" cannot find the SDK to use.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Tools/jit/_targets.py
):
return
stencil_groups = asyncio.run(self._build_stencils())
stencil_groups = ASYNCIO_RUNNER.run(self._build_stencils())

@ronaldoussoren ronaldoussoren Feb 20, 2024

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is the right way to enable running multiple builds, I'm a very light asyncio user. Without this change the second run fails due to running with a different runloop.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Tools/jit/build.py
args.target.build(pathlib.Path.cwd(), comment=comment)

else:
# Multiple triples specified, assume this is a macOS multiarchitecture build

@ronaldoussoren ronaldoussoren Feb 20, 2024

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bit is not ideal, multiple files are created and that likely breaks dependency handling in the Makefile.

It should be possible to publish everything in a single header file, but that requires more significant changes to the builder.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Tools/jit/build.py
with open("jit_stencils.h", "w") as fp:
for idx, target in enumerate(args.target):
cpu, _, _ = target.triple.partition("-")
fp.write(f"#{'if' if idx == 0 else 'elif'} defined(__{cpu}__)\n")

@ronaldoussoren ronaldoussoren Feb 20, 2024

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works for x86_64 and arm64, but might break for ancient systems running PPC and PPC64. I don't care for those.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@brandtbucher

brandtbucher commented Feb 20, 2024

Copy link
Copy Markdown
Member

Thanks for working on this! Juggling a few tasks right now, but I'll try this out later this week.

ronaldoussoren reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher

brandtbucher commented Feb 22, 2024

Copy link
Copy Markdown
Member

Something like this fixes CI for me, and generalizes this to all platforms:

diff --git a/Tools/jit/_targets.py b/Tools/jit/_targets.py
index 37b3e596bd..fd82eda697 100644
--- a/Tools/jit/_targets.py
+++ b/Tools/jit/_targets.py
@@ -37,6 +37,7 @@
 @dataclasses.dataclass
 class _Target(typing.Generic[_S, _R]):
     triple: str
+    condition: str
     _: dataclasses.KW_ONLY
     alignment: int = 1
     prefix: str = ""
@@ -108,7 +109,6 @@ async def _compile(
         o = tempdir / f"{opname}.o"
         args = [
             f"--target={self.triple}",
-            "-isysroot", "/Users/ronald/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk",
             "-DPy_BUILD_CORE",
             "-D_DEBUG" if self.debug else "-DNDEBUG",
             f"-D_JIT_OPCODE={opname}",
@@ -383,15 +383,21 @@ def _handle_relocation(
 def get_target(host: str) -> _COFF | _ELF | _MachO:
     """Build a _Target for the given host "triple" and options."""
     if re.fullmatch(r"aarch64-apple-darwin.*", host):
-        return _MachO(host, alignment=8, prefix="_")
+        condition = "defined(__aarch64__) && defined(__APPLE__)"
+        return _MachO(host, condition, alignment=8, prefix="_")
     if re.fullmatch(r"aarch64-.*-linux-gnu", host):
-        return _ELF(host, alignment=8)
+        condition = "defined(__aarch64__) && defined(__linux__)"
+        return _ELF(host, condition, alignment=8)
     if re.fullmatch(r"i686-pc-windows-msvc", host):
-        return _COFF(host, prefix="_")
+        condition = "defined(_M_IX86)"
+        return _COFF(host, condition, prefix="_")
     if re.fullmatch(r"x86_64-apple-darwin.*", host):
-        return _MachO(host, prefix="_")
+        condition = "defined(__x86_64__) && defined(__APPLE__)"
+        return _MachO(host, condition, prefix="_")
     if re.fullmatch(r"x86_64-pc-windows-msvc", host):
-        return _COFF(host)
+        condition = "defined(_M_X64)"
+        return _COFF(host, condition)
     if re.fullmatch(r"x86_64-.*-linux-gnu", host):
-        return _ELF(host)
+        condition = "defined(__x86_64__) && defined(__linux__)"
+        return _ELF(host, condition)
     raise ValueError(host)
diff --git a/Tools/jit/build.py b/Tools/jit/build.py
index 800d8e31b0..7872004a90 100644
--- a/Tools/jit/build.py
+++ b/Tools/jit/build.py
@@ -42,8 +42,7 @@
 
         with open("jit_stencils.h", "w") as fp:
             for idx, target in enumerate(args.target):
-                cpu, _, _ = target.triple.partition("-")
-                fp.write(f"#{'if' if idx == 0 else 'elif'} defined(__{cpu}__)\n")
+                fp.write(f"#{'if' if idx == 0 else 'elif'} {target.condition}\n")
                 fp.write(f'#   include "jit_stencils-{target.triple}.h"\n')
 
             fp.write("#else\n")
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher brandtbucher added the topic-JIT label May 31, 2024
@savannahostrowski

savannahostrowski commented Sep 26, 2024

Copy link
Copy Markdown
Member

@ronaldoussoren Hey there! Are you planning on carrying this forward? I have the branch pulled down locally and have patched things up with the main, etc. I'm happy to carry it forward if you're working on other things. Just let me know!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ned-deily

ned-deily commented Nov 19, 2024

Copy link
Copy Markdown
Member

While preparing for the 3.14.0a2 release, I realized too late that this issue had not yet been resolved thus preventing supporting the JIT in the python.org macOS installer build. My apologies for that. I'm looking at the issue now. If anyone does have any updates for it, it would be great to get them into the draft PR soon so we can re-target this for 3.14.0a3.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ned-deily
ned-deily self-requested a review November 19, 2024 17:20
@ronaldoussoren

ronaldoussoren commented Nov 22, 2024

Copy link
Copy Markdown
Contributor Author

@ronaldoussoren Hey there! Are you planning on carrying this forward? I have the branch pulled down locally and have patched things up with the main, etc. I'm happy to carry it forward if you're working on other things. Just let me know!

Feel free to carry it forward. I'm afraid I won't have time to work on this until at least the end of the year.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@savannahostrowski savannahostrowski mentioned this pull request Mar 26, 2025
GH-114809: Add support for macOS multi-arch builds with the JIT enabled #131751
Merged
@brandtbucher

brandtbucher commented May 19, 2025

Copy link
Copy Markdown
Member

The other PR was merged.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher brandtbucher closed this May 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ned-deily ned-deily Awaiting requested review from ned-deily
@brandtbucher brandtbucher Awaiting requested review from brandtbucher brandtbucher will be requested when the pull request is marked ready for review brandtbucher is a code owner
@diegorusso diegorusso Awaiting requested review from diegorusso diegorusso will be requested when the pull request is marked ready for review diegorusso is a code owner
@erlend-aasland erlend-aasland Awaiting requested review from erlend-aasland erlend-aasland will be requested when the pull request is marked ready for review erlend-aasland is a code owner
@corona10 corona10 Awaiting requested review from corona10 corona10 will be requested when the pull request is marked ready for review corona10 is a code owner
@AA-Turner AA-Turner Awaiting requested review from AA-Turner AA-Turner will be requested when the pull request is marked ready for review AA-Turner is a code owner
@emmatyping emmatyping Awaiting requested review from emmatyping emmatyping will be requested when the pull request is marked ready for review emmatyping is a code owner
@itamaro itamaro Awaiting requested review from itamaro itamaro will be requested when the pull request is marked ready for review itamaro is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

topic-JIT

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@ronaldoussoren @brandtbucher @savannahostrowski @ned-deily
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.