GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GH-113464: Get LLVM from `cpython-bin-deps` on Windows by brandtbucher · Pull Request #133278 · python/cpython

Conversation

@brandtbucher

@brandtbucher brandtbucher commented May 2, 2025 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This makes LLVM easier to acquire for Windows JIT builds (like the one @zooba is going to do next week). We can now get it from https://github.com/python/cpython-bin-deps/tree/llvm.

I also cleaned up a couple things I tripped over while working on this:

  • Fix the regen target to work better with recent changes to how jit_stencils.h is built (and clean up after itself).
  • Don't touch jit_stencils.h unless it's actually changing, to avoid triggering a rebuild.
  • Issue: JIT Compilation #113464

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
brandtbucher added 2 commits May 1, 2025 17:20
@brandtbucher
Get LLVM from cpython-bin-deps, and clean things up a bit
d8cfc0a
@brandtbucher
blurb add
b57fe8b
@brandtbucher brandtbucher added OS-windows build The build process and cross-build topic-JIT labels May 2, 2025
@brandtbucher
brandtbucher requested a review from zooba May 2, 2025 00:27
@brandtbucher brandtbucher self-assigned this May 2, 2025
@brandtbucher
brandtbucher requested review from a team and savannahostrowski as code owners May 2, 2025 00:27
@bedevere-app bedevere-app Bot mentioned this pull request May 2, 2025
JIT Compilation #113464
Closed
@bedevere-app bedevere-app Bot added the awaiting core review label May 2, 2025
@zooba

zooba commented May 2, 2025

Copy link
Copy Markdown
Member

Looks great! Though so far, this is the part that makes me sad (watch the timestamps). I guess it's unavoidable though.

Fri, 02 May 2025 00:38:42 GMT
 Fetching external libraries...
Fri, 02 May 2025 00:38:42 GMT
 Fetching bzip2-1.0.8...
Fri, 02 May 2025 00:38:46 GMT
 Fetching mpdecimal-4.0.0...
Fri, 02 May 2025 00:38:47 GMT
 Fetching sqlite-3.49.1.0...
Fri, 02 May 2025 00:38:48 GMT
 Fetching xz-5.2.5...
Fri, 02 May 2025 00:38:49 GMT
 Fetching zlib-ng-2.2.4...
Fri, 02 May 2025 00:38:50 GMT
 Fetching external binaries...
Fri, 02 May 2025 00:38:50 GMT
 Fetching libffi-3.4.4...
Fri, 02 May 2025 00:38:51 GMT
 Fetching openssl-bin-3.0.16.2...
Fri, 02 May 2025 00:38:53 GMT
 Fetching tcltk-8.6.[15](https://github.com/python/cpython/actions/runs/14786163251/job/41515167379?pr=133278#step:4:16).0...
Fri, 02 May 2025 00:38:55 GMT
 Fetching llvm-19.1.7.0...
Fri, 02 May 2025 00:42:13 GMT
 Finished.

Just to confirm, these lines in the build output mean that it's working, yes? (I think the last one isn't from your script, but it's not an error, so that implies success)

  
  =============================================================
  JIT support for x86_64-pc-windows-msvc is still experimental!
             Please report any issues you encounter.           
  =============================================================
  
  Regenerate cases
  Generated sources are up to date
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@zooba

zooba commented May 2, 2025

Copy link
Copy Markdown
Member

Another possibility is to integrate the LLVM fetch into the script itself, so that we can check whether the files are up to date first and only download LLVM if it's needed (using/ignoring a separate install, as appropriate - I could see this being a case where we'd prefer to ignore it when we don't know which version it is)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Wulian233

Wulian233 commented May 2, 2025

Copy link
Copy Markdown
Contributor

Do we need to change 18 to 19 here to unify the build?

cpython/PCbuild/readme.txt

Lines 69 to 76 in a6ddd07

You can also use a specific version of clang-cl downloaded from
https://github.com/llvm/llvm-project/releases, e.g.
clang+llvm-18.1.8-x86_64-pc-windows-msvc.tar.xz.
Given you have extracted that to <my-clang-dir>, you can use it like so
build.bat --pgo "/p:PlatformToolset=ClangCL" "/p:LLVMInstallDir=<my-clang-dir>" "/p:LLVMToolsVersion=18"
Setting LLVMToolsVersion to the major version is enough, although you
can be specific and use 18.1.8 in the above example, too.

zooba reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher

brandtbucher commented May 2, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Do we need to change 18 to 19 here to unify the build?

Not sure what you mean by "unify the build", but that LLVM version is unrelated. That's for if you want to build all of CPython with LLVM; this change is just for generating the templates the JIT compiler uses, which has stricter version requirements and can be different than the compiler or version used to build the interpreter itself.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher

brandtbucher commented May 2, 2025

Copy link
Copy Markdown
Member Author

Looks great! Though so far, this is the part that makes me sad (watch the timestamps). I guess it's unavoidable though.

Yeah, it's a bummer that it takes a few minutes (maybe we could trim stuff we don't need from the hosted install, but I imagine most of the size is in the parts that we depend on). But if I understand correctly, it should download only once per machine that's doing Windows builds and remain cached, so it's not like Windows JIT core devs' daily build times just went up or anything.

Plus, we don't actually download it unless you're building the JIT, so most people should be unaffected (for now).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@zooba

zooba commented May 2, 2025

Copy link
Copy Markdown
Member

Plus, we don't actually download it unless you're building the JIT, so most people should be unaffected (for now).

Ah, yeah, I didn't catch that first time through.

And yeah, it'll be once per machine. I was mostly thinking about CI, which uses a fresh machine each time, and also CI that bypasses build.bat (msbuild pcbuild.proj is the typical way, and it's not unreasonable to do it when building releases).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher

brandtbucher commented May 2, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

And yeah, it'll be once per machine. I was mostly thinking about CI, which uses a fresh machine each time, and also CI that bypasses build.bat (msbuild pcbuild.proj is the typical way, and it's not unreasonable to do it when building releases).

Yeah. For the former we need to install it anyways. For the latter, they'll just need to make sure to set the right options to omit the LLVM install when it's not needed if they want to avoid waiting.

(Another option could be to make the LLVM install default to "false" in get_externals.bat, and have build.bat enable it when needed. Not sure if that makes things better, or just more confusing.)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@brandtbucher
brandtbucher merged commit bfcbb28 into python:main May 2, 2025
@bedevere-app bedevere-app Bot removed the awaiting core review label May 2, 2025
Pranjal095 pushed a commit to Pranjal095/cpython that referenced this pull request Jul 12, 2025
@brandtbucher @Pranjal095
pythonGH-113464: Get LLVM from cpython-bin-deps on Windows (pythonGH-…
b1ce102
…133278)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zooba zooba Awaiting requested review from zooba
@savannahostrowski savannahostrowski Awaiting requested review from savannahostrowski

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@brandtbucher brandtbucher

Labels

build The build process and cross-build OS-windows topic-JIT

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@brandtbucher @zooba @Wulian233
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.