GeezerVison

This script simplifies the process of configuring, compiling and packaging an XCframework for an Apple platform. This is an analog of the scripts used to build Android and Emscripten artefacts. It is a precursor to adding GitHub Action builds for iOS, and producing iOS build artefacts.

At present, it only supports iOS, but it has been constructed so that it could be used on any Apple platform (including, potentially, a redistributable macOS XCframework).

The simplest usage of this script is:

  $ python Apple ci iOS

which will:

• Clean any pre-existing build artefacts
• Configure and make a Python that can be used for the build
• Configure and make a Python for each supported iOS architecture and ABI
• Combine the outputs of the builds from the previous step into a single
  XCframework, merging binaries into a "fat" binary if necessary
• Clone a copy of the testbed, configured to use the XCframework
• Construct a tarball containing the release artefacts
• Run the test suite using the generated XCframework.

This is the complete sequence that would be needed in CI to build and test a candidate release artefact. The output of the command will detect if it's running in a GitHub Actions and use output groups to make the log output easier to digest.

Each individual step can be invoked individually - there are commands to clean, configure-build, make-build, configure-host, make-host, package, and test.

There is also a build command that can be used to combine the configure and make steps for the build Python, an individual host, all hosts, or all builds.

One of the steps is to manage the download of binary artefacts; these artefacts are downloaded to the cross-build/downloads folder by default, but an alternate cache location can be provided.

There are three potentially controversial parts to this PR:

Moving iOS to Apple/iOS

As part of recognising that iOS is one of (potentially) many Apple platforms, and to prevent future proliferation of top-level directories for other Apple platforms, it migrates the iOS folder into a top level Apple folder. This change isn't strictly required, but if we don't make this change now, there is potential for a proliferation of directories if/when we add support for tvOS, watchOS, visionOS, macCatalyst. It also provides a convenient home for "cross platform" apple concerns, like the testbed script and the build script itself, as well as place for the existing content in the Mac directory to migrate to as part of a cleanup of macOS builds - something that @ned-deily has expressed an interest in doing ¹

Universal simulator binaries

This PR generates a build that includes x86_64 simulator binaries. The iOS simulator slice of the XCframework is effectively a "universal" build covering both x86_64 and ARM64; although it has to be compiled in 2 passes and binaries merged after the fact. It's possible to both compile and test x86_64 binaries on ARM64 machines (pass --simulator "iPhone 16e,arch=x86_64" to the ci or test target, or to the testbed script), although running the test suite in emulator mode currently causes some test failure related to build-details.json not being fully cross-platform aware.

x86_64 isn't a Tier 3 platform for Python due to the difficulties of commissioning a buildbot; but x86_64 is still a supported platform for Apple. If we're going to produce official binaries, it seems to me like we should still support x86_64.

If we choose not to support x86_64, then there will be some changes required to the build script and testbed, as the name of the simulator slice will change from ios_arm64_x86_64_simulator to ios_arm64_simulator. It would also allow removing the most complicated part of this script - the part that does the merging of binaries.

Backporting to 3.13 and 3.14

I've proposed this for backport to 3.13 and 3.14. This is primarily so that the buildbots can be migrated to use this script, and CI jobs can be added for iOS. The directory reorganization is a significant change to make in a backport, especially in the year-old 3.13 release; however, when I floated this idea at the CPython core team summit, @Yhg1s suggested ¹ that while it is a big change, the set of affected iOS users would be small, so the impact wasn't a huge concern.

NOTE: This PR currently contains a duplicate of the iOS/Resources/bin directory; this is needed because the location is hard-coded into the buildbots. Once the location change is confirmed, I can add the new path to the buildbot configuration, and remove the duplicate bin scripts.

• Issue: Add mechanism for building iOS XCframeworks #138171

📚 Documentation preview 📚: https://cpython-previews--138176.org.readthedocs.build/