GeezerVison

Part of the PEP 730 work to add iOS support.

Adds:

• the configure/Makefile changes need build an iOS/tvOS/watchOS framework,
• a testbed Xcode project
• resources used by the framework builds.

Apologies in advance for the large PR; this is by far the largest single patch needed for PEP 730. Unfortunately, all these changes are interrelated - the resources are used by the build, and the makefile references the testbed. I couldn't work out a way to make the patch smaller. By way of partial defence - once you remove the autogenerated files (such as configure), and the new XCode project and stub binaries, the patch is a lot smaller 😅

Why tvOS and watchOS?

PEP 730 only proposes adding Tier 3 support for iOS. The tvOS and watchOS portions of this patch are strictly optional. I've included those portions of the patch because they're mostly symmetrical with the iOS changes, and it saves me maintaining these patches externally. However, I'm definitely aware of the project-level optics of a tvOS and watchOS folder appearing in the CPython repo. Unfortunately, the platforms are just different enough that I felt they needed their own standalone folders.

If the existence of the tvOS and watchOS folders (and/or the references in the configure script) are an impediment to merging, I'll gladly remove them.

Differences between macOS and iOS Frameworks

iOS frameworks are slightly different to macOS frameworks.

• macOS frameworks are "multi-version", as they are designed to support having multiple framework versions in a single Framework folder. iOS frameworks can't be shared between apps, so they don't support "multi-version" format.
• The metadata required by Info.plist is slightly different.
• The install name for the library in the framework must be based on an @rpath, rather than an absolute path, as the install path for the framework will be app-specific.
• iOS frameworks can only include binary .dylib artefacts and headers; and the headers won't be copied into the final distribution binary. As a result, the standard library and support binaries cannot be distributed inside the framework - they must be distributed parallel to the Framework folder. To accomodate this, the make install target for iOS will generate a file structure like the following:

--enable-framework location

• bin
• include symlink to -> Python.framework/Headers
• lib
  • python3.X
    • ...
• Python.framework
  • Headers
    • ...
  • Python (the dylib binary)
  • Info.plist

The task of copying the standard library into an appropriate location in the final app bundle is left as a task for the developer using the framework.

Summary of changes

configure changes

• Adds a RESSRCDIR variable to allow sharing of macOS and iOS Makefile steps
• Adds an INSTALLTARGETS variable so that iOS framework builds don't install binaries, manfiles, etc.
• Adds a PYTHONFRAMEWORKINSTALLNAMEPREFIX variable; this is used as the install name for the library, allowing iOS frameworks to specify an @rpath-based name
• Evaluates MACHDEP earlier in the configure process so that ac_sys_system is available
• Modifies _PYTHON_HOST_PLATFORM evaluation so that it includes the OS, ABI and architecture (e.g., ios-iphoneos-arm64 or ios-iphonesimulator-x86_64). This is used as the sysconfig identifier,
• Adds a IOS_DEPLOYMENT_TARGET variable, tracking the iOS minimum API version.
• Differentiates between SOABI_PLATFORM and PLATFORM_TRIPLET. SOABI_PLATFORM is used in binary module names, and includes the ABI, but not the OS or CPU architecture (e.g., math.cpython-313-iphonesimulator.dylib). PLATFORM_TRIPLET is used as the multiarch value, and contains the ABI and architecture (e.g., iphoneos-arm64)
• Skips the automated detection of a couple of system functions where autoconf gets the answer wrong - autoconf only checks whether a symbol can be linked, and doesn't use header files; iOS headers raise warnings when some symbols are used. There are also methods that exist and can be compiled, but raise errors if used at runtime.
• Automatically skips the configuration of ac_cv_file__dev_ptmx and ac_cv_file__dev_ptc, which are usually mandatory command line arguments for cross-platform builds. iOS is a reliable cross-build target, so we know these aren't available.
• Presumptively adds the 3 PEP 730 platforms to the PY_SUPPORT_TIER=3 list.
• Disables the standard lib modules that aren't available on iOS

Makefile changes

• Removes the use of the deprecated -Wl,-single_module flag on macOS
• Modifies the macOS framework builds to allow for iOS frameworks as well, using the RESSRCDIR, INSTALLTARGETS, and PYTHONFRAMEWORKINSTALLNAMEPREFIX variables.
• Adds a testiOS target to build and run the XCode testbed project, and extract the test results.

config.sub changes

The config.sub script has been updated to the 2024-01-01 version; additional patches have been applied to support the -simulator suffix, and the arm64_32 CPU architecture.

The additional patches were submitted upstream to autoconf in August last year. The official 2024-01-01 update includes the portions of that patch that added support for tvOS and watchOS, but didn't include updates adding the -simulator suffix or the arm64_32 CPU architecture. Unfortunately, the autoconf submission process is a bit of a black hole - you email them patches and get no further correspondence, even when the patch is merged. I've re-submitted the suffix and arm64_32 architecture patches; in the meantime, they've been manually applied here.

Support files

• Adds folders to store iOS/tvOS/watchOS build resources and documentation.

• Adds template plist files for iOS/tvOS/watchOS frameworks.

• Adds stub binaries that wrap compiler tooling. Xcode doesn't expose explicit compilers for iOS/tvOS/watchOS; instead, it uses an xcrun script that resolves to a full compiler path (e.g., xcrun --sdk iphoneos clang to get the clang for an iPhone device). However, using this script poses 2 problems:

  • The output of xcrun includes paths that are then machine specific, resulting in a sysconfig module that cannot be shared between users
  • It results in CC/CPP/LD/AR definitions that include spaces. There is a lot of C ecosystem tooling (including distutils and setuptools) that assumes that you can split a command line at the first space to get the path to the compiler executable; this isn't the case when using xcrun.

  So - to avoid these problems, the Resources/bin folders for each platform includes shell script wrappers around the tools needed by configure, named using the scheme that autoconf expects by default. These scripts are relocatable, and will always resolve to the appropriate local system paths. By including these scripts in the "installed" bin folder, the contents of the sysconfig module becomes useful for end-users to compile their own modules.

Other changes

• Adds an iOS testbed Xcode project. This test project uses the XCTest framework to expose a single test - that test instantiates a embedded Python interpreter, and runs the equivalent of python -m test. If the CPython test suite passes, the single XCTest passes; otherwise, the XCTest fails. The test suite is configured to run on an "iPhone SE (3rd gen)" simulator - this is a stable device identifier that isn't subject to annual device refresh cycles.
• There's no testbed project for tvOS or watchOS at this time, as they're not being targeted for Tier 3.
• The iOS/tvOS/watchOS folders will eventually contain READMEs detailing how to use these changes. I've omitted the README from this PR to keep the patch smaller and focussed; also the build still won't actually work until there's a few more patches in place.

• Issue: Add support for iOS as a target platform #114099