GeezerVison

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

All commit authors signed the Contributor License Agreement.

Documentation build overview

📚 cpython-previews | 🛠️ Build #33118577 | 📁 Comparing 376224c against main (bb5e41e)

  🔍 Preview build  

Apart from the more minor comments below, my main question is how this would integrate into cibuildwheel.

I agree that cibuildwheel would be an obvious consumer for this sort of feature - but other than potentially consistency in language or option names, does the usage in cibuildwheel require any special consideration in the design here?

The Android (and potentially iOS) runners have arguments; the cibuildwheel interface may choose to provide a way to leverage that option to expose the capability. In some sense, that option already exists in the form of the TEST_RUNTIME setting.

To my mind, the feature analog here is really with Docker, not cibuildwheel - Docker is the "sandboxed test runner" from which files need to be exfiltrated; cibuildwheel calls docker, and a cibuildwheel user may need some of those files.

@freakboy3742: This will probably come up on iOS at some point; what do you think? I guess it would be easier on iOS because the simulator doesn't have an isolated filesystem, but the command-line option in the testbed could be the same. Let's agree an approach and then discuss it with the cibuildwheel developers before we continue on this PR.

Agreed that it could be useful, and it would make sense to attempt to maintain parity between the iOS and Android testbed runners parity wherever we can.

If we go this way, we can remove the --output-dir option from this PR. I would also rename --pull to --output, both for consistency with the cibuildwheel option name and to focus on the intent rather than the mechanism.

Fully aware this is a bike shed - but I'm not convinced by this; To me, --output is a flag you use to say where you want a tool to put its output, not a path to the output to be preserved.

AFAICT, it makes sense to preserve two flags - where the content is coming from, and where it's going to; with the latter being --output. --pull doesn't feel quite right to me... but ultimately it is a bikeshed, as long as the flag makes some sort of sense.

There would then be an obvious mapping to TEST_OUTPUTS et all from cibuildwheel, but that's a "plumbing" question, not a design question for this feature.

For further discussion about the integration in cibuildwheel, I created pypa/cibuildwheel#2899.

AFAICT, it makes sense to preserve two flags - where the content is coming from, and where it's going to; with the latter being --output

I think we need the --output-dir (or only --output) option for cibuildwheel because multiple wheels can be built at once in a single cibuildwheel call and thus the tests also run multiple times. If the output were always placed in the root directory, the files would overwrite each other. Therefore, there should be a separate folder for each test call, which could be named like the build identifier (e.g., cp314-android_arm64_v8a).

--pull doesn't feel quite right to me... but ultimately it is a bikeshed, as long as the flag makes some sort of sense.

I borrowed the name --pull from the adb pull command (even if we are not using this command). This makes some sense in an Android context, but not if you want to unify it with iOS. Alternatively, it could be called --copy-back or just --copy.

I think we need the --output-dir (or only --output) option for cibuildwheel because multiple wheels can be built at once in a single cibuildwheel call and thus the tests also run multiple times.

I agree the flag is needed; I'd lean to --output for simplicity.

--pull doesn't feel quite right to me... but ultimately it is a bikeshed, as long as the flag makes some sort of sense.

I borrowed the name --pull from the adb pull command (even if we are not using this command). This makes some sense in an Android context, but not if you want to unify it with iOS. Alternatively, it could be called --copy-back or just --copy.

I don't know that I have a better name than --pull - so I'm inclined to just go with that.

@mhsmith I've addressed all the points you mentioned, so this PR is ready for another review.

I agree the flag is needed; I'd lean to --output for simplicity.

After thinking a bit more about the parameter name --output or --output-dir, I would still stick with --output-dir, because other tools like cibuildwheel also use --output-dir for a similar parameter.

This PR is stale because it has been open for 90 days with no activity.

Sorry for the slow response. I'll be busy for the next few weeks preparing my projects for Python 3.15, and then I'll have some time to come back to this.

Python 3.13 is about to go into security-only status, and even before that, the 2-month release cycle has been an inconvenience for cibuildwheel. So I'm also thinking of splitting out the testbed from CPython into a separate package so it's easier to update, and easier to use in other contexts (#158018). That should be done before adding any new features.