gh-114099 - Add iOS testbed, plus Makefile target to invoke it.

[freakboy3742]

Part of the PEP 730 work to add iOS support.

This PR adds an iOS testbed project, plus the Makefile target to support running it from the command line. It is the last of the code that was originally submitted as #115063.

The bulk of the PR is an iOS Xcode project; open iOS/testbed/iOSTestbed.xcodeproj from the command line will open the project in Xcode. The project itself is a bare stub app with no UI, and an XCTest test suite that contains 1 test - "run the CPython test suite"; the XCTest passes or fails depending on the overall success or failure of the CPython test suite.

Most of the project files are as generated by the new project wizard; differences of note include:

• iOS/testbed/iOSTestbed/main.m marks some signals as ignored that the CPython test suite exercises, but iOS doesn't like to be unhandled.
• iOS/testbed/iOSTestbedTests/iOSTestbedTests.m is an Xcode test suite with a single test that initialises a CPython interpreter and runs the test module. Command line options need to be passed in by modifying the argv array at the start of the file, as there's no real analog for a test suite command line (at least, not that I've found)
• Build scripts have been added to post-process the standard library into "framework" form.
• The build setting for Enable Testability has been set to Yes; this prevents stripping of binaries in release configurations. This is required because an iOS app may not link against symbols, but it will use them.
• The build setting for User Script Sandboxing has been set to No. This allows the post-processing scripts to modify and sign framework binaries.

There are 2 other changes:

• An extra dependency was added to the frameworkinstallmobileheaders target; I discovered that parallel make install builds would fail because it would try to correct the header install before actually installing the headers.
• The configure script has been modified to use the iOS wrapper binaries rather than gcc as a default compiler when CC et al aren't defined. This makes configure easier to invoke (as you just need to have the path correct), and makes the failure mode more predictable (as an attempt to compile with gcc will get quite far along before failing). Manually specifying overrides for CC et al remains available as an option.

Running the test suite

As of this PR, it is possible to build CPython as a framework, and start the test suite.

The new Makefile target is make testios. This target will run the test suite on an iPhone SE 3rd edition; this is a simulator that is reliably available, as it's a model that doesn't get updated all that often.

Running the testbed requires that an iOS framework build for a simulator target has been compiled and installed into the stub Python.xcframework folder that the testbed project contains. The minimum invocation for running the test suite is:

export PATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin" 
./configure --host=arm64-apple-ios12.0-simulator --build=arm64-apple-darwin --with-build-python=/path/to/python3.13 --enable-framework
make clean
make -j4 all
make testios

(adjusting as necessary to point to a appropriate build-python). A cold start of the iOS simulator takes 2-3 minutes, and for most of that time, the simulator appears to be doing nothing. Don't be alarmed when the console output for the test run says Testing started and then appears to do nothing for a long time. If you're running the test suite from inside Xcode, it's a lot faster once the simulator is warm.

However, if you invoke this target, the test suite will currently fail because it can't find binary modules at runtime. Correcting this will be the subject of the next PR.

• Issue: Add support for iOS as a target platform #114099

[sobolevn]

Super excited for this feature 🎉
Thank you!

[sobolevn]

Suggested change

	.PHONY: testiOS
	testiOS:
	.PHONY: test-iOS
	test-iOS:

?

[freakboy3742]

My thinking here was to be consistent with testuniversal, cleantest, buildbottest et al. However, I'm happy to change this name if there's a general preference for test-iOS over testiOS.

[mhsmith]

All existing makefile targets are lower-case, even acronyms like multissltest.

[freakboy3742]

I guess that's true... although I suspect I'd twitch every time I need to spell it with capitalization that breaks the branding guide 😀

[freakboy3742]

I've updated this to testios

[mhsmith]

I think this entire block is redundant, because this is already the default location that will be derived from config.home. In which case, the whole chain that passes the Python version from the configure script can be removed as well.

[freakboy3742]

Agreed this block isn't needed. The testbed project still needs a version number, but I guess it's not critical that this version number be the same as the Python version, as it's not going to be a released app, and there are other details in the logs that identify the Python version, so I guess we can drop the configuration portion as well.

[mhsmith]

The curses and tkinter tests will be skipped automatically due to their stdlib modules not being built, so there's no need to exclude them here.

[ned-deily]

Overall, it looks pretty good to me. I like the adding of the default tool stub selections into configure. I might have more comments after the follow-on PRs to get a fully working import system and the test suite can run.

One thing that I would like you to give some thought to, if you haven't already, is to make it possible to run make testiOS when the build directory is not the source directory. As a perhaps unwritten goal, we try to support multiple builds with different configure args from one source directory with the ultimate goal of a read-only source directory. Requiring the writable testbed directory to be in the source directory hinders those goals. Even something as simple as having the make testiOS rule copy the iOS/testbed directory from the source directory to the build directory would be fine. If that presents formidable obstacles, we wouldn't necessarily need to solve it immediately but I'd like your opinion on it.

[freakboy3742]

One thing that I would like you to give some thought to, if you haven't already, is to make it possible to run make testiOS when the build directory is not the source directory. As a perhaps unwritten goal, we try to support multiple builds with different configure args from one source directory with the ultimate goal of a read-only source directory.

That definitely makes sense as a goal; I think I've addressed that goal with my most recent commit.

To make this work, I've slightly modified the configure script. Previously, --enable-framework on iOS required the specification of an install location. In this version, a bare --enable-framework is now legal, and will create an "installed" version of the iOS framework in iOS/Frameworks/arm64-simulator (or similar). Specifying an explicit path will still work if you want.

Then, the testios target has install as a dependency; before running the test, it copies the testbed project into a timestamped copy in the ./build folder, and copies the "installed" framework from wherever it was constructed into the build version of the testbed. The project is then executed as before.

This means each testbed run is completely isolated from past runs; it also means there will be multiple copies of the testbed project and the full "installed" iOS framework - but most of them will be in the build folder, so they're easily cleaned up.

I've retained the instructions to purge the "source" version of the testbed as part of the clean target, in case anyone wants to use the older approach of "in tree" testbed builds - my feeling is that an in-tree build will be easier to manage for day-to-day development than the copied version in the build folder.

Does that approach makes sense? Is there an alternative approach that I haven't considered?

[ned-deily]

Thanks for addressing the previous comments. It's clear that, primarily due to the constraints of using an Xcode-provided simulator and of the necessary iOS framework structure, we are doing a bit of shoehorning here with a mixture of a Unix-style autoconf build and an Xcode project. But, if we can resolve the noted issues, I think this should be a reasonable compromise between the two styles.

[mhsmith]

I encountered this on Android as well. It turns out Python normally ignores these signals itself, but this is disabled by default with an isolated config. This can be fixed by adding the following line to the initialization code:

config.install_signal_handlers = 1;

Then it should be possible to remove these manual ignores.

[freakboy3742]

@ned-deily I've pushed an update that I think addresses all the issues you've flagged with the install dependency build location, and custom framework names; plus the signal configuration issue flagged by @mhsmith.

There's one additional piece - in #115774, @pablogsal added a new test module (_testexternalinspection.c) that was breaking the iOS build. AFAICT, iOS doesn't have a libproc.h, and the methods that are being appear to be Darwin kernel specific, but not exposed outside macOS. I've added extra pre-processing to make the test compile and pass.

[pablogsal]

There's one additional piece - in #115774, @pablogsal added a new test module (_testexternalinspection.c) that was breaking the iOS build. AFAICT, iOS doesn't have a libproc.h, and the methods that are being appear to be Darwin kernel specific, but not exposed outside macOS. I've added extra pre-processing to make the test compile and pass.

You can skip it in iOS if you want or add something to configure so it's not compiled in iOS

[ned-deily]

Thanks for resolving the comments nicely! It looks good to me now, with the caveat that, as advertised, the tests don't yet run under the simulator pending the additional changes to make extension module loading work. Unless someone has an objection, I think we can merge this PR and move on to the next.

[freakboy3742]

Thanks for resolving the comments nicely! It looks good to me now, with the caveat that, as advertised, the tests don't yet run under the simulator pending the additional changes to make extension module loading work. Unless someone has an objection, I think we can merge this PR and move on to the next.

The good news is that the next PR in the chain adds the extension module loader, which means the test suite will run (although there are 4 failures for platform/sysconfig-related reasons). That PR is ready to go - I can push it as soon as this one lands. The PR after that one will resolve those 4 test failures.

[freakboy3742]

The good news is that the next PR in the chain adds the extension module loader, which means the test suite will run (although there are 4 failures for platform/sysconfig-related reasons). That PR is ready to go - I can push it as soon as this one lands. The PR after that one will resolve those 4 test failures.

Hrm... after rebasing with main, I'm getting a weird error I wasn't getting a couple of days ago. Investigating...

[ned-deily]

Hrm... after rebasing with main, I'm getting a weird error I wasn't getting a couple of days ago. Investigating...

Is this the fatal_error_exit + 13 (pylifecycle.c:2817) that shows up in the iOSTestbed crash reports?

[freakboy3742]

I did get that one - fixed it by rebuilding the host python. This one is a test failure the importlib loader tests that I wasn't seeing before...

[freakboy3742]

Ok - found the problem. It was a weird import cache issue that I hadn't seen previously for some reason. PR incoming.