Build self-contained Swift Android cross-compilation SDK artifactbundle

[marcprux]

The Android SDK that is currently being generated by CI creates a destination.json file that can be used for cross-compilation to Android with a command like:

swift build --destination destination.json

An example destination.json that targets Android aarch64 from a Linux x86_64 host might look like:

{
  "version": 1,
  "target": "aarch64-unknown-linux-android24",
  "toolchain-bin-dir": "/home/runner/work/swift-android-sdk/swift-android-sdk/sdk-config/swift-5.10-RELEASE-ubuntu22.04/usr/bin",
  "sdk": "/usr/local/lib/android/sdk/ndk/26.2.11394342/toolchains/llvm/prebuilt/linux-x86_64/sysroot",
  "extra-cc-flags": [ "-fPIC", "-I/home/runner/work/swift-android-sdk/swift-android-sdk/sdk-config/swift-release-android-aarch64-24-sdk/usr/include" ],
  "extra-swiftc-flags": [
    "-resource-dir", "/home/runner/work/swift-android-sdk/swift-android-sdk/sdk-config/swift-release-android-aarch64-24-sdk/usr/lib/swift",
    "-tools-directory", "/usr/local/lib/android/sdk/ndk/26.2.11394342/toolchains/llvm/prebuilt/linux-x86_64/bin",
    "-Xcc", "-I/home/runner/work/swift-android-sdk/swift-android-sdk/sdk-config/swift-release-android-aarch64-24-sdk/usr/include",
    "-L/home/runner/work/swift-android-sdk/swift-android-sdk/sdk-config/swift-release-android-aarch64-24-sdk/usr/lib"
  ],
  "extra-cpp-flags": [ "-lstdc++" ]
}

One shortcoming of the destination.json format is that it relies on hardcoded absolute paths for the various tools, which means that it would need to be created separately for each host installation.

The SE-0387 – Swift SDKs for Cross-Compilation proposal, which has been accepted and implemented in Swift 5.10 and higher, allows for packaging all the SDK components into a single .artifactbundle which can be installed and managed using the swift sdk (or swift experimental-sdk for 5.10) command.

An SDK can be installed by manually unpacking an artifactbundle zip file, or can be installed like:

swift experimental-sdk install https://github.com/swift-android-sdk/swift-android-sdk/releases/download/5.10.1/swift-5.10.1-RELEASE_android-24-sdk.artifactbundle

Once installed, the SDKs can be listed:

zap ~ % swift experimental-sdk list
5.10-RELEASE_ubuntu_jammy_aarch64
5.10.1-RELEASE_android_24_aarch64
5.10.1-RELEASE_android_24_armv7
5.10.1-RELEASE_android_24_x86_64

After that, cross-compiling a Swift package is a simple matter of running:

swift build --experimental-swift-sdk 5.10.1-RELEASE_android_24_x86_64

Creating an artifactbundle would enable us to bundle the host toolchain, the Android NDK (or a subset thereof), and the Swift Android SDK for each architecture all together in a single artifact.

A 5.10.1-RELEASE_android_24.artifactbundle/info.json on macOS might look something like this:

{
  "schemaVersion": "1.0",
  "artifacts": {
    "5.10.1-RELEASE_android_24_x86_64": {
      "version": "0.0.1",
      "type": "swiftSDK",
      "variants": [
        {
          "supportedTriples": ["x86_64-apple-macosx", "arm64-apple-macosx"],
          "path": "5.10.1-RELEASE_android_24/x86_64-unknown-linux-android24"
        }
      ]
    },
    "5.10.1-RELEASE_android_24_aarch64": {
      "version": "0.0.1",
      "type": "swiftSDK",
      "variants": [
        {
          "supportedTriples": ["x86_64-apple-macosx", "arm64-apple-macosx"],
          "path": "5.10.1-RELEASE_android_24/aarch64-unknown-linux-android24"
        }
      ]
    },
    "5.10.1-RELEASE_android_24_armv7": {
      "version": "0.0.1",
      "type": "swiftSDK",
      "variants": [
        {
          "supportedTriples": ["x86_64-apple-macosx", "arm64-apple-macosx"],
          "path": "5.10.1-RELEASE_android_24/armv7-unknown-linux-android24"
        }
      ]
    }
  }
}

And a 5.10.1-RELEASE_android_24.artifactbundle/5.10.1-RELEASE_android_24/x86_64-unknown-linux-android24/toolset.json file might look like:

{
  "schemaVersion" : "1.0",
  "rootPath": "swift.xctoolchain/usr/bin",
  "swiftCompiler" : {
    "path": "swift.xctoolchain/usr/bin/swiftc",
    "extraCLIOptions" : [
      "-resource-dir", "../../../../swift-android.sdk/usr/lib/swift",
      "-L../../../../swift-android.sdk/usr/lib/x86_64-linux-android"
    ]
  },
  "cCompiler" : {
    "path": "../../../../android-ndk-darwin-x86_64/bin/clang",
    "extraCLIOptions" : [
      "-fPIC"
    ]
  },
  "cxxCompiler" : {
    "path": "../../../../android-ndk-darwin-x86_64/bin/clang++",
    "extraCLIOptions" : [
      "-lstdc++"
    ]
  },
  "linker" : {
    "path" : "../../../../android-ndk-darwin-x86_64/bin/ld.lld"
  },
  "librarian" : {
    "path" : "../../../../android-ndk-darwin-x86_64/bin/llvm-ar"
  },
  "debugger" : {
    "path" : "../../../../android-ndk-darwin-x86_64/bin/lldb"
  },
  "testRunner": {
    "path" : "../../../../scripts/run-swift-tests-on-emulator.sh"
  }
}

I've gotten an approximation of this setup working locally, and it is able to build various packages (swift-crypto, swift-collections, etc.). I was thinking that we could add a job to the CI that takes the outputs from the individual per-architecture toolchain builds and assembles them into artifactbundles that could then be surfaced as releases.

One example of such an assembly script can be found at static-linux/scripts/build.sh. Eventually, we might contribute it as a "recipe" in the swift-sdk-generator repo (e.g., LinuxRecipe.swift and WebAssemblyRecipe.swift), but a simple shell script is likely the easiest way to start.

[finagolfin]

Sure, that is the "SDK bundle" I've been mentioning in the doc for the last year, just haven't put it together yet. 😉 I've been busy this summer with making sure Swift 6+ keeps working well with the new Android overlay, but once we get the Foundation rewrite in Swift 6 ported to Android, the SDK bundle is next. If you have time and want to submit a pull for the CI to generate the SDK bundle, go for it.

I only would not want to distribute a host toolchain or any portion of the NDK (the NDK comes with a license barring redistribution, though its never been enforced AFAIK) as part of the SDK bundle, only distributing the Android SDK and config files you highlight instead.

[finagolfin]

Looking through the SDK bundle spec, I see no way to specify different toolsets for different hosts, even though the spec says it supports running the same bundle on different hosts, which I've reported at swiftlang/swift-package-manager#8029.

There are a few ways we could try to work around this:

1. ship multiple separate SDK bundles, one for each host platform, with only the toolset for that host
2. try to use only the toolset executables from the host toolchain, eg don't use the NDK clang instead when linking, which would require passing more compiler flags in and still may not work
3. don't ship an SDK bundle but simply a multi-arch SDK that contains all supported host toolsets, then have users manually change the toolset config file for their host

I'm leaning towards the third option, as the toolset executables are a fraction of the size of the full multi-arch SDK.

Maybe once the linked issue with SDK bundles is fixed, we can support those.

[finagolfin]

Hmm, I may have got 2. working, will test it on the CI soon.

[finagolfin]

I've merged a pull to create SDK bundles, #180, the daily CI will start producing them in a couple hours.

I ended up including the Android NDK sysroot and LLVM libcompiler-rt libraries alone from the NDK, as I believe those are all completely open source. I also went with 2. above, ie not including any toolset executables from the NDK in this Swift 6 Android SDK bundle, which I will release soon.