Fetch the Swift toolchain as part of the CI job

[philwo]

I'm trying to remove parts from our Docker containers that are only used by one or few projects on CI.

Instead, we can download & extract them during the shell_commands step of the projects. This also allows the projects themselves to control the exact versioning of their dependencies and makes it easier to test upgrades.

[philwo]

I don't really understand why the builds are failing with this error, independent of the Swift version:

(15:22:07) FATAL: bazel crashed due to an internal error. Printing stack trace:
java.lang.RuntimeException: Unrecoverable error while evaluating node 'ConfiguredTargetKey{label=//examples/xplatform/c_from_swift:c_from_swift, config=BuildConfigurationValue.Key[58765066c537a2addebe865dea2674b9eb2786d3980bb8070f08dabcf685a2a8]}' (requested by nodes )
	at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:563)
	at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:398)
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)
Caused by: net.starlark.java.eval.Starlark$UncheckedEvalException: java.lang.IllegalArgumentException: swift_counter.modulewrap.o (Starlark stack: [_swift_binary_impl@/workdir/swift/internal/swift_binary_test.bzl:323:61, _swift_linking_rule_impl@/workdir/swift/internal/swift_binary_test.bzl:229:50, register_link_binary_action@/workdir/swift/internal/linking.bzl:255:26, link@<builtin>])
	at net.starlark.java.eval.Starlark.fastcall(Starlark.java:610)
	at net.starlark.java.eval.Eval.evalCall(Eval.java:642)
	at net.starlark.java.eval.Eval.eval(Eval.java:460)
	at net.starlark.java.eval.Eval.execReturn(Eval.java:242)
	at net.starlark.java.eval.Eval.exec(Eval.java:279)
	at net.starlark.java.eval.Eval.execStatements(Eval.java:81)
	at net.starlark.java.eval.Eval.execFunctionBody(Eval.java:65)
	at net.starlark.java.eval.StarlarkFunction.fastcall(StarlarkFunction.java:160)
	at net.starlark.java.eval.Starlark.fastcall(Starlark.java:606)
	at net.starlark.java.eval.Eval.evalCall(Eval.java:642)
	at net.starlark.java.eval.Eval.eval(Eval.java:460)
	at net.starlark.java.eval.Eval.execAssignment(Eval.java:109)
	at net.starlark.java.eval.Eval.exec(Eval.java:261)
	at net.starlark.java.eval.Eval.execStatements(Eval.java:81)
	at net.starlark.java.eval.Eval.execFunctionBody(Eval.java:65)
	at net.starlark.java.eval.StarlarkFunction.fastcall(StarlarkFunction.java:160)
	at net.starlark.java.eval.Starlark.fastcall(Starlark.java:606)
	at net.starlark.java.eval.Eval.evalCall(Eval.java:642)
	at net.starlark.java.eval.Eval.eval(Eval.java:460)
	at net.starlark.java.eval.Eval.execAssignment(Eval.java:109)
	at net.starlark.java.eval.Eval.exec(Eval.java:261)
	at net.starlark.java.eval.Eval.execStatements(Eval.java:81)
	at net.starlark.java.eval.Eval.execFunctionBody(Eval.java:65)
	at net.starlark.java.eval.StarlarkFunction.fastcall(StarlarkFunction.java:160)
	at net.starlark.java.eval.Starlark.fastcall(Starlark.java:606)
	at com.google.devtools.build.lib.analysis.starlark.StarlarkRuleConfiguredTargetUtil.buildRule(StarlarkRuleConfiguredTargetUtil.java:130)
	at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createRule(ConfiguredTargetFactory.java:365)
	at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createConfiguredTarget(ConfiguredTargetFactory.java:194)
	at com.google.devtools.build.lib.skyframe.SkyframeBuildView.createConfiguredTarget(SkyframeBuildView.java:938)
	at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.createConfiguredTarget(ConfiguredTargetFunction.java:1018)
	at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.compute(ConfiguredTargetFunction.java:371)
	at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:477)
	... 7 more
Caused by: java.lang.IllegalArgumentException: swift_counter.modulewrap.o
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
	at com.google.devtools.build.lib.rules.cpp.CppLinkActionBuilder.addNonCodeInput(CppLinkActionBuilder.java:1334)
	at com.google.devtools.build.lib.rules.cpp.CppLinkActionBuilder.addNonCodeInputs(CppLinkActionBuilder.java:1320)
	at com.google.devtools.build.lib.rules.cpp.CppLinkActionBuilder.addLinkParams(CppLinkActionBuilder.java:1465)
	at com.google.devtools.build.lib.rules.cpp.CcLinkingHelper.createDynamicLinkAction(CcLinkingHelper.java:739)
	at com.google.devtools.build.lib.rules.cpp.CcLinkingHelper.createCcLinkActions(CcLinkingHelper.java:463)
	at com.google.devtools.build.lib.rules.cpp.CcLinkingHelper.link(CcLinkingHelper.java:372)
	at com.google.devtools.build.lib.rules.cpp.CcModule.link(CcModule.java:1989)
	at com.google.devtools.build.lib.bazel.rules.cpp.BazelCcModule.link(BazelCcModule.java:140)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at net.starlark.java.eval.MethodDescriptor.call(MethodDescriptor.java:135)
	at net.starlark.java.eval.BuiltinFunction.fastcall(BuiltinFunction.java:77)
	at net.starlark.java.eval.Starlark.fastcall(Starlark.java:606)
	... 38 more
bazel build failed with exit code 37

Will continue to investigate...

[keith]

I think a commit was merged ignoring the failing CI, so it's possible it introduce a bug

[philwo]

I can try to rebase on the last passing commit to check whether this PR works as intended.

Which Swift version would you prefer to use?

[keith]

newest release version (so 5.4.x) would be ideal, but realistically the difference in the minor releases above 5 should be inconsequential for CI here, so if you can get one of these working I'm happy to merge and potentially update later

[keith]

Potentially related https://forums.swift.org/t/android-crosscompilation-on-macos-swiftc-fails-due-missed-symbol-swift-addnewdsoimage/19210/3

[philwo]

That seems to work. I will be able to find the culprit that way and will let you know here :)

[philwo]

Results:

Up to commit b0ea825 your CI was green: https://buildkite.com/bazel/rules-swift-swift/builds/2733

The next commit 8cf3635 broke the build, but with a different error message and across all platforms: https://buildkite.com/bazel/rules-swift-swift/builds/2734#9d2420e3-d852-48e4-9dca-9c546fce5a82

The next commit 93a91f4 seems to have fixed it on macOS, but from that commit on your Linux builds are failing with the strange crash visible at HEAD, too: https://buildkite.com/bazel/rules-swift-swift/builds/2735#3b4e3037-2b9b-469f-8768-300479e885a3

[philwo]

I'll rebase this against master and then we can just submit this so that your CI works, then you can look at how to fix this breakage much more easily :)

[philwo]

Done. Feel free to merge.

[keith]

Thanks!