Following instructions for a custom kotlinc distribution causes builds to fail #1019
Assignees
Comments
|
Git bisecting suggests 113e91b was the cause of this issue. That's not a huge surprise, since it updates Kotlin to 1.8 |
|
related: #998 |
6 tasks
BenHenning
added a commit
to oppia/oppia-android
that referenced
this issue
Jun 13, 2024
## Explanation Fixes #5370 Fixes part of #59 This PR updates the project to use Bazel 6.5.0 instead of 4.0.0. Note that most of the changes done so far in addressing #59 are centered around the concept of simplifying the Bazel maintenance as much as possible so that it's not too much more difficult than Gradle by the time we fully remove Gradle support from the project. While Bazel will always require more effort, there are many things that can be done to narrow the gap. This is a major step in that process since Bazel 4.x required using a custom Android toolchain (https://github.com/oppia/oppia-bazel-tools) which is not at all user friendly. Plus, there are many compatibility and performance improvements in later versions of Bazel that we want to be able to incorporate within the broader Oppia Android project. Bazel 6.x was specifically chosen because: - Bazel 4.x was missing support for the new D8 version which made it impossible to upgrade past 29.0.2 build tools bazelbuild/bazel#13989. - Bazel 5.x had some additional compatibility issues with the D8 change, so we weren't able to use it, either: bazelbuild/bazel#15957. - Bazel 7.x (which wasn't released when this work was originally done) introduces new bzlmod support that causes some additional build headaches that can be figured out later. - Bazel 6.5.0 specifically was chosen since it's the latest 6.x version (as of this edit) and seems to work correctly with existing unit tests. Some other important details to note: - rules_kotlin 1.7.x is needed at a minimum for Bazel 5.x+ support. However, an additional fix was needed (bazelbuild/rules_kotlin#940) in order to fix a deviation in functionality that occurred starting in Bazel 5.x's java_plugin support which led to some file duplication in rules_kotlin (that was fortunately easy to fix). Unfortunately, this change wasn't backported to 1.7.x so this PR makes use of a custom patch to rules_kotlin 1.7.1 (https://github.com/oppia/rules_kotlin) that includes the needed change. We'll get this change properly once we can upgrade to 1.8.x, though that will also require updating Kotlin itself to 1.8.x due to bazelbuild/rules_kotlin#1019. - Bazel 6.x (maybe 5.x) requires at least build tools 30.0.0 since it completely removed support for the old D8 compat dexer. 32.0.0 was chosen in this PR as it's simply a newer, more up-to-date build tools (and removes D8 completely). With this upgrade to Bazel 6.x we'll be able to update the build tools version more often (so long as it doesn't introduce AGP incompatibilities since we can't upgrade Gradle). - As of Bazel 6.x, we're able to reenable Java header compilation and incremental dexing, both of which should have _significant_ performance improvements for incremental builds of the app (and in fact we will have build errors if we disable incremental dexing). - In CI, we opted to **not** support build tools 29.0.2 or old builds of the app. Instead, we'll rely on build tools failing for certain PRs as an indicator that those PRs will require an update (once this PR is merged) in order to have CI run correctly. This is a lot easier than trying to figure out how to support before/after changes with some fairly complex environment differences. - There are a bunch of version updates that were needed to support the minimum version of Kotlin for rules 1.7.x (1.6 I think) as well as JDK 11 (which I think was needed for Bazel 5.x), and these have largely been taken care of in previous PRs to this one (though the JDK 11 update in CI was done in this PR, along with wiki documentation updates to address #5370). One such case of a necessary version upgrade: google/dagger#2511. - There was a change needed for the databinding java_plugin declaration to specify that it generates an API (in order for it to be used correctly in builds). - rules_java needed to be updated to support the newer version of Bazel. - The desugaring hack needed for kotlinx-coroutines-core-jvm was removed since it's no longer needed with the build tools & Bazel upgrade introduced in this PR. - This includes one small change in third-party to change all single-export wrappers that don't have additional plugins being enabled to aliases instead. This is more semantically correct as the wrappers may lose information (which caused problems when investigating adding Jetpack Compose support in #5401). While this isn't directly required for the Bazel upgrade, this is the last PR needed for Jetpack Compose support so it's being added here for simplicity. - ``.bazelrc`` was updated to configure tools, tests, and builds to all use the remote JDK 11 available via Bazel rather than ever using the user's local JDK. This should improve build hermeticity and consistency across different user environments (see https://bazel.build/docs/bazel-and-java). - Setup docs were updated to remove setting up JDK 11 (or Java at all for Linux & Mac) now that the user no longer needs to install Java (see previous point) except for Windows. The Python instructions were also removed since Bazel 6.x includes fixes for Android tools that previously depended on Python 2.x. - CI was unchanged for Java setup since, as far as I can tell, it's still needed for sdkmanager. There was also some small cleanup in unit_tests.yml that I noticed when updating CI versions. ## Essential Checklist - [x] The PR title and explanation each start with "Fix #bugnum: " (If this PR fixes part of an issue, prefix the title with "Fix part of #bugnum: ...".) - [x] Any changes to [scripts/assets](https://github.com/oppia/oppia-android/tree/develop/scripts/assets) files have their rationale included in the PR explanation. - [x] The PR follows the [style guide](https://github.com/oppia/oppia-android/wiki/Coding-style-guide). - [x] The PR does not contain any unnecessary code changes from Android Studio ([reference](https://github.com/oppia/oppia-android/wiki/Guidance-on-submitting-a-PR#undo-unnecessary-changes)). - [x] The PR is made from a branch that's **not** called "develop" and is up-to-date with "develop". - [x] The PR is **assigned** to the appropriate reviewers ([reference](https://github.com/oppia/oppia-android/wiki/Guidance-on-submitting-a-PR#clarification-regarding-assignees-and-reviewers-section)). ## For UI-specific PRs only N/A -- This is a build infrastructure change. It shouldn't impact the end user experience. --------- Co-authored-by: Adhiambo Peres <[email protected]> Co-authored-by: Sean Lip <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Following the instructions in
rules_kotlin's readme, the following should be enough to use the Kotlin 1.6.21 compiler release:After setting the
compiler_release, builds fail with the following exception:INFO: Invocation ID: 80fa3576-d0ce-4f07-869b-b27a58543d5b INFO: Analyzed 4 targets (32 packages loaded, 331 targets configured). INFO: Found 4 targets... ERROR: /Volumes/Dev/src/github.pie.apple.com/bazel/apple_federation/examples/kotlin_temp/jvm/src/example/BUILD.bazel:3:15: KotlinCompile //jvm/src/example:example { kt: 1, java: 0, srcjars: 0 } for darwin_arm64 failed: (Exit 1): build failed: error executing command (from target //jvm/src/example:example) bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/io_bazel_rules_kotlin/src/main/kotlin/build '--flagfile=bazel-out/darwin_arm64-fastbuild/bin/jvm/src/example/example-kt.jar-0.params' warning: language version 1.8 is experimental, there are no backwards compatibility guarantees for new language and library features exception: java.lang.AbstractMethodError: Receiver class io.bazel.kotlin.plugin.jdeps.JdepsGenExtension does not define or inherit an implementation of the resolved method 'abstract org.jetbrains.kotlin.analyzer.AnalysisResult doAnalysis(com.intellij.openapi.project.Project, org.jetbrains.kotlin.descriptors.ModuleDescriptor, org.jetbrains.kotlin.context.ProjectContext, java.util.Collection, org.jetbrains.kotlin.resolve.BindingTrace, org.jetbrains.kotlin.container.ComponentProvider)' of interface org.jetbrains.kotlin.resolve.extensions.AnalysisHandlerExtension. at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:123) at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:99) at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:301) at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:55) at org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:113) at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:292) at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:102) at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli$default(KotlinToJVMBytecodeCompiler.kt:60) at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:172) at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:54) at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:91) at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:43) at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:93) at io.bazel.kotlin.compiler.BazelK2JVMCompiler.exec(BazelK2JVMCompiler.kt:32) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at io.bazel.kotlin.builder.toolchain.KotlinToolchain$KotlinCliToolInvoker.compile(KotlinToolchain.kt:213) at io.bazel.kotlin.builder.tasks.jvm.CompilationTaskKt$compileKotlin$1$2.invoke(CompilationTask.kt:415) at io.bazel.kotlin.builder.tasks.jvm.CompilationTaskKt$compileKotlin$1$2.invoke(CompilationTask.kt:415) at io.bazel.kotlin.builder.toolchain.CompilationTaskContext.executeCompilerTask(CompilationTaskContext.kt:126) at io.bazel.kotlin.builder.toolchain.CompilationTaskContext.executeCompilerTask$default(CompilationTaskContext.kt:118) at io.bazel.kotlin.builder.tasks.jvm.CompilationTaskKt.compileKotlin(CompilationTask.kt:415) at io.bazel.kotlin.builder.tasks.jvm.KotlinJvmTaskExecutor$execute$1$1$1$1.invoke(KotlinJvmTaskExecutor.kt:60) at io.bazel.kotlin.builder.tasks.jvm.KotlinJvmTaskExecutor$execute$1$1$1$1.invoke(KotlinJvmTaskExecutor.kt:58) at io.bazel.kotlin.builder.toolchain.CompilationTaskContext.execute(CompilationTaskContext.kt:153) at io.bazel.kotlin.builder.toolchain.CompilationTaskContext.execute(CompilationTaskContext.kt:145) at io.bazel.kotlin.builder.tasks.jvm.KotlinJvmTaskExecutor$execute$1.invoke(KotlinJvmTaskExecutor.kt:58) at io.bazel.kotlin.builder.tasks.jvm.KotlinJvmTaskExecutor$execute$1.invoke(KotlinJvmTaskExecutor.kt:54) at io.bazel.kotlin.builder.toolchain.CompilationTaskContext.execute(CompilationTaskContext.kt:153) at io.bazel.kotlin.builder.toolchain.CompilationTaskContext.execute(CompilationTaskContext.kt:145) at io.bazel.kotlin.builder.tasks.jvm.KotlinJvmTaskExecutor.execute(KotlinJvmTaskExecutor.kt:54) at io.bazel.kotlin.builder.tasks.KotlinBuilder.executeJvmTask(KotlinBuilder.kt:210) at io.bazel.kotlin.builder.tasks.KotlinBuilder.build(KotlinBuilder.kt:103) at io.bazel.kotlin.builder.tasks.CompileKotlin.invoke(CompileKotlin.kt:27) at io.bazel.worker.PersistentWorker$workTo$1.invoke(PersistentWorker.kt:97) at io.bazel.worker.PersistentWorker$workTo$1.invoke(PersistentWorker.kt:97) at io.bazel.worker.WorkerContext$TaskContext.resultOf(WorkerContext.kt:128) at io.bazel.worker.WorkerContext.doTask(WorkerContext.kt:156) at io.bazel.worker.PersistentWorker$start$1$1$producer$1$1.invoke$lambda$0(PersistentWorker.kt:70) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Aug 24, 2023 1:56:50 PM worker request 0 SEVERE: Compilation failure: compile phase failed: warning: language version 1.8 is experimental, there are no backwards compatibility guarantees for new language and library features exception: java.lang.AbstractMethodError: Receiver class io.bazel.kotlin.plugin.jdeps.JdepsGenExtension does not define or inherit an implementation of the resolved method 'abstract org.jetbrains.kotlin.analyzer.AnalysisResult doAnalysis(com.intellij.openapi.project.Project, org.jetbrains.kotlin.descriptors.ModuleDescriptor, org.jetbrains.kotlin.context.ProjectContext, java.util.Collection, org.jetbrains.kotlin.resolve.BindingTrace, org.jetbrains.kotlin.container.ComponentProvider)' of interface org.jetbrains.kotlin.resolve.extensions.AnalysisHandlerExtension. at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:123) at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:99) at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:301) at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:55) at org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:113) at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:292) at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:102) at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli$default(KotlinToJVMBytecodeCompiler.kt:60) at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:172) at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:54) at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:91) at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:43) at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:93) at io.bazel.kotlin.compiler.BazelK2JVMCompiler.exec(BazelK2JVMCompiler.kt:32) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at io.bazel.kotlin.builder.toolchain.KotlinToolchain$KotlinCliToolInvoker.compile(KotlinToolchain.kt:213) at io.bazel.kotlin.builder.tasks.jvm.CompilationTaskKt$compileKotlin$1$2.invoke(CompilationTask.kt:415) at io.bazel.kotlin.builder.tasks.jvm.CompilationTaskKt$compileKotlin$1$2.invoke(CompilationTask.kt:415) at io.bazel.kotlin.builder.toolchain.CompilationTaskContext.executeCompilerTask(CompilationTaskContext.kt:126) at io.bazel.kotlin.builder.toolchain.CompilationTaskContext.executeCompilerTask$default(CompilationTaskContext.kt:118) at io.bazel.kotlin.builder.tasks.jvm.CompilationTaskKt.compileKotlin(CompilationTask.kt:415) at io.bazel.kotlin.builder.tasks.jvm.KotlinJvmTaskExecutor$execute$1$1$1$1.invoke(KotlinJvmTaskExecutor.kt:60) at io.bazel.kotlin.builder.tasks.jvm.KotlinJvmTaskExecutor$execute$1$1$1$1.invoke(KotlinJvmTaskExecutor.kt:58) at io.bazel.kotlin.builder.toolchain.CompilationTaskContext.execute(CompilationTaskContext.kt:153) at io.bazel.kotlin.builder.toolchain.CompilationTaskContext.execute(CompilationTaskContext.kt:145) at io.bazel.kotlin.builder.tasks.jvm.KotlinJvmTaskExecutor$execute$1.invoke(KotlinJvmTaskExecutor.kt:58) at io.bazel.kotlin.builder.tasks.jvm.KotlinJvmTaskExecutor$execute$1.invoke(KotlinJvmTaskExecutor.kt:54) at io.bazel.kotlin.builder.toolchain.CompilationTaskContext.execute(CompilationTaskContext.kt:153) at io.bazel.kotlin.builder.toolchain.CompilationTaskContext.execute(CompilationTaskContext.kt:145) at io.bazel.kotlin.builder.tasks.jvm.KotlinJvmTaskExecutor.execute(KotlinJvmTaskExecutor.kt:54) at io.bazel.kotlin.builder.tasks.KotlinBuilder.executeJvmTask(KotlinBuilder.kt:210) at io.bazel.kotlin.builder.tasks.KotlinBuilder.build(KotlinBuilder.kt:103) at io.bazel.kotlin.builder.tasks.CompileKotlin.invoke(CompileKotlin.kt:27) at io.bazel.worker.PersistentWorker$workTo$1.invoke(PersistentWorker.kt:97) at io.bazel.worker.PersistentWorker$workTo$1.invoke(PersistentWorker.kt:97) at io.bazel.worker.WorkerContext$TaskContext.resultOf(WorkerContext.kt:128) at io.bazel.worker.WorkerContext.doTask(WorkerContext.kt:156) at io.bazel.worker.PersistentWorker$start$1$1$producer$1$1.invoke$lambda$0(PersistentWorker.kt:70) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) INFO: Elapsed time: 1.687s, Critical Path: 0.09s INFO: 2 processes: 2 internal. FAILED: Build did NOT complete successfullyI've created an example workspace that demonstrates this problem.
The text was updated successfully, but these errors were encountered: