Building Android app (official tutorial) fails on Linux with JDK8

[dimsuz]

Initially I posted to #6164 because I saw a comment with an exact stack trace I get: #6164 (comment). Was advised by @jin to open a new issue instead.

Description of the problem / feature request:

I'm trying to follow an official Android Tutorial and cannot build the example app on a Linux sytem with JDK8.

To reproduce:

1. Have a system with JDK 8.
2. Follow the tutorial

The command bazel build //src/main:app gives me a 'Can't start JVM' error, so I figured that's because I use java 8 (because Android):

$ javac -version
javac 1.8.0_192

So I try to tell bazel about this:

bazel build --java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8 --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8 //src/main:app

And get an error in response:

INFO: Invocation ID: 6850ce2f-2b64-4627-8377-966538caba8d
INFO: Build options have changed, discarding analysis cache.
INFO: Analysed target //src/main:app (0 packages loaded, 957 targets configured).
INFO: Found 1 target...
ERROR: /home/dima/tmp/bazel-examples/android/tutorial/src/main/BUILD:1:1: Sharding classes for dexing for //src/main:app failed (Exit 1) mapper failed: error executing command bazel-out/host/bin/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/ziputils/mapper --output_jar bazel-out/k8-fastbuild/bin/src/main/_dx/app/classes.jar --output_resources ... (remaining 6 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
	at com.google.devtools.build.android.ziputils.ZipIn.loadEndOfCentralDirectory(ZipIn.java:491)
	at com.google.devtools.build.android.ziputils.ZipIn.endOfCentralDirectory(ZipIn.java:96)
	at com.google.devtools.build.android.ziputils.SplitZip.run(SplitZip.java:294)
	at com.google.devtools.build.android.ziputils.DexMapper.main(DexMapper.java:70)
Target //src/main:app failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 4.184s, Critical Path: 4.01s
INFO: 23 processes: 13 processwrapper-sandbox, 10 worker.
FAILED: Build did NOT complete successfully

Any hints on how to fix this? (Seeing how all relevant issues are closed I suspect I might be doing something wrong)

What operating system are you running Bazel on?

Arch Linux

What's the output of bazel info release?

Reporting from home, can't include full output (from work), but it's version 0.20.0

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

I use Arch Linux bazel package

[jin]

cc @cushon

[cushon]

@iirina @lberki

How is DexMapper built? It looks like it's being build against a JDK 9 bootclasspath.

[jin]

I have 1.8 as javac -version and using Bazel 0.20.0, and I can build the app without passing in any flags.

The command bazel build //src/main:app gives me a 'Can't start JVM' error, so I figured that's because I use java 8 (because Android):

This shouldn't happen because Bazel is using its embedded JDK to start itself, and should pick up your system JDK 8 automatically to build the Bazel app.

[dimsuz]

I feel like I should include the full "can't create jvm"-error output, here it is, just reproduced on another computer with JDK8:

[dima@fprocket tutorial]$ bazel build //src/main:app
INFO: Invocation ID: 77c23a98-8d25-44db-b43b-a3bdf344a137
INFO: Analysed target //src/main:app (35 packages loaded, 958 targets configured).
INFO: Found 1 target...
ERROR: /home/dima/.cache/bazel/_bazel_dima/2e8a1ad2443e84777a19274cf4f0232e/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/dexer/BUILD:3:1: Building external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/dexer/libdexer.jar (15 source files) and running annotation processors (AutoValueProcessor) failed: Worker process did not return a WorkResponse:

---8<---8<--- Start of log, file at /home/dima/.cache/bazel/_bazel_dima/2e8a1ad2443e84777a19274cf4f0232e/bazel-workers/worker-2-Javac.log ---8<---8<---
Unrecognized VM option 'CompactStrings'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
---8<---8<--- End of log ---8<---8<---
Target //src/main:app failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 8.094s, Critical Path: 1.85s
INFO: 7 processes: 7 processwrapper-sandbox.
FAILED: Build did NOT complete successfully

Due to having this error I started trying to pass toolchain-related flags to the build command.

[davido]

@dimsuz Is that problem Android related? Can you try to build regular java application in your environment? E.g.:

  $ git clone https://github.com/davido/bazel_printy
  $ cd bazel_printy
  $ bazel build --workspace_status_command=tools/workspace-status.sh :printy_stamped
  $ java -jar bazel-genfiles/printy_stamped.jar --version
fbba4ba-dirty

[dimsuz]

@davido Firstly, I had to add --java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8 --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8 to avoid Error: Could not create the Java Virtual Machine. message I mentioned above.

After this I tried your command and it gave me the following (another) error.

Click to expand

[dima@dimsuzkode bazel_printy]$ bazel build --java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8 --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8 --workspace_status_command=tools/workspace-status.sh :printy_stamped
WARNING: The following rc files are no longer being read, please transfer their contents or import their path into one of the standard rc files:
/home/dima/tmp/bazel_printy/tools/bazel.rc
INFO: Invocation ID: 41616394-19a2-4ac5-8ae5-5f528df5567e
INFO: Analysed target //:printy_stamped (18 packages loaded, 716 targets configured).
INFO: Found 1 target...
ERROR: /home/dima/tmp/bazel_printy/BUILD:9:1: Building libprinty_lib.jar (1 source file) failed (Exit 1)
src/main/java/org/gerritcon/mv2016/Printy.java:12: error: An unhandled exception was thrown by the Error Prone static analysis plugin.
public class Printy {
       ^
     Please report this at https://github.com/google/error-prone/issues/new and include the following:
 
     error-prone version: 2.3.2-SNAPSHOT
     BugPattern: NullAway
     Stack Trace:
     java.lang.IllegalStateException: To run the NullAway analysis plugin please specify analysis options with -XepOpt:NullAway:[...]. You should at least specify annotated packages, using the -XepOpt:NullAway:AnnotatedPackages=[...] flag.
    at com.uber.nullaway.DummyOptionsConfig.isExcludedClass(DummyOptionsConfig.java:61)
    at com.uber.nullaway.NullAway.isExcludedClass(NullAway.java:1406)
    at com.uber.nullaway.NullAway.matchClass(NullAway.java:867)
    at com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:582)
    at com.google.errorprone.scanner.ErrorProneScanner.visitClass(ErrorProneScanner.java:146)
    at com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:808)
    at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
    at com.google.errorprone.scanner.Scanner.scan(Scanner.java:82)
    at com.google.errorprone.scanner.Scanner.scan(Scanner.java:42)
    at com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
    at com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
    at com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:144)
    at com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:603)
    at com.google.errorprone.scanner.ErrorProneScanner.visitCompilationUnit(ErrorProneScanner.java:146)
    at com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:591)
    at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:56)
    at com.google.errorprone.scanner.Scanner.scan(Scanner.java:64)
    at com.google.errorprone.scanner.ErrorProneScannerTransformer.apply(ErrorProneScannerTransformer.java:43)
    at com.google.errorprone.ErrorProneAnalyzer.finished(ErrorProneAnalyzer.java:152)
    at com.google.devtools.build.buildjar.javac.plugins.errorprone.ErrorPronePlugin.postFlow(ErrorPronePlugin.java:121)
    at com.google.devtools.build.buildjar.javac.BlazeJavaCompiler.flow(BlazeJavaCompiler.java:115)
    at com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1363)
    at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:959)
    at com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:100)
    at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:142)
    at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:96)
    at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:90)
    at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:114)
    at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder$1.invokeJavac(SimpleJavaLibraryBuilder.java:106)
    at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:54)
    at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:109)
    at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:117)
    at com.google.devtools.build.buildjar.BazelJavaBuilder.processRequest(BazelJavaBuilder.java:105)
    at com.google.devtools.build.buildjar.BazelJavaBuilder.runPersistentWorker(BazelJavaBuilder.java:67)
    at com.google.devtools.build.buildjar.BazelJavaBuilder.main(BazelJavaBuilder.java:45)
Target //:printy_stamped failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 2.559s, Critical Path: 2.13s
INFO: 5 processes: 4 processwrapper-sandbox, 1 worker.
FAILED: Build did NOT complete successfully

[ulfjack]

I see a similar error with a Java binary built on a Mac using Bazel 0.22, and trying to run that binary with a Jdk 8.

Exception in thread "my-thread" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;

Running like this:

JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/ bazel run //app

It seems like the Java compiler used by Bazel 0.22 is miscompiling the code in some way.

[ulfjack]

The code works with Jdk 11, but not with Jdk 8.

[ulfjack]

This only seems to happen when I compile a jar and use that in another project.

[ulfjack]

I have a repro:

java/BUILD:
java_binary(
  name = "java",
  srcs = glob(["*.java"]),
  main_class = "Main",
)

java/Main.java:
import java.nio.ByteBuffer;
public class Main {
  public static void main(String[] args) {
    ByteBuffer buffer = ByteBuffer.allocate(4096);
    buffer.clear();
    System.out.println("Hello World!");
  }
}

$ bazel build //java:java_deploy.jar
$ /Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/bin/java -jar bazel-bin/java/java_deploy.jar
Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
	at Main.main(Main.java:5)

I have both Jdk 8 and Jdk 11 installed on my Mac, and Jdk 11 is the default (what /usr/bin/java points to).

[cushon]

Jdk 11 is the default

@ulfjack I think that's WAI, then. The target --javabase defaults to your default local JDK, so that build is implicitly targeting JDK 11 and then actually being run on JDK 8.

It's equivalent to this:

JAVA_HOME=$(/usr/libexec/java_home -v 11) bazel build //java:java_deploy.jar
$(/usr/libexec/java_home -v 1.8)/bin/java -jar bazel-bin/java/java_deploy.jar
...
Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
	at Main.main(Main.java:5)

or this:

bazel build --define=ABSOLUTE_JAVABASE=$(/usr/libexec/java_home -v 11) --javabase=@bazel_tools//tools/jdk:absolute_javabase //java:java_deploy.jar
$(/usr/libexec/java_home -v 1.8)/bin/java -jar bazel-bin/java/java_deploy.jar
...
Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
	at Main.main(Main.java:5)

This is a situation where you probably want to explicitly set --javabase:

bazel build --define=ABSOLUTE_JAVABASE=$(/usr/libexec/java_home -v 1.8) --javabase=@bazel_tools//tools/jdk:absolute_javabase //java:java_deploy.jar
$(/usr/libexec/java_home -v 1.8)/bin/java -jar bazel-bin/java/java_deploy.jar
...
Hello World!

[ulfjack]

Oh, I see why this happens now. Someone overrode Buffer clear() with ByteBuffer clear() (covariant return types), which means code compiled against a newer Jdk is no longer compatible with an older one.

However, that means whoever checked in a jar in the deps of the dexer / sharder / whatever tool is used there used a newer Jdk. If the Android rules maintainers want to continue to allow running with Jdk 8, they'll need to replace the check-in jar with one compiled against Jdk 8.

[ulfjack]

The .jar is bundled into Bazel as part of the release process, so apparently that's running with a newer version of the Jdk. Shouldn't we be running the embedded tools with the embedded Jdk then?

[ulfjack]

@dimsuz says he's running with the host_java_toolchain flag, which makes Bazel use that for the embedded tools, I guess.

[ulfjack]

Except he's using the arch linux package, which apparently doesn't include a Jdk? I think you'll need to file a bug against the arch linux bazel package about the NoSuchMethodError.

[dimsuz]

@ulfjack great findings!

there's a package file list here in Package Contents section, it seems to include some jars, but no jdk ones AFAIKT.

So I have to use a system one.

[ulfjack]

Things to try:

• Bazel 0.22 without flags
• Pass only --java_toolchain, but not --host_java_toolchain
• Install Jdk 10 (or 11) in addition to Jdk 8, and run with --java_toolchain=...jdk8
• Previous step, but also add --host_java_toolchain=...jdk10 (or 11)

One of these should work. If none of these work, but you need to keep using jdk8, then that'll require custom Bazel binaries / custom-built Android tools, I think.

[ulfjack]

I saw the arch list, and it says "java-environment=10", but I don't know what that means. Technically, it's sufficient if you have a Jdk 10 installed - it doesn't have to be the default Jdk (that's possible at least on on Debian Linux and MacOS).

[dimsuz]

Arch has the ability to have few of JDK implemenations installed (openjdk, sun + different versions), and it has a tool to select one of them as the default. I have to set the default to JDK 8, because Android Studio wouldn't let me build Gradle-based projects on JDK > 8 for some reason (I don't recall the exact error atm).

That string is some kind of dynamic dependency specification I guess.

I will try the things you listed above. I have JDK 10 installed:

$ archlinux-java status
Available Java environments:
  java-10-openjdk
  java-11-jdk
  java-8-jdk (default)

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 2+ years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[github-actions]

This issue has been automatically closed due to inactivity. If you're still interested in pursuing this, please reach out to the triage team (@bazelbuild/triage). Thanks!