Build scripts incompatible with Java 9

[kentdobias]

The packaged Android.mk is incompatible with Pie builds since the prebuilt JDK packaged with the ASOP is now version 9, which the included gradle scripts do not support. This leads to a build failure, with result

[  0% 132/92470] build packages/apps/GmsCore/play-services-co...d/outputs/apk/release/play-services-core-release-unsigned.apk
FAILED: packages/apps/GmsCore/play-services-core/build/outputs/apk/release/play-services-core-release-unsigned.apk 
/bin/bash -c "(rm -Rf packages/apps/GmsCore/play-services-core/build ) && (mkdir -p /home/pants/dev/lineage-microg-docker-volumes/src/LINEAGE_16_0/out/target/common/obj/APPS/GmsCore_intermediates ) && (ln -s /home/pants/dev/lineage-microg-docker-volumes/src/LINEAGE_16_0/out/target/common/obj/APPS/GmsCore_intermediates packages/apps/GmsCore/play-services-core/build ) && (echo \"sdk.dir=/opt/android-sdk\" > packages/apps/GmsCore/local.properties ) && (cd packages/apps/GmsCore && git submodule update --recursive --init ) && (cd packages/apps/GmsCore/play-services-core && JAVA_TOOL_OPTIONS=\" -Dfile.encoding=UTF8\" ../gradlew assembleRelease )"
Picked up JAVA_TOOL_OPTIONS:  -Dfile.encoding=UTF8

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring project ':microg-ui-tools'.
> java.lang.NullPointerException (no error message)

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s

It would be nice if the build system were updated.

[NormalRa]

In the meantime, as a temporary fix, it is possible to enforce building with Java 8 via the use of EXPERIMENTAL_USE_OPENJDK9 environment variable since both jdk8 and jdk9 are available as prebuilts in the Pie build system. Additionally, one can also set OVERRIDE_ANDROID_JAVA_HOME for some other setups.

Unfortunately, switching to jdk8 was not as straightforward as I had hoped it would be because of the following error:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

Regardless of the fact that I have enough system memory, this error would occur whenever jdk8 attempted to compile the Dialer app/module. The reason for this issue eludes me completely, but it can be mitigated by increasing the JavacHeapSize in build/soong/java/config/config.go and adding ${JavacHeapFlags} to CommonJdkFlags static variable.

I̶ ̶h̶a̶v̶e̶ ̶y̶e̶t̶ ̶t̶o̶ ̶t̶e̶s̶t̶ ̶t̶h̶e̶ ̶c̶o̶m̶p̶i̶l̶e̶d̶ ̶s̶y̶s̶t̶e̶m̶ ̶i̶m̶a̶g̶e̶ ̶t̶h̶o̶u̶g̶h̶. Tested a Pie build and it seems to be working fine.

[chris42]

I tried to do a build for Pie with my AOSP, however failed so far to do so. As for reference and if someone else is trying this: Following errors I encountered:

• ANDROID_BUILD_TOP in Android.mk is deprecated and needs to be adapted
• NullPointerException as above seems to be a Gradle Version Issue (see here). The workaround works, as seems the upgrade of gradle

Right now I run into proguard errors, however have not found any solution to this.

[chris42]

Sorry for taking a time to come back, was busy with some other stuff. I just finished compiling it with AOSP Pie (Still need to test the ROM though). Following things I encountered:

ANDROID_BUILD_TOP in Android.mk is deprecated, hence I adapted the Android.mk in GmsCore like this:

[...]
gmscore_root  := $(LOCAL_PATH)
gmscore_dir   := play-services-core
gmscore_out   := out/target/common/obj/APPS/$(LOCAL_MODULE)_intermediates
gmscore_build := $(gmscore_root)/$(gmscore_dir)/build
gmscore_apk   := build/outputs/apk/release/play-services-core-release-unsigned.apk

...
.PHONY: preps_gms
preps_gms:
        rm -rf $(gmscore_build); \
        mkdir -p $(gmscore_out); \
        ln -s ../../../../$(gmscore_out) $(gmscore_build)

$(gmscore_root)/$(gmscore_dir)/$(gmscore_apk): preps_gms
        echo "sdk.dir=$(ANDROID_HOME)" > $(gmscore_root)/local.properties; \
        cd $(gmscore_root) && git submodule update --recursive --init; \
        cd $(gmscore_dir) && JAVA_TOOL_OPTIONS="$(JAVA_TOOL_OPTIONS) -Dfile.encoding=UTF8" ../gradlew assembleRelease
[...]

The .PHONY target I already used in Oreo, as I got some errors on the link when compiling with multiple threads (It was not created properly).
Additionally I would add LOCAL_PRIVILEGED_MODULE := true to the Android.mk to have it in /system/priv-app.

The NullPointerException from above was solved by upgrading gradle. I saw that @mar-v-in did this as well in his latest commits to GmsCore. I went through all the subprojects and aligned the gradle and wrapper versions. That worked for me.
Upgrading gradle also produced an out of memory error for me, as apparently the gradle daemon starts with 512MB instead of 1GB in newest version.
org.gradle.jvmargs=-Xmx1g in the the gradle.properties changes that back.

The ProGuard warnings were

Warning: org.microg.wearable.MessageListener: can't find referenced class java.lang.invoke.StringConcatFactory
Warning: org.microg.wearable.MessageListener: can't find referenced class java.lang.invoke.StringConcatFactory
Warning: org.microg.wearable.SocketWearableConnection: can't find referenced class java.lang.invoke.StringConcatFactory
Warning: org.microg.wearable.SocketWearableConnection: can't find referenced class java.lang.invoke.StringConcatFactory
Warning: org.microg.wearable.WearableConnection: can't find referenced class java.lang.invoke.StringConcatFactory
Warning: org.microg.wearable.WearableConnection: can't find referenced class java.lang.invoke.StringConcatFactory

Hence a -dontwarn java.lang.invoke.StringConcatFactory in proguard.flags solved that.

@mar-v-in Would be nice if you could take a look on what I did. I can create a pull request as well if the changes are ok.

P.S.: In the changes on 26.05 you also introduced a local.properties file, which does not yet exist. Empty file works, but just a note on the side.

[chris42]

So tested the ROM and found, that you will need to privapp whitelist

<permission name="android.permission.LOCATION_HARDWARE"/>
<permission name="android.permission.CHANGE_DEVICE_IDLE_TEMP_WHITELIST"/>

for com.google.android.gms. Otherwise the ROM won't boot. @Nanolx figured that out and put already a mechanism in his installer.
For build time creation I added Permission XMLs to the Android.mk
After that the ROM boots, and I can get all green checkmarks on microg Settings.

[Nanolx]

@chris42 yes, the major problem with

<permission name="android.permission.LOCATION_HARDWARE"/>

is indeed, that not all ROMs expect this one to be whitelisted, so I previously had the issue that for some users uNlp did only partially work if not whitelisted, while for others the ROM did not boot when whitelisted. Also Oreo and Pie are very different in what they expect whitelisted, while Pie also refuses booting on some ROMs if a permission is whitelisted that shouldn't be - and not just because of missing whitelists.

A complete mess. I already pinged @ale5000-git so I expect anytime in the future his installer will get the automatic detection aswell, though he's currently busy.

For the curious here are the functions which auto-generate the whitelists for NanoDroid starting with version 21.0-beta2, namely dump_system_privapp_permissions() and create_privapp_permissions_whitelist(): https://gitlab.com/Nanolx/NanoDroid/blob/master/CommonInstaller#L782

[chris42]

@Nanolx Might be, that some ROM builders put the whitelist right into the build. I am using a Sony AOSP, which is pretty pure.
I looked into your solution, but I am not sure, if I can replicate that during build time. Right now I just copy over the whitelist with the GMSCore build.
EDIT: And some might just deactivate the enforcement of the whiltelist?

[Nanolx]

Unless I'm mistaken, shouldn't the Androud build system autocreate a proper privapp permission whitelist? Am 1. Juni 2019 20:27:49 MESZ schrieb Christian <[email protected]>:

…

@Nanolx Might be, that some ROM builders put the whitelist right into the build. I am using a Sony AOSP, which is pretty pure. I looked into your solution, but I am not sure, if I can replicate that during build time. Right now I just copy over the whitelist with the GMSCore build. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #729 (comment)

-- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

[chris42]

I would guess so as well, however I do not understand the build system well enough, to tell what is wrong.
Apparently having the uses-permission defined in the AndroidManifest.xml for play-services-core is not enough.
The documentation I found is also only talking about editing and creating it apart from the build.
Not sure if @mar-v-in has more experience in it and there is an easier/safer way than copying the permissions manually.

[arn4v]

Has anyone been able to compile GmsCore inline with Android 10?

I get this error: packages/apps/GmsCore/Android.mk:36: error: writing to readonly directory: "packages/apps/GmsCore/play-services-core/build/outputs/apk/release/play-services-core-release-unsigned.apk"
19:53:21 ckati failed with: exit status 1