Update for Android Gradle plugin 2.5 #121

rschiu · 2017-03-31T23:29:24Z

Use compileClasspath Configuration for each variant instead
of the compile Configuration for the source sets for the
extract include proto tasks.

adrien-aubel · 2017-05-18T22:36:56Z

Thanks @rschiu, this let me build with Android Studio 3.0 Canary 1 / Gradle 4.0-milestone-1 / build tools 3.0.0-alpha1.

Hopefully this will make it in a 0.8.2 released soon? :) (CC @zhangkun83)

LouisCAD · 2017-06-12T22:43:59Z

Why this PR still has not been merged almost a month later? My Android project is dying to try the new Android Studio!

zhangkun83 · 2017-06-12T23:40:21Z

I am still waiting for @rschiu to post the latest changes to this PR.

lucastsantos · 2017-06-19T11:54:02Z

looks like it's rebased now 🙇

zhangkun83 · 2017-06-20T23:48:46Z

src/main/groovy/com/google/protobuf/gradle/ProtobufPlugin.groovy

-        def extractIncludeProtosTask = maybeAddExtractIncludeProtosTask(sourceSetName)
+      // Extract proto file from the compile classpath configuration exposed in Android Gradle plugin 2.5.  Revert to
+      // original behavior if the configuration is not found.
+      Configuration compileConfiguration = variant.hasProperty("compileConfiguration") ? variant.compileConfiguration : null


Since compileConfiguration is not used in the else-branch, why not:

if (variant.hasProperty("compileConfiguration")) { ... variant.compileConfiguration.incoming.artifactView{ ...

zhangkun83 · 2017-06-20T23:51:15Z

src/main/groovy/com/google/protobuf/gradle/ProtobufPlugin.groovy

+            maybeAddExtractIncludeProtosTask(
+                    variant.name,
+                    compileConfiguration.incoming.artifactView{ attributes{ it.attribute(artifactType, "jar") }}.files,
+                    testedCompileConfiguration == null ?


Same here:

variant.hasProperty("testedVariant") ? variant.testedVariant.compileConfiguration.incoming ... : null

zhangkun83 · 2017-06-21T00:16:28Z

gradle/wrapper/gradle-wrapper.properties

@@ -3,4 +3,4 @@ distributionBase=GRADLE_USER_HOME
 distributionPath=wrapper/dists
 zipStoreBase=GRADLE_USER_HOME
 zipStorePath=wrapper/dists
-distributionUrl=https\://services.gradle.org/distributions/gradle-3.0-all.zip
+distributionUrl=https\://services.gradle.org/distributions/gradle-4.0-rc-3-all.zip


Please add 4.0-rc-3 to ProtobufJavaPluginTest.gradleVersions and ProtobufAndroidPluginTest.

zhangkun83 · 2017-06-21T00:27:29Z

src/main/groovy/com/google/protobuf/gradle/ProtobufPlugin.groovy

+      // Extract proto file from the compile classpath configuration exposed in Android Gradle plugin 2.5.  Revert to
+      // original behavior if the configuration is not found.
+      Configuration compileConfiguration = variant.hasProperty("compileConfiguration") ? variant.compileConfiguration : null
+      if (compileConfiguration != null) {


protobuf-gradle-plugin/testProjectAndroid/build.gradle is still using com.android.tools.build:gradle:2.2.0-beta1. This new branch is not tested. You will need some work on the test code (ProtobufAndroidPluginTest and its friends) to test both the old and the new Android plugins.

Use compileClasspath Configuration for each variant instead of the compile Configuration for the source sets for the extract include proto tasks.

rschiu · 2017-06-26T23:57:00Z

@zhangkun83 Sorry, I didn't realized you did the review last week. I made the changes. PTAL.
The latest plugin requires newer buildtools. The updated test uses compileSdk 25 and build tools 25.0.2, but I do not know how to update the SDK for the tests. Can you help me with that?

zhangkun83 · 2017-06-27T01:20:32Z

@rschiu I probably know less about Android SDK than you are. You can select from the SDK UI and install the packages, can't you?
Btw, when you push new changes to the PR, just do a normal commit, do not squash/rebase. This way the review history can be preserved.

LouisCAD · 2017-06-27T07:26:18Z

@rschiu You can now use SDK 26 and build tools 26.0.0

zhangkun83 · 2017-06-27T16:27:34Z

@rschiu I tested your PR with Android tools installed. It seems compileArmFreeappDebugJavaWithJavac doesn't have the generated files in its input:

org.gradle.testkit.runner.UnexpectedBuildFailure: Unexpected build execution failure in /usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid with arguments [-DANDROID_PLUGIN_VERSION=3.0.0-alpha4, testProjectAndroid:build]

Output:
The com.google.protobuf plugin was already applied to the project: :testProjectAndroid and will not be applied again after plugin: android
Configuration 'compile' in project ':testProjectAndroid' is deprecated. Use 'implementation' instead.
:testProjectAndroid:preBuild UP-TO-DATE
:testProjectAndroid:preArmFreeappDebugBuild
Download https://plugins.gradle.org/m2/io/grpc/grpc-core/maven-metadata.xml
:testProjectAndroid:compileArmFreeappDebugAidl
:testProjectAndroid:compileArmFreeappDebugRenderscript
:testProjectAndroid:checkArmFreeappDebugManifest
:testProjectAndroid:generateArmFreeappDebugBuildConfig
:testProjectAndroid:extractArmFreeappDebugProto
:testProjectAndroid:extractArmProto
:testProjectAndroid:extractDebugProto
:testProjectAndroid:extractFreeappProto
:testProjectLite:extractIncludeProto
:testProjectLite:extractProto
:testProjectLite:generateProto
:testProjectLite:compileJavawarning: [options] bootstrap class path not set in conjunction with -source 1.7
Note: /usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectLite/build/generated/source/proto/main/javalite/io/grpc/testing/Messages.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
1 warning

:testProjectLite:processResources
:testProjectLite:classes
:testProjectLite:jar
:testProjectAndroid:extractIncludeArmFreeappDebugProto
:testProjectAndroid:extractProto
:testProjectAndroid:generateArmFreeappDebugProto
:testProjectAndroid:generateArmFreeappDebugResValues
:testProjectAndroid:generateArmFreeappDebugResources
:testProjectAndroid:mergeArmFreeappDebugResources
:testProjectAndroid:createArmFreeappDebugCompatibleScreenManifests
:testProjectAndroid:processArmFreeappDebugManifest
:testProjectAndroid:splitsDiscoveryTaskArmFreeappDebug
:testProjectAndroid:processArmFreeappDebugResources
:testProjectAndroid:generateArmFreeappDebugSources
:testProjectAndroid:javaPreCompileArmFreeappDebug
:testProjectAndroid:compileArmFreeappDebugJavaWithJavac/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:104: error: package nano.Messages does not exist
    private nano.Messages.SimpleContext context_;
                         ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:122: error: package nano.Messages does not exist
    public nano.Messages.SimpleContext getContext() {
                        ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:132: error: package nano.Messages does not exist
    private void setContext(nano.Messages.SimpleContext value) {
                                         ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:147: error: package nano.Messages.SimpleContext does not exist
        nano.Messages.SimpleContext.Builder builderForValue) {
                                   ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:158: error: package nano.Messages does not exist
    private void mergeContext(nano.Messages.SimpleContext value) {
                                           ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:349: error: package nano.Messages does not exist
      public nano.Messages.SimpleContext getContext() {
                          ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:359: error: package nano.Messages does not exist
      public Builder setContext(nano.Messages.SimpleContext value) {
                                             ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:372: error: package nano.Messages.SimpleContext does not exist
          nano.Messages.SimpleContext.Builder builderForValue) {
                                     ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:384: error: package nano.Messages does not exist
      public Builder mergeContext(nano.Messages.SimpleContext value) {
                                               ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:40: error: package nano.Messages does not exist
    nano.Messages.SimpleContext getContext();
                 ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:123: error: package nano.Messages does not exist
      return context_ == null ? nano.Messages.SimpleContext.getDefaultInstance() : context_;
                                             ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:160: error: package nano.Messages does not exist
          context_ != nano.Messages.SimpleContext.getDefaultInstance()) {
                                   ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:162: error: package nano.Messages does not exist
          nano.Messages.SimpleContext.newBuilder(context_).mergeFrom(value).buildPartial();
                       ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:456: error: package nano.Messages.SimpleContext does not exist
                  nano.Messages.SimpleContext.Builder subBuilder = null;
                                             ^
/usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/build/generated/source/proto/armFreeappDebug/javalite/io/grpc/helloworldexample/Helloworld.java:460: error: package nano.Messages does not exist
                  context_ = input.readMessage(nano.Messages.SimpleContext.parser(), extensionRegistry);
                                                            ^
Note: /usr/local/google/home/zhangkun/github/protobuf-gradle-plugin-upstream/build/tests/testProjectAndroid/testProjectAndroid/src/main/java/io/grpc/helloworldexample/HelloworldActivity.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
15 errors
 FAILED

zhangkun83 · 2017-06-27T16:21:20Z

src/main/groovy/com/google/protobuf/gradle/ProtobufPlugin.groovy


-        def extractIncludeProtosTask = maybeAddExtractIncludeProtosTask(sourceSetName)
+      // Extract proto file from the compile classpath configuration exposed in Android Gradle plugin 2.5.  Revert to
+      // original behavior if the configuration is not found.


Just say the if-branch is for Android plugin 2.5+, and the else-branch is for below 2.5.
There is no "original" behavior outside the context of this PR.

rschiu · 2017-06-27T18:05:36Z

@zhangkun83 How did you run the test to get that build failure? I have updated the tools to 26, but wasn't able to reproduce it.

zhangkun83 · 2017-06-27T18:08:38Z

Did your run succeed instead?I just installed various SDK packages and ran ./gradlew clean test.
I got failures first telling me some packages were missing. I installed them and was able to get the compile errors.

rschiu · 2017-06-27T19:00:41Z

Ya, I re-ran the tests and all runs succeeded. I will dive deeper into it.

LouisCAD · 2017-07-01T01:32:48Z

@rschiu Could you provide an example of before/after gradle script for the workaround you describe on your first message of this PR, please?

LouisCAD · 2017-07-12T07:15:57Z

@rschiu What is blocking the progress of this PR and issue #129?
Anything I can do to help the Android Studio 3.0 compatible version come sooner?

rschiu · 2017-07-12T21:33:28Z

@zhangkun83 I have updated to the latest Android Gradle plugin. Tests are passing on Travis, can you take another look?

rschiu · 2017-07-12T22:39:12Z

@LouisCAD I am not sure what you mean by a before/after gradle script for the workaround. The change described in the first message in implemented in this PR.

zhangkun83 · 2017-07-14T21:01:01Z

@rschiu Have you tried testing locally? I keep getting the same failure as mentioned above when testing it locally.

rschiu · 2017-07-14T21:12:00Z

Yes, the test pass locally as well.

zhangkun83 · 2017-07-14T22:35:22Z

Hmm, it succeeded on my workstation after I git clean. Seems some left-over files that are not cleaned up by gradle clean.

rschiu · 2017-07-20T02:01:55Z

I have tried quite a few things, including running my own docker instance, but I am unable to reproduce the error locally. Is anyone else able to do this on their local machine? Any ideas?

nmulcahey · 2017-07-21T00:04:00Z

It runs locally for me, I suggest clearing the Travis CI cache and trying to run the build again. $HOME/.gradle may also be too aggressive of a cache anyways.

https://docs.travis-ci.com/user/caching/#Clearing-Caches

…d-plugin-2.5

Update plugin to alpha7

rschiu · 2017-07-21T00:28:13Z

I assume you would need correct access to do that. Could someone on the protobuf team try clearing the cache?

zhangkun83 · 2017-07-21T17:02:57Z

I cleared the cache and re-ran the test. It's still broken.

<< is deprecated by Gradle

This reverts commit cb5ac79.

rschiu · 2017-07-24T19:30:02Z

I was finally able to reproduce the problem with a docker instance limiting the memory. So this seems to be a memory issue. Is it possible to increase the memory size of the build server?

zhangkun83 · 2017-07-24T20:11:45Z

Currently Travis is allocating 4GB of memory. How much memory did you allocate in your docker instance? I wonder if you need to really increase the memory, or just tune up the java heap size. Anyway, it seems the way to have memory in Travis is to change sudo: false to sudo: required in .travis.yml

rschiu · 2017-07-24T21:35:52Z

That's kind of odd. Is there anything else that affect the available memory? I reproduced the problem when I limit the memory from the default of 2g to 1.5g.
At some point, I also tried to increase Gradle's heap size from 1g to 1.5g, but the daemon got killed because of that https://travis-ci.org/google/protobuf-gradle-plugin/builds/255465047

rschiu · 2017-07-25T22:28:32Z

Ok, I changed to sudo:required to get VM with larger memory and it's working now. The tests ran successfully twice in a row, the previous failure was due to codenarc. PTAL

zhangkun83 · 2017-07-26T00:12:33Z

LGTM

Use compileClasspath Configuration for each variant instead of the compile Configuration for the source sets for the extract include proto tasks.

Support for Android plugin 2.5+ was added in google#121. While it made extractIncludeProtosTask to copy the protos to a different destination with Android plugin 2.5+ than with older verisons, it didn't make generateProtoTask to "include" the new location. This causes Android tests to fail with Gradle 5.0 and Android plugin 3.1.0. The bug has remained undetected because test was never run with Android 2.5+.

rschiu force-pushed the update-for-android-plugin-2.5 branch from 61f66d3 to 84b420b Compare April 19, 2017 00:26

rschiu force-pushed the update-for-android-plugin-2.5 branch from 84b420b to 16b199a Compare May 16, 2017 01:16

zhangkun83 mentioned this pull request May 19, 2017

Error on build in Android Studio 3 Canary & Gradle 4 #129

Closed

rschiu force-pushed the update-for-android-plugin-2.5 branch from 16b199a to 753be10 Compare June 13, 2017 21:03

zhangkun83 reviewed Jun 21, 2017

View reviewed changes

Update for Android Gradle plugin 3.0

c5b0bdb

Use compileClasspath Configuration for each variant instead of the compile Configuration for the source sets for the extract include proto tasks.

rschiu force-pushed the update-for-android-plugin-2.5 branch from 753be10 to c5b0bdb Compare June 26, 2017 23:22

zhangkun83 reviewed Jun 27, 2017

View reviewed changes

rschiu added 3 commits June 27, 2017 14:57

Update buildtools version in test to 26

854ffd3

Update travis to use android sdk and buildtools 26

02dad29

Modify build cache directory in android test

9e86544

Update android plugin in tests.

5fc91a1

rschiu added 2 commits July 19, 2017 14:33

Added extra logging to investigate test flakiness.

b5666d9

Give more memory to gradle to run dex in process during android test

ab5c01e

rschiu added 2 commits July 20, 2017 17:04

Merge remote-tracking branch 'upstream/master' into update-for-androi…

c0468ff

…d-plugin-2.5

Revert changes to increase jvm heap size

ab2c661

Update plugin to alpha7

rschiu added 6 commits July 21, 2017 12:23

Clean android build cache to ensure it doesn't affect other tests

e5799e9

Covert << in checkJavaVersion to doLast

4b938cd

<< is deprecated by Gradle

Fix codearc warning

faab6fb

Limit predex thread count

c7ec477

Disable predex

cb5ac79

Revert "Disable predex"

f07f9bd

This reverts commit cb5ac79.

rschiu added 6 commits July 24, 2017 18:17

Configure java heap size for predex

ba624d8

Update build-tools to 26.0.1

8caab49

Update build-tools used in test

554a67e

Set sudo:required to use VM with larger memory.

0b7e1e6

Actually set sudo:required

036f68f

Fix codenarc failure

e3ac636

zhangkun83 merged commit 8fac7ab into google:master Jul 26, 2017

zpencer mentioned this pull request Sep 13, 2017

Support for gradle experimental android plugin #85

Closed

artem-zinnatullin mentioned this pull request Sep 15, 2017

Support for Android Gradle Plugin 3.x uber/okbuck#502

Closed

Update for Android Gradle plugin 2.5 #121

Update for Android Gradle plugin 2.5 #121

Conversation

rschiu commented Mar 31, 2017

adrien-aubel commented May 18, 2017

LouisCAD commented Jun 12, 2017

zhangkun83 commented Jun 12, 2017

lucastsantos commented Jun 19, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rschiu commented Jun 26, 2017

zhangkun83 commented Jun 27, 2017

LouisCAD commented Jun 27, 2017

zhangkun83 commented Jun 27, 2017

Choose a reason for hiding this comment

rschiu commented Jun 27, 2017

zhangkun83 commented Jun 27, 2017

rschiu commented Jun 27, 2017

LouisCAD commented Jul 1, 2017

LouisCAD commented Jul 12, 2017

rschiu commented Jul 12, 2017

rschiu commented Jul 12, 2017

zhangkun83 commented Jul 14, 2017

rschiu commented Jul 14, 2017

zhangkun83 commented Jul 14, 2017

rschiu commented Jul 20, 2017

nmulcahey commented Jul 21, 2017

rschiu commented Jul 21, 2017

zhangkun83 commented Jul 21, 2017

rschiu commented Jul 24, 2017

zhangkun83 commented Jul 24, 2017

rschiu commented Jul 24, 2017

rschiu commented Jul 25, 2017

zhangkun83 commented Jul 26, 2017