Protobuf related build issue with Tink + Firebase (Android)

[kinex]

I am trying to integrate Tink to my Flutter app which also uses Firebase. With 1.3.0-rc3 I get this build errors in Android:

Execution failed for task ':app:checkDevelopmentDebugDuplicateClasses'.
> 1 exception was raised by workers:
  java.lang.RuntimeException: java.lang.RuntimeException: Duplicate class com.google.protobuf.AbstractMessageLite found in modules protobuf-javalite-3.10.0.jar (com.google.protobuf:protobuf-javalite:3.10.0) and protobuf-lite-3.0.1.jar (com.google.protobuf:protobuf-lite:3.0.1)
  Duplicate class com.google.protobuf.AbstractMessageLite$Builder found in modules protobuf-javalite-3.10.0.jar (com.google.protobuf:protobuf-javalite:3.10.0) and protobuf-lite-3.0.1.jar (com.google.protobuf:protobuf-lite:3.0.1)
  Duplicate class com.google.protobuf.AbstractMessageLite$Builder$LimitedInputStream found in modules protobuf-javalite-3.10.0.jar (com.google.protobuf:protobuf-javalite:3.10.0) and protobuf-lite-3.0.1.jar (com.google.protobuf:protobuf-lite:3.0.1)
  Duplicate class com.google.protobuf.AbstractParser found in modules protobuf-javalite-3.10.0.jar (com.google.protobuf:protobuf-javalite:3.10.0) and protobuf-lite-3.0.1.jar (com.google.protobuf:protobuf-lite:3.0.1)
  Duplicate class com.google.protobuf.AbstractProtobufList found in modules protobuf-javalite-3.10.0.jar (com.google.protobuf:protobuf-javalite:3.10.0) and protobuf-lite-3.0.1.jar (com.google.protobuf:protobuf-lite:3.0.1)
  Duplicate class com.google.protobuf.Any found in modules classes.jar (com.google.firebase:protolite-well-known-types:17.0.0) and protobuf-javalite-3.10.0.jar (com.google.protobuf:protobuf-javalite:3.10.0)
.
.
.

I have tried various suggestions mentioned in other protobuf related issues here but no solution found so far.

Issue disappears if I downgrade Tink to version 1.3.0-rc1, so I am using it as a workaround for now.

[chuckx]

In between 1.3.0-rc1 and 1.3.0-rc3, we migrated from protobuf-lite to protobuf-javalite.

From the protobuf side, this is a major and incompatible upgrade, as stated in the protobuf-javalite documentation:

Note that the latest version of Java Lite is not compatible with the 3.0.0 version.

For reference, it also contains a link to the documentation of protobuf-lite, referring to it as "the older version of Java Lite (v3.0.0)".

Can you share some more context around your dependencies? Are you directly depending on protobuf-lite 3.0.1 (the old version) or is it an indirect dependency?

[kinex]

I am not depending on it directly. It seems all dependencies come from Firebase packages (cloud_firestore, firebase_remote_config). Here are some snippets from the dependencies in my app:

....

+--- project :cloud_firestore
|    +--- com.google.firebase:firebase-firestore:21.3.0
|    |    +--- androidx.annotation:annotation:1.1.0
|    |    +--- com.google.android.gms:play-services-base:17.0.0
|    |    |    +--- androidx.collection:collection:1.0.0 -> 1.1.0 (*)
|    |    |    +--- androidx.core:core:1.0.0 -> 1.1.0 (*)
|    |    |    +--- androidx.fragment:fragment:1.0.0 -> 1.1.0 (*)
|    |    |    +--- com.google.android.gms:play-services-basement:17.0.0
|    |    |    |    +--- androidx.collection:collection:1.0.0 -> 1.1.0 (*)
|    |    |    |    +--- androidx.core:core:1.0.0 -> 1.1.0 (*)
|    |    |    |    \--- androidx.fragment:fragment:1.0.0 -> 1.1.0 (*)
|    |    |    \--- com.google.android.gms:play-services-tasks:17.0.0
|    |    |         \--- com.google.android.gms:play-services-basement:17.0.0 (*)
|    |    +--- com.google.android.gms:play-services-basement:17.0.0 (*)
|    |    +--- com.google.android.gms:play-services-tasks:17.0.0 (*)
|    |    +--- com.google.firebase:firebase-auth-interop:18.0.0
|    |    |    +--- com.google.android.gms:play-services-base:17.0.0 (*)
|    |    |    +--- com.google.android.gms:play-services-basement:17.0.0 (*)
|    |    |    +--- com.google.android.gms:play-services-tasks:17.0.0 (*)
|    |    |    \--- com.google.firebase:firebase-common:18.0.0 -> 19.3.0
|    |    |         +--- com.google.android.gms:play-services-basement:17.0.0 (*)
|    |    |         +--- com.google.android.gms:play-services-tasks:17.0.0 (*)
|    |    |         +--- com.google.auto.value:auto-value-annotations:1.6.5
|    |    |         \--- com.google.firebase:firebase-components:16.0.0
|    |    |              \--- androidx.annotation:annotation:1.1.0
|    |    +--- com.google.firebase:firebase-common:19.3.0 (*)
|    |    +--- com.google.firebase:firebase-database-collection:17.0.0
|    |    +--- com.google.firebase:protolite-well-known-types:17.0.0
|    |    |    \--- com.google.protobuf:protobuf-lite:3.0.1
|    |    +--- com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
|    |    +--- com.squareup.okhttp:okhttp:2.7.5
|    |    |    \--- com.squareup.okio:okio:1.6.0 -> 1.15.0
|    |    +--- io.grpc:grpc-android:1.21.0
|    |    +--- io.grpc:grpc-okhttp:1.21.0
|    |    |    +--- io.grpc:grpc-core:[1.21.0] -> 1.21.0
|    |    |    |    +--- io.opencensus:opencensus-api:0.21.0
|    |    |    |    +--- io.grpc:grpc-api:[1.21.0] -> 1.21.0
|    |    |    |    |    +--- io.grpc:grpc-context:1.21.0
|    |    |    |    |    +--- com.google.errorprone:error_prone_annotations:2.3.2
|    |    |    |    |    +--- com.google.code.findbugs:jsr305:3.0.2
|    |    |    |    |    +--- org.codehaus.mojo:animal-sniffer-annotations:1.17
|    |    |    |    |    \--- com.google.guava:guava:26.0-android
|    |    |    |    |         +--- org.checkerframework:checker-compat-qual:2.5.2
|    |    |    |    |         \--- com.google.j2objc:j2objc-annotations:1.1
|    |    |    |    +--- com.google.code.gson:gson:2.7 -> 2.8.5
|    |    |    |    +--- com.google.android:annotations:4.1.1.4
|    |    |    |    \--- io.opencensus:opencensus-contrib-grpc-metrics:0.21.0
|    |    |    |         \--- io.opencensus:opencensus-api:0.21.0
|    |    |    +--- com.squareup.okhttp:okhttp:2.5.0 -> 2.7.5 (*)
|    |    |    \--- com.squareup.okio:okio:1.13.0 -> 1.15.0
|    |    +--- io.grpc:grpc-protobuf-lite:1.21.0
|    |    |    +--- io.grpc:grpc-api:1.21.0 (*)
|    |    |    +--- com.google.protobuf:protobuf-lite:3.0.1
|    |    |    \--- com.google.guava:guava:26.0-android (*)
|    |    \--- io.grpc:grpc-stub:1.21.0
|    |         \--- io.grpc:grpc-api:1.21.0 (*)
|    \--- io.flutter:flutter_embedding_debug:1.0.0-2994f7e1e682039464cb25e31a78b86a3c59b695 (*)

....

+--- project :firebase_remote_config
|    +--- com.google.firebase:firebase-config:19.0.3
|    |    +--- com.google.firebase:firebase-abt:19.0.0
|    |    |    +--- com.google.android.gms:play-services-basement:17.0.0 (*)
|    |    |    +--- com.google.firebase:firebase-common:19.0.0 -> 19.3.0 (*)
|    |    |    +--- com.google.firebase:firebase-measurement-connector:18.0.0
|    |    |    |    \--- com.google.android.gms:play-services-basement:17.0.0 (*)
|    |    |    \--- com.google.protobuf:protobuf-lite:3.0.1
|    |    +--- com.google.firebase:firebase-common:19.2.0 -> 19.3.0 (*)
|    |    +--- com.google.firebase:firebase-iid:18.0.0
|    |    |    +--- com.google.android.gms:play-services-basement:16.0.1 -> 17.0.0 (*)
|    |    |    +--- com.google.android.gms:play-services-stats:16.0.1
|    |    |    |    \--- com.google.android.gms:play-services-basement:16.0.1 -> 17.0.0 (*)
|    |    |    +--- com.google.android.gms:play-services-tasks:16.0.1 -> 17.0.0 (*)
|    |    |    +--- com.google.firebase:firebase-common:17.0.0 -> 19.3.0 (*)
|    |    |    \--- com.google.firebase:firebase-iid-interop:16.0.1
|    |    |         +--- com.google.android.gms:play-services-base:16.0.1 -> 17.0.0 (*)
|    |    |         \--- com.google.android.gms:play-services-basement:16.0.1 -> 17.0.0 (*)
|    |    +--- com.google.firebase:firebase-measurement-connector:18.0.0 (*)
|    |    \--- com.google.protobuf:protobuf-lite:3.0.1
|    \--- io.flutter:flutter_embedding_debug:1.0.0-2994f7e1e682039464cb25e31a78b86a3c59b695 (*)

....

+--- com.google.firebase:protolite-well-known-types:{strictly 17.0.0} -> 17.0.0 (c)
+--- io.grpc:grpc-protobuf-lite:{strictly 1.21.0} -> 1.21.0 (c)
+--- com.google.protobuf:protobuf-lite:{strictly 3.0.1} -> 3.0.1 (c)

....

+--- project :flutter_encryption
|    +--- io.flutter:flutter_embedding_debug:1.0.0-2994f7e1e682039464cb25e31a78b86a3c59b695 (*)
|    +--- org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.3.50 (*)
|    +--- com.github.tozny:java-aes-crypto:1.1.0
|    \--- com.google.crypto.tink:tink-android:1.3.0-rc3
|         \--- com.google.protobuf:protobuf-javalite:3.11.1

....

+--- com.google.protobuf:protobuf-javalite:{strictly 3.11.1} -> 3.11.1 (c)

....

[chuckx]

Thanks for the additional detail. I'm looking into how to best accommodate this situation.

[chuckx]

FYI, we ended up reaching out to the Firebase team and they are working on upgrading their dependency. Please see this issue to track their progress.

In the meantime, we're going to move forward with finalizing the 1.3.0 release with the protobuf-javalite dependency as is.

Looking ahead, we're investigating options for shading the dependency to prevent such conflicts in the future. However, we need to figure out an approach with minimal compatibility issues and verify that it won't bloat the Android release artifacts too much.

[kinex]

OK. I will stick to 1.3.0-rc1 until they fix it. I have not noticed any issues at least so far (my beta app is already in Google Play+AppStore). Thanks!

[thaidn]

Thanks for your understanding. Long-term speaking, we're going to completely remove the protobuf dependency.

I guess this can be closed for now, please reopen if you run into any other issues.

[mohamedagamy]

Duplicate class com.google.protobuf.AbstractMessageLite found in modules protobuf-javalite-3.11.1.jar (com.google.protobuf:protobuf-javalite:3.11.1) and protobuf-lite-3.0.1.jar (com.google.protobuf:protobuf-lite:3.0.1)
Note that I was using firebase , remote config
and the issue solved by removing remote config
so how can I make both of them work together.

[thaidn]

@mohamedagamy The Firebase team is working on upgrading their dependency. Please see this issue to track their progress.

In the meantime, @kinex said that 1.3.0-rc1 works fine with Firebase. Have you tried that?

[thaidn]

Please reopen this if my last comment doesn't help you. We'll get to the bottom of this protobuf issue in 1.4.0 release and in a future release we'll completely remove protobuf. Thank you for your patience!

[mgray88]

#253
https://issuetracker.google.com/issues/154434594
Linking these issues in case anyone else runs into this issue with androidx.security:security-crypto:1.0.0-rc01

[thaidn]

This is hopefully fixed in HEAD-SNAPSHOT. Please try it out to see if it helps. Instructions below.

1. Add to your Gradle the below config

repositories {
    maven { url "https://oss.sonatype.org/content/repositories/snapshots" }
}

dependencies {
    implementation 'com.google.crypto.tink:tink-android:HEAD-SNAPSHOT'
}

2. Remove other versions of Tink from your Gradle config

3. Recompile and... profit?

[mohamedagamy]

sorry for being late but I have updated Gradle to be like this and it is working :)
The following gradle configuration fixed the issue 👍
def firebase_config = '19.1.0'
implementation("com.google.firebase:firebase-config:$firebase_config", {
// protobuf-lite that firebase-config and Tink depends on.
// protobuf-lite:3.0.1 , protobuf-javalite:3.11.1'
//exclude group: 'com.google.protobuf'
exclude group: 'com.google.protobuf', module: 'protobuf-javalite'
exclude group: 'com.google.protobuf', module: 'protobuf-lite'
})

implementation 'com.google.crypto.tink:tink-android:HEAD-SNAPSHOT'

[amytang0]

Can confirm this also works for our case. Any rough timelines on when this change will release officially?

[thaidn]

We plan to release 1.4.0-RC2 by the end of this week.

[thaidn]

https://github.com/google/tink/releases/tag/v1.4.0-rc2 is here. Please upgrade and let us know if you encounter any issues.

[ilexbor]

implementation "com.google.crypto.tink:tink-android:1.4.0-rc2"

FATAL EXCEPTION: main
Process: PID: 31461
java.lang.ExceptionInInitializerError
at r.k.c.a.d0.a.a(Unknown Source:0)
at kotlin.controller.Application.onCreate(:5)
at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1189)
at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6549)
at android.app.ActivityThread.access$1500(ActivityThread.java:226)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1907)
at android.os.Handler.dispatchMessage(Handler.java:107)
at android.os.Looper.loop(Looper.java:224)
at android.app.ActivityThread.main(ActivityThread.java:7509)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:539)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:995)
Caused by: java.lang.RuntimeException: Field configName_ for r.k.c.a.j0.g3 not found. Known fields are [public static final r.k.c.a.j0.g3 r.k.c.a.j0.g3.j, public static volatile r.k.c.a.k0.a.z0 r.k.c.a.j0.g3.k]
at r.k.c.a.k0.a.u0.a(Unknown Source:62)
at r.k.c.a.k0.a.u0.a(:43)
at r.k.c.a.k0.a.u0.a(Unknown Source:19)
at r.k.c.a.k0.a.k0.a(:23)
at r.k.c.a.k0.a.c1.a(Unknown Source:19)
at r.k.c.a.k0.a.c1.a(Unknown Source:4)
at r.k.c.a.k0.a.z$a.a(:2)
at r.k.c.a.d0.a.(Unknown Source:6)
... 12 more

[thaidn]

@ilexbor a workaround can be found here #361 (comment). It'll be included in 1.4.0 release which is coming very soon.

[jaafarnejm]

This error means that you are importing two packages that use Protobuf for your project, one of them has a distribution that's conflicting with the other.

If you encountered this problem on Flutter, you can reconsider the version number of your dependencies in pubspec.yaml, and replace "any" by an exact version number.
For example, under "dependencies:" change:

barcode_scan: any

To:

barcode_scan: ^2.0.0

I hope I helped.