Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spark job running on GCP Dataproc: java.lang.ClassNotFoundException: org/eclipse/jetty/alpn/ALPN #2263

Closed
jresgar opened this issue Nov 1, 2019 · 2 comments
Closed

Spark job running on GCP Dataproc: java.lang.ClassNotFoundException: org/eclipse/jetty/alpn/ALPN #2263

jresgar opened this issue Nov 1, 2019 · 2 comments
Assignees
@jresgar
Labels
api: bigtable Issues related to the googleapis/java-bigtable-hbase API. type: question Request for information or clarification. Not an issue.

Comments

@jresgar
Copy link

jresgar commented Nov 1, 2019

I'm trying to run a Spark job on GCP Dataproc that stores to GCP Bigtable. The job works locally out of IntelliJ as well as from a local spark submit action. However, when submitting the Spark job in Dataproc in Spark Client or Cluster mode, I'm hitting some dependency conflict. I'm still pretty new to Java, Spark, and GCP, so I haven't been able to track down the cause of this DLL Hell. In a similar issue (#2140), some documentation (https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty) was posted that mentioned using grpc-netty-shaded when running out of Spark, but that didn't seem to work, unless I'm not excluding enough of the dependencies for the shaded dependency to take over. Any help is appreciated.

Relevant dependencies:

[INFO]    com.google.api.grpc:proto-google-cloud-bigtable-v2:jar:0.62.0:compile
[INFO]    com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:jar:0.62.0:compile
[INFO]    io.grpc:grpc-grpclb:jar:1.19.0:compile
[INFO]    com.google.api.grpc:grpc-google-cloud-bigtable-v2:jar:0.62.0:compile
[INFO]    com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:jar:0.62.0:compile
[INFO]    io.grpc:grpc-auth:jar:1.19.0:compile
[INFO]    com.google.api.grpc:proto-google-cloud-logging-v2:jar:0.57.0:compile
[INFO]    io.grpc:grpc-stub:jar:1.19.0:compile
[INFO]    io.grpc:grpc-context:jar:1.19.0:compile
[INFO]    com.google.cloud.bigtable:bigtable-hbase:jar:1.12.0:compile
[INFO]    com.google.api:gax-grpc:jar:1.44.0:compile
[INFO]    com.google.api.grpc:proto-google-common-protos:jar:1.15.0:compile
[INFO]    com.google.cloud.bigtable:bigtable-hbase-1.x:jar:1.12.0:compile
[INFO]    io.grpc:grpc-core:jar:1.23.0:compile
[INFO]    io.grpc:grpc-protobuf:jar:1.19.0:compile
[INFO]    com.google.cloud:google-cloud-core-grpc:jar:1.74.0:compile
[INFO]    com.google.api.grpc:grpc-google-common-protos:jar:1.16.0:compile
[INFO]    io.opencensus:opencensus-contrib-grpc-util:jar:0.21.0:compile
[INFO]    com.google.cloud:google-cloud-bigtable:jar:0.97.0:compile
[INFO]    io.grpc:grpc-api:jar:1.23.0:compile
[INFO]    com.google.api.grpc:proto-google-iam-v1:jar:0.12.0:compile
[INFO]    io.grpc:grpc-netty-shaded:jar:1.23.0:compile
[INFO]    com.google.cloud.bigtable:bigtable-client-core:jar:1.12.0:compile
[INFO]    io.grpc:grpc-alts:jar:1.19.0:compile

Initial error:

java.lang.ClassNotFoundException: org/eclipse/jetty/alpn/ALPN
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.JettyTlsUtil.isJettyAlpnConfigured(JettyTlsUtil.java:64)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.GrpcSslContexts.findJdkProvider(GrpcSslContexts.java:266)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:248)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:171)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.GrpcSslContexts.forClient(GrpcSslContexts.java:120)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.NettyChannelBuilder.buildTransportFactory(NettyChannelBuilder.java:436)
	at repackaged.com.google.bigtable.repackaged.io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:509)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.createNettyChannel(BigtableSession.java:662)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession$3.create(BigtableSession.java:527)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.io.ChannelPool.<init>(ChannelPool.java:244)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.createChannelPool(BigtableSession.java:544)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.createChannelPool(BigtableSession.java:530)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.createManagedPool(BigtableSession.java:556)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.getDataChannelPool(BigtableSession.java:353)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.<init>(BigtableSession.java:268)
	at org.apache.hadoop.hbase.client.AbstractBigtableConnection.<init>(AbstractBigtableConnection.java:147)
	at org.apache.hadoop.hbase.client.AbstractBigtableConnection.<init>(AbstractBigtableConnection.java:112)
	at repackaged.com.google.cloud.bigtable.hbase1_x.BigtableConnection.<init>(BigtableConnection.java:49)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at repackaged.com.google.cloud.bigtable.hbase.BigtableConfiguration.connect(BigtableConfiguration.java:183)
	at repackaged.com.google.cloud.bigtable.hbase.BigtableConfiguration.connect(BigtableConfiguration.java:159)
	at com.walmart.recordsale.ejTransactionConsumer$.connectToBigTable(ejTransactionConsumer.scala:96)
	at com.walmart.recordsale.ejTransactionConsumer$.runConsumer(ejTransactionConsumer.scala:150)
	at com.walmart.recordsale.ejTransactionConsumer$.main(ejTransactionConsumer.scala:282)
	at com.walmart.recordsale.ejTransactionConsumer.main(ejTransactionConsumer.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Exception in thread "main" java.lang.IllegalStateException: Could not find an appropriate constructor for repackaged.com.google.cloud.bigtable.hbase1_x.BigtableConnection
	at repackaged.com.google.cloud.bigtable.hbase.BigtableConfiguration.connect(BigtableConfiguration.java:186)
	at repackaged.com.google.cloud.bigtable.hbase.BigtableConfiguration.connect(BigtableConfiguration.java:159)
	at com.walmart.recordsale.ejTransactionConsumer$.connectToBigTable(ejTransactionConsumer.scala:96)
	at com.walmart.recordsale.ejTransactionConsumer$.runConsumer(ejTransactionConsumer.scala:150)
	at com.walmart.recordsale.ejTransactionConsumer$.main(ejTransactionConsumer.scala:282)
	at com.walmart.recordsale.ejTransactionConsumer.main(ejTransactionConsumer.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at repackaged.com.google.cloud.bigtable.hbase.BigtableConfiguration.connect(BigtableConfiguration.java:183)
	... 15 more
Caused by: java.lang.IllegalStateException: Could not find TLS ALPN provider; no working netty-tcnative, Conscrypt, or Jetty NPN/ALPN available
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:258)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:171)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.GrpcSslContexts.forClient(GrpcSslContexts.java:120)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.NettyChannelBuilder.buildTransportFactory(NettyChannelBuilder.java:436)
	at repackaged.com.google.bigtable.repackaged.io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:509)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.createNettyChannel(BigtableSession.java:662)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession$3.create(BigtableSession.java:527)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.io.ChannelPool.<init>(ChannelPool.java:244)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.createChannelPool(BigtableSession.java:544)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.createChannelPool(BigtableSession.java:530)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.createManagedPool(BigtableSession.java:556)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.getDataChannelPool(BigtableSession.java:353)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.<init>(BigtableSession.java:268)
	at org.apache.hadoop.hbase.client.AbstractBigtableConnection.<init>(AbstractBigtableConnection.java:147)
	at org.apache.hadoop.hbase.client.AbstractBigtableConnection.<init>(AbstractBigtableConnection.java:112)
	at repackaged.com.google.cloud.bigtable.hbase1_x.BigtableConnection.<init>(BigtableConnection.java:49)
	... 20 more

Secondary error:

java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative_x86_64, netty_tcnative]
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:104)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.handler.ssl.OpenSsl.loadTcNative(OpenSsl.java:526)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:93)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.GrpcSslContexts.defaultSslProvider(GrpcSslContexts.java:244)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:171)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.GrpcSslContexts.forClient(GrpcSslContexts.java:120)
	at repackaged.com.google.bigtable.repackaged.io.grpc.netty.NettyChannelBuilder.buildTransportFactory(NettyChannelBuilder.java:436)
	at repackaged.com.google.bigtable.repackaged.io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:509)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.createNettyChannel(BigtableSession.java:662)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession$3.create(BigtableSession.java:527)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.io.ChannelPool.<init>(ChannelPool.java:244)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.createChannelPool(BigtableSession.java:544)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.createChannelPool(BigtableSession.java:530)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.createManagedPool(BigtableSession.java:556)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.getDataChannelPool(BigtableSession.java:353)
	at repackaged.com.google.bigtable.repackaged.com.google.cloud.bigtable.grpc.BigtableSession.<init>(BigtableSession.java:268)
	at org.apache.hadoop.hbase.client.AbstractBigtableConnection.<init>(AbstractBigtableConnection.java:147)
	at org.apache.hadoop.hbase.client.AbstractBigtableConnection.<init>(AbstractBigtableConnection.java:112)
	at repackaged.com.google.cloud.bigtable.hbase1_x.BigtableConnection.<init>(BigtableConnection.java:49)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at repackaged.com.google.cloud.bigtable.hbase.BigtableConfiguration.connect(BigtableConfiguration.java:183)
	at repackaged.com.google.cloud.bigtable.hbase.BigtableConfiguration.connect(BigtableConfiguration.java:159)
	at com.walmart.recordsale.ejTransactionConsumer$.connectToBigTable(ejTransactionConsumer.scala:96)
	at com.walmart.recordsale.ejTransactionConsumer$.runConsumer(ejTransactionConsumer.scala:150)
	at com.walmart.recordsale.ejTransactionConsumer$.main(ejTransactionConsumer.scala:282)
	at com.walmart.recordsale.ejTransactionConsumer.main(ejTransactionConsumer.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
	Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: repackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative_linux_x86_64
		at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:224)
		at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:96)
		... 38 more
	Caused by: java.io.FileNotFoundException: META-INF/native/librepackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative_linux_x86_64.so
		at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:173)
		... 39 more
		Suppressed: java.lang.UnsatisfiedLinkError: no repackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative_linux_x86_64 in java.library.path
			at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
			at java.lang.Runtime.loadLibrary0(Runtime.java:870)
			at java.lang.System.loadLibrary(System.java:1122)
			at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
			at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:349)
			at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
			... 39 more
			Suppressed: java.lang.UnsatisfiedLinkError: no repackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative_linux_x86_64 in java.library.path
				at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
				at java.lang.Runtime.loadLibrary0(Runtime.java:870)
				at java.lang.System.loadLibrary(System.java:1122)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
				at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
				at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
				at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
				at java.lang.reflect.Method.invoke(Method.java:498)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
				at java.security.AccessController.doPrivileged(Native Method)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
				... 40 more
	Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: repackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative_linux_x86_64_fedora
		at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:224)
		at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:96)
		... 38 more
	Caused by: java.io.FileNotFoundException: META-INF/native/librepackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative_linux_x86_64_fedora.so
		at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:173)
		... 39 more
		Suppressed: java.lang.UnsatisfiedLinkError: no repackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative_linux_x86_64_fedora in java.library.path
			at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
			at java.lang.Runtime.loadLibrary0(Runtime.java:870)
			at java.lang.System.loadLibrary(System.java:1122)
			at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
			at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:349)
			at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
			... 39 more
			Suppressed: java.lang.UnsatisfiedLinkError: no repackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative_linux_x86_64_fedora in java.library.path
				at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
				at java.lang.Runtime.loadLibrary0(Runtime.java:870)
				at java.lang.System.loadLibrary(System.java:1122)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
				at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
				at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
				at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
				at java.lang.reflect.Method.invoke(Method.java:498)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
				at java.security.AccessController.doPrivileged(Native Method)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
				... 40 more
	Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: repackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative_x86_64
		at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:224)
		at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:96)
		... 38 more
	Caused by: java.io.FileNotFoundException: META-INF/native/librepackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative_x86_64.so
		at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:173)
		... 39 more
		Suppressed: java.lang.UnsatisfiedLinkError: no repackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative_x86_64 in java.library.path
			at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
			at java.lang.Runtime.loadLibrary0(Runtime.java:870)
			at java.lang.System.loadLibrary(System.java:1122)
			at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
			at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:349)
			at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
			... 39 more
			Suppressed: java.lang.UnsatisfiedLinkError: no repackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative_x86_64 in java.library.path
				at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
				at java.lang.Runtime.loadLibrary0(Runtime.java:870)
				at java.lang.System.loadLibrary(System.java:1122)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
				at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
				at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
				at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
				at java.lang.reflect.Method.invoke(Method.java:498)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
				at java.security.AccessController.doPrivileged(Native Method)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
				... 40 more
	Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: repackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative
		at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:224)
		at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:96)
		... 38 more
	Caused by: java.io.FileNotFoundException: META-INF/native/librepackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative.so
		at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:173)
		... 39 more
		Suppressed: java.lang.UnsatisfiedLinkError: no repackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative in java.library.path
			at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
			at java.lang.Runtime.loadLibrary0(Runtime.java:870)
			at java.lang.System.loadLibrary(System.java:1122)
			at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
			at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:349)
			at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136)
			... 39 more
			Suppressed: java.lang.UnsatisfiedLinkError: no repackaged_com_google_bigtable_repackaged_io_grpc_netty_shaded_netty_tcnative in java.library.path
				at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
				at java.lang.Runtime.loadLibrary0(Runtime.java:870)
				at java.lang.System.loadLibrary(System.java:1122)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
				at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
				at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
				at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
				at java.lang.reflect.Method.invoke(Method.java:498)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369)
				at java.security.AccessController.doPrivileged(Native Method)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361)
				at repackaged.com.google.bigtable.repackaged.io.grpc.netty.shaded.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339)
				... 40 more
@yoshi-automation yoshi-automation added triage me I really want to be triaged. 🚨 This issue needs some love. labels Nov 2, 2019
@crwilcox
Copy link

crwilcox commented Nov 6, 2019

@kolea2 to triage.

@kolea2 kolea2 added the type: question Request for information or clarification. Not an issue. label Nov 7, 2019
@yoshi-automation yoshi-automation removed 🚨 This issue needs some love. triage me I really want to be triaged. labels Nov 7, 2019
@jresgar
Copy link
Author

jresgar commented Nov 7, 2019

I think I was able to resolve the dependency issues. I'm now able to deploy my job on Dataproc with no errors.

@jresgar jresgar closed this as completed Nov 7, 2019
@google-cloud-label-sync google-cloud-label-sync bot added the api: bigtable Issues related to the googleapis/java-bigtable-hbase API. label Jan 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jresgar jresgar

Labels
api: bigtable Issues related to the googleapis/java-bigtable-hbase API. type: question Request for information or clarification. Not an issue.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@crwilcox @yoshi-automation @kolea2 @jresgar