-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issues getting JSON credentials to work in JAR file #1502
Comments
I am experiencing the latter issue. I use the GOOGLE_APPLICATION_CREDENTIALS environment variable and call
I also checked added org.json as dependency in my pom.xml to see if a newer one (20160810) or the one that google-cloud-core project was using (20151123) and both are producing the same error as above. I am using maven and org.codehaus.mojo exec-maven-plugin 1.2.1 I am going to try to use the builder approach and manually set the credential values in the meantime. |
I figured it out. @jsnowacki , not sure if you are running into the same issue as me, but for me it turns out that I had a conflicting version of the org.json package. If this helps, this is what I ran into: one of my dependencies (https://github.com/jimmoores/quandl4j) depends on jackson-datatype-json-org ( https://github.com/jimmoores/quandl4j/blob/master/pom.xml#L303 ) which depends on a very old version of org.apache.geronimo.bundles -- this package provides the old implementation of org.json.JSONTokener , this outdated class does not have the constructor that accepts an InputStream as the parameter, it supports String or Reader -- However.. because this class depends on the InputStream version of the constructor ( https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/google-cloud-core/src/main/java/com/google/cloud/ServiceOptions.java#L452 ) , a java.lang.NoSuchMethodError exception is thrown. What I ended up doing is using a suggested work-around here: to make sure that I am not using an older version of org.json, I also added this to my pom.xml to make sure that only the latest version of org.json.JSONTokener is used, which includes the constructor with the InputStream as the parameter.
You probably have a similar problem with a different dependency in your project. also (you may need to find alternative commands/examples for SBT) Hopefully this helps! |
@jsnowacki can you confirm if @val-litvak 's suggestion works for you? |
@val-litvak @garrettjonesgoogle unfortunately this workaround does not work for me . It might be down to using SBT assembly correctly. Currently I did not have time to spent on this issue in more depth, but it is likely I will come back to it. The workaround I use at the moment is the import is done in Python, which is not ideal but works at the moment. |
@garrettjonesgoogle FatJar's have known issues. |
@lesv I can't quite tell why you posted a link relating to FatJar on this particular issue. |
It's looking like I was switching between issues and left the comment on the wrong one. Not quite sure how that happened. |
@garrettjonesgoogle It looks like a dependency issue since our json version is over 1 year old. Does it concern you to update our gcloud json version to the latest? |
@shinfan we should see what kind of changes to json that it implies. |
Marked as closed. This issue is caused by a dependency conflict introduced by an older org.json package used by an external dependency (also known as dependency hell). I think it should be the customers' responsibility of managing their dependency tree and there is not much we can do in google-cloud-java to prevent such issue. |
…) (#859) Fixes #1502 Source-Link: googleapis/synthtool@26ea255 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:7a9a7eb50ca2af5bfffab3abd2f38d408735c990a74bacf9b7fde2af0a086a0b
…) (#796) Fixes #1502 Source-Link: googleapis/synthtool@26ea255 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:7a9a7eb50ca2af5bfffab3abd2f38d408735c990a74bacf9b7fde2af0a086a0b
…) (#901) Fixes #1502 Source-Link: googleapis/synthtool@26ea255 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:7a9a7eb50ca2af5bfffab3abd2f38d408735c990a74bacf9b7fde2af0a086a0b
…) (#973) Fixes #1502 Source-Link: googleapis/synthtool@26ea255 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:7a9a7eb50ca2af5bfffab3abd2f38d408735c990a74bacf9b7fde2af0a086a0b
…) (#294) Fixes #1502 Source-Link: googleapis/synthtool@26ea255 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:7a9a7eb50ca2af5bfffab3abd2f38d408735c990a74bacf9b7fde2af0a086a0b
…) (#729) Fixes #1502 Source-Link: googleapis/synthtool@26ea255 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:7a9a7eb50ca2af5bfffab3abd2f38d408735c990a74bacf9b7fde2af0a086a0b
…) (#945) Fixes #1502 Source-Link: googleapis/synthtool@26ea255 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:7a9a7eb50ca2af5bfffab3abd2f38d408735c990a74bacf9b7fde2af0a086a0b
…) (#77) Fixes #1502 Source-Link: googleapis/synthtool@26ea255 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:7a9a7eb50ca2af5bfffab3abd2f38d408735c990a74bacf9b7fde2af0a086a0b
…) (#768) Fixes #1502 Source-Link: googleapis/synthtool@26ea255 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:7a9a7eb50ca2af5bfffab3abd2f38d408735c990a74bacf9b7fde2af0a086a0b
🤖 I have created a release *beep* *boop* --- ## [1.2.5](googleapis/java-artifact-registry@v1.2.4...v1.2.5) (2022-08-04) ### Documentation * **owlbot-java:** explaining why not using formatter in pom.xml ([#1511](https://github.com/googleapis/java-artifact-registry/issues/1511)) ([#294](googleapis/java-artifact-registry#294)) ([9c0e05c](googleapis/java-artifact-registry@9c0e05c)), closes [#1502](https://github.com/googleapis/java-artifact-registry/issues/1502) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [2.3.3](googleapis/java-scheduler@v2.3.2...v2.3.3) (2022-08-05) ### Documentation * **owlbot-java:** explaining why not using formatter in pom.xml ([#1511](https://github.com/googleapis/java-scheduler/issues/1511)) ([#796](googleapis/java-scheduler#796)) ([0b6e26f](googleapis/java-scheduler@0b6e26f)), closes [#1502](https://github.com/googleapis/java-scheduler/issues/1502) ### Dependencies * update dependency com.google.cloud:google-cloud-pubsub-bom to v1.120.10 ([#795](googleapis/java-scheduler#795)) ([39cd11f](googleapis/java-scheduler@39cd11f)) * update dependency com.google.cloud:google-cloud-shared-dependencies to v3 ([#792](googleapis/java-scheduler#792)) ([62a3aee](googleapis/java-scheduler@62a3aee)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [3.0.5](googleapis/java-recaptchaenterprise@v3.0.4...v3.0.5) (2022-08-04) ### Documentation * **owlbot-java:** explaining why not using formatter in pom.xml ([#1511](https://github.com/googleapis/java-recaptchaenterprise/issues/1511)) ([#859](googleapis/java-recaptchaenterprise#859)) ([cd266f5](googleapis/java-recaptchaenterprise@cd266f5)), closes [#1502](https://github.com/googleapis/java-recaptchaenterprise/issues/1502) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [3.4.0](googleapis/java-monitoring@v3.3.6...v3.4.0) (2022-08-06) ### Features * Added support for evaluating missing data in AlertPolicy ([#906](googleapis/java-monitoring#906)) ([e51e43c](googleapis/java-monitoring@e51e43c)) ### Documentation * **owlbot-java:** explaining why not using formatter in pom.xml ([#1511](https://github.com/googleapis/java-monitoring/issues/1511)) ([#901](googleapis/java-monitoring#901)) ([bb07735](googleapis/java-monitoring@bb07735)), closes [#1502](https://github.com/googleapis/java-monitoring/issues/1502) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [2.3.6](googleapis/java-tasks@v2.3.5...v2.3.6) (2022-08-05) ### Documentation * **owlbot-java:** explaining why not using formatter in pom.xml ([#1511](https://github.com/googleapis/java-tasks/issues/1511)) ([#729](googleapis/java-tasks#729)) ([ad9bcd2](googleapis/java-tasks@ad9bcd2)), closes [#1502](https://github.com/googleapis/java-tasks/issues/1502) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [3.6.7](googleapis/java-dlp@v3.6.6...v3.6.7) (2022-08-05) ### Documentation * **owlbot-java:** explaining why not using formatter in pom.xml ([#1511](https://github.com/googleapis/java-dlp/issues/1511)) ([#945](googleapis/java-dlp#945)) ([f29bd22](googleapis/java-dlp@f29bd22)), closes [#1502](https://github.com/googleapis/java-dlp/issues/1502) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [1.1.7](googleapis/java-optimization@v1.1.6...v1.1.7) (2022-08-06) ### Documentation * **owlbot-java:** explaining why not using formatter in pom.xml ([#1511](https://github.com/googleapis/java-optimization/issues/1511)) ([#77](googleapis/java-optimization#77)) ([7177414](googleapis/java-optimization@7177414)), closes [#1502](https://github.com/googleapis/java-optimization/issues/1502) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [4.8.0](googleapis/java-dialogflow@v4.7.5...v4.8.0) (2022-08-05) ### Features * deprecated the filter field and add resource_definition ([d58232f](googleapis/java-dialogflow@d58232f)) * provide new parameter cx_current_page, the unique identifier of the CX page to override the `current_page` in the session. Add filter field to ListAnswerRecordsRequest. And add AudioInput to analysis requests ([d58232f](googleapis/java-dialogflow@d58232f)) ### Dependencies * update dependency com.google.cloud:google-cloud-core to v2.8.8 ([#979](googleapis/java-dialogflow#979)) ([aee2eab](googleapis/java-dialogflow@aee2eab)) ### Documentation * add more meaningful comments ([d58232f](googleapis/java-dialogflow@d58232f)) * add more meaningful comments ([d58232f](googleapis/java-dialogflow@d58232f)) * added an explicit note that DetectIntentRequest's text input is limited by 256 characters ([d58232f](googleapis/java-dialogflow@d58232f)) * **owlbot-java:** explaining why not using formatter in pom.xml ([#1511](https://github.com/googleapis/java-dialogflow/issues/1511)) ([#973](googleapis/java-dialogflow#973)) ([3202efe](googleapis/java-dialogflow@3202efe)), closes [#1502](https://github.com/googleapis/java-dialogflow/issues/1502) * updated some method comments and added an explicit note that DetectIntentRequest's text input is limited by 256 characters ([#954](googleapis/java-dialogflow#954)) ([d58232f](googleapis/java-dialogflow@d58232f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [2.5.2](googleapis/java-container@v2.5.1...v2.5.2) (2022-09-16) ### Bug Fixes * Change vpc network ([#783](googleapis/java-container#783)) ([dcc91b6](googleapis/java-container@dcc91b6)) ### Documentation * **owlbot-java:** Explaining why not using formatter in pom.xml ([#1511](https://github.com/googleapis/java-container/issues/1511)) ([#768](googleapis/java-container#768)) ([a6c7635](googleapis/java-container@a6c7635)), closes [#1502](https://github.com/googleapis/java-container/issues/1502) ### Dependencies * Update dependency com.google.cloud:google-cloud-shared-dependencies to v3.0.3 ([#781](googleapis/java-container#781)) ([a1e886e](googleapis/java-container@a1e886e)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [2.3.15](https://togithub.com/googleapis/java-bigquerydatatransfer/compare/v2.3.14...v2.3.15) (2022-10-03) ### Dependencies * Update dependency cachetools to v5 ([#1511](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1511)) ([b95ccc8](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/b95ccc8d07c7fc5b6f50917560304bf7d1db8b02)) * Update dependency certifi to v2022.9.24 ([#1497](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1497)) ([8280ad9](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/8280ad9d94f44c41dbc57aef7007359fa43a65ec)) * Update dependency charset-normalizer to v2.1.1 ([#1501](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1501)) ([7f61538](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/7f6153862f189554d10b82b755dcccb9f24c3044)) * Update dependency click to v8.1.3 ([#1502](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1502)) ([d315f8e](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/d315f8e433a64f1a1e4673688991b807285f5435)) * Update dependency com.google.cloud:google-cloud-shared-dependencies to v3.0.4 ([#1523](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1523)) ([5eb622d](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/5eb622d8bcf05a9852c342a73d119e4d703a0c27)) * Update dependency com.google.protobuf:protobuf-java-util to v3.21.7 ([#1521](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1521)) ([7bb94b7](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/7bb94b735d7822337f5f6b671b425081f4822813)) * Update dependency gcp-releasetool to v1.8.8 ([#1498](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1498)) ([9801601](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/9801601eacc79432efa9ce243af8bda87bdeadb2)) * Update dependency google-cloud-core to v2.3.2 ([#1499](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1499)) ([1e73bff](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/1e73bfffe09f8ae7f5216e8e4cd3e7d4aa520893)) * Update dependency googleapis-common-protos to v1.56.4 ([#1500](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1500)) ([3b4b24c](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/3b4b24ccff9a199f78c8e48ea73e7c04f4e2bb5e)) * Update dependency jinja2 to v3.1.2 ([#1503](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1503)) ([4864061](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/48640616122833134b28e42bc689109849a77279)) * Update dependency keyring to v23.9.3 ([#1504](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1504)) ([cc523de](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/cc523de50fefe993dbfbedbebbbd5a9754071342)) * Update dependency markupsafe to v2.1.1 ([#1505](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1505)) ([1e0ba5e](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/1e0ba5e094f98858affb53f77ceb61eff677b7cd)) * Update dependency protobuf to v3.20.2 ([#1506](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1506)) ([7a1e267](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/7a1e267ce2a02b9c506769de09bfefc76d103cb8)) * Update dependency protobuf to v4 ([#1512](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1512)) ([12c0c67](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/12c0c67819a05f4bafab1c280fd3a8b15ee0193b)) * Update dependency pyjwt to v2.5.0 ([#1507](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1507)) ([98f2d58](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/98f2d580e1a78aeeebbbc5db8d2b99461255b86f)) * Update dependency requests to v2.28.1 ([#1508](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1508)) ([9906ca8](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/9906ca8c8a67390106b524d561f1e973126aecfa)) * Update dependency typing-extensions to v4.3.0 ([#1509](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1509)) ([2d7eb9f](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/2d7eb9f93181fbf183ae2d633fdc63748c82110e)) * Update dependency zipp to v3.8.1 ([#1510](https://togithub.com/googleapis/java-bigquerydatatransfer/issues/1510)) ([dd98eef](https://togithub.com/googleapis/java-bigquerydatatransfer/commit/dd98eef23cdbac8b9c365504c2456776bed786a9)) --- This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
I am trying to download some files from Google Cloud Storage to HDFS in a Spark (2.0.2) driver and then process the files in Spark. Everything works fine executed locally but not if I compile it into fat jar and deploy on Spark in YARN client mode (driver works locally); in both cases pulling is from the same cloud storage bucket using the same JSON credentials file. When I use the credentials from InputStream:
I gent the below error:
I also tried with GOOGLE_APPLICATION_CREDENTIALS set and
In this case I get different error:
Just to clarify, the credentials JSON file and org.json package are both in the JAR. Also, org.json has this method and seems to be in the right version inside JAR.
The build is done via SBT. I use sbt-assenbly plugin to build fat jar. GC Java library is in version 0.8.0:
Also source and target is set to Java 8:
The text was updated successfully, but these errors were encountered: