2.14.0 Regression: GraalVM Native Image fails #1751
Comments
product-auto-label
bot
added
the
api: storage
BenWhitehead
added
priority: p1
|
Thank you for bringing this to my attention. I spent some time today investigating what we can do to improve the situation. Unfortunately, we can't use the same solution here as was implemented for compute in googleapis/java-compute#721. In our case, as of 2.14.0 java-storage does need many of the grpc modules in it's dependencies, so we can't exclude them. Searching around the web it seems like this type of error isn't all that uncommon, and unfortunately is not always one size fits all to address given the complexities of logging frameworks in the Java ecosystem. We do have CI setup to test compatibility with native-image -- both for jdk 11 and jdk 17 -- however, we do not have any logging frameworks -- in particular Slf4J -- in use in any of our code, so we are unable to trigger this specific failure. In fact nothing in our entire dependency chain declares a dependency on Slf4J, this happens do to Netty's logging compatibility trying to use the logger it discovers on the class path. Your means of fixing this error, are to define a
|
BenWhitehead
added
priority: p2
|
Thank you for the workaround. I can confirm it works. However for a minor version upgrade it doesn't feel right to be forced to adjust my own config due to transitive dependency issues on your end. I suspect I won't be the only one facing this over time and this will create a steady support load for you. Is there a way to opt-out of GRPC completely and use REST instead? And if so, which transitive dependencies can then be safely excluded? |
|
You're correct it's not ideal, after talking with some folks on the broader java libraries team, unfortunately this issues is not unique to storage and compute. I've asked them to look into a better long term solution for this scenario. I'll have to do some deeper investigation as to how difficult it would be to remove gRPC dependencies. In general I'm a bit surprised this is happening when using the existing JSON based implementation as we shouldn't be initializing gRPC specific classes unless necessary. Would it be at all possible for you to share with me how you're initializing your Apologies for this causing havoc for you on the minor release. |
|
It's actually about as trivial as it gets: To ensure the transport selection without incurring unnecessary dependencies, I see two ways that could work:
This would make it trivial to exclude
In this case the user would choose which artifact to include. Currently 1. is more orthogonal to http client selection where one presently also needs to exclude unnecessary dependencies such as |
|
Thanks for sharing, I'll do some investigation and see what I can figure out. Unfortunately, at this time we aren't able to restructure the modules as there are common middleware classes which are shared between http and grpc but are not public. Most likely it will end up taking the form of In the case of "only want grpc" there is actually only 1 dependency that would be dropped, as the rest of the http middleware is used by the other identity and auth middleware. |
|
I was unsatisfied with my dependencies increasing by 30MB with 2.14.0 despite not needing or wanting gRPC, but I found that the situation can be brought under control by excluding:
And then reincluding On a more general note, it doesn't seem like the client should depend on a shaded copy of Netty. This will result in duplicate Netty classes in any application which already uses Netty, directly or transitively. It's easy enough to work around but still a bit of a landmine for the unwary. |
|
I spent some time today trying to pair things down as close to 2.13.x and got the following exclusion list. I was able to successfully run our full JSON integration tests suite with these exclusions.
The shaded version of Netty for gRPC was not a light decision made by them, but was done in order to ensure compatibility between the version of Netty used by gRPC. There are many places gRPC -- and Google Cloud Libraries -- is deployed which do not have the ability to change the version of Netty packaged in their platforms/frameworks. |
|
And @axelfontaine a small note on this: we haven't adopted our configuration files for GraalVM 22.3 yet. This not fully related to the error you mentioned but just wanted to provide a heads up since you mentioned GraalVM 22.3.0 in the issue description. |
|
@BenWhitehead @axelfontaine - I have updated an app from GraalVm, 22.2 to 22.3 (and Spring Boot 3 which requires it) and the previous codebase does not build anymore. Not using gRPC or Netty directly. |
|
A quick update on this folks, we're keeping this issue open as we continue to look into the slf4j compatibility issue. To close the loop on #1751 (comment), google-cloud-storage should be compatible with GraalVM 22.3 as of 2.16.0. For any issues related to specifically GraalVM 22.3, please create a new issue. |
|
Thanks again for reporting this issue! We've added a Common Issues section in our documentation page which now contains more information and suggested workarounds on the SLF4J compatibility issue. Please take a look at it if you are still blocked on this issue. |
On the same project as in googleapis/gax-java#1781
Building a native image with GraalVM 22.3.0, Java 17, google-cloud-storage 2.13.1 and google-cloud-compute 1.13.0 works great.
Bumping google-cloud-storage to 2.14.0 (no other changes) causes the native image build to fail with:
1:
2:
3:
The text was updated successfully, but these errors were encountered: