Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Register the correct native library for Snappy, during native image generation #16069

Merged
merged 1 commit into from
Mar 28, 2021
Merged

Conversation

jaikiran
Copy link
Member

Fixes #16042

The commit here fixes the issue where the Snappy native library that was being registered as a native image resource was using the wrong resource path (it was using org/xerial/snappy/native/Linux/amd64/libsnappyjava.so instead of org/xerial/snappy/native/Linux/x86_64/libsnappyjava.so). The commit here uses the OSInfo API from the snappy library to get hold of the correct path (just like we do in the io.quarkus.kafka.client.runtime.KafkaRecorder#loadSnappy()) and thus letting that API deal with the details.

P.S: This integration-tests/kafka-client fails even with the latest released version of GraalVM, so this isn't related to any GraalVM dev version. It looks like this native-image tests isn't enabled in our CI, which is why this wasn't caught earlier. Perhaps we should enable these tests in our CI (as a separate PR)?

@jaikiran
Copy link
Member Author

jaikiran commented Mar 27, 2021

It looks like this native-image tests isn't enabled in our CI, which is why this wasn't caught earlier. Perhaps we should enable these tests in our CI (as a separate PR)?

I take that back. Looks like this is indeed enabled in our CI runs https://github.com/quarkusio/quarkus/blob/main/.github/native-tests.json#L68 (I see kafka-snappy in there). I don't remember the CI setup details. Do the native image generation for these tests happen in a container? If so, then that explains why it isn't reproducible there. There's a specific check in the KafkaProcessor which sets up the correct resource path, if the deployment processing code is running in a container[1]

[1] https://github.com/quarkusio/quarkus/blob/main/extensions/kafka-client/deployment/src/main/java/io/quarkus/kafka/client/deployment/KafkaProcessor.java#L196

Copy link
Member

@cescoffier cescoffier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just made a comment. It looks good to me.

@@ -199,7 +200,7 @@ private void handleSnappy(BuildProducer<ReflectiveClassBuildItem> reflectiveClas
String path = root + dir + "/" + snappyNativeLibraryName;
nativeLibs.produce(new NativeImageResourceBuildItem(path));
} else { // otherwise the native lib of the platform this build runs on
String dir = getOs() + "/" + getArch();
String dir = OSInfo.getNativeLibFolderPathForCurrentOS();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is getOS and getArch still used?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good you asked :) I hadn't noticed that with this change, these methods will now be unused. I've updated this PR to remove them altogether.

@cescoffier cescoffier merged commit 40ee586 into quarkusio:main Mar 28, 2021
@quarkus-bot quarkus-bot bot added this to the 1.14 - main milestone Mar 28, 2021
@cescoffier cescoffier added the kind/bug Something isn't working label Mar 28, 2021
@jaikiran jaikiran deleted the qk-16042 branch March 28, 2021 05:45
@gsmet gsmet modified the milestones: 1.14 - main, 1.13.1.Final Apr 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/kafka kind/bug Something isn't working
Projects
None yet
3 participants