Tokenizers maven dependency fails to compile on CentOS 7

[priyadarshi9]

Description

I am using the tokenizers dependency in my project.

<dependency>
	<groupId>ai.djl.huggingface</groupId>
	<artifactId>tokenizers</artifactId>
	<version>0.18.0</version>
</dependency>

The compilation is failing with the following error:

Caused by: java.lang.UnsatisfiedLinkError: /root/.djl.ai/tokenizers/0.12.0-0.18.0-linux-x86_64/libtokenizers.so: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by /root/.djl.ai/tokenizers/0.12.0-0.18.0-linux-x86_64/libtokenizers.so)

Expected Behavior

The compilation should proceed successfully without any such errors.

Error Message

Caused by: java.lang.UnsatisfiedLinkError: /root/.djl.ai/tokenizers/0.12.0-0.18.0-linux-x86_64/libtokenizers.so: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by /root/.djl.ai/tokenizers/0.12.0-0.18.0-linux-x86_64/libtokenizers.so)

I ran this command to check the GLIBC version on the system and it threw out following:

(base) [root@deep02 lib64]# ldd --version

ldd (GNU libc) 2.17
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

What can I do to fix this error? Please help with some suggestions.

[frankfliu]

@priyadarshi9
I updated our CI to build tokenizer on centos 7, see: #1874

once this PR is merged, you can try our 0.19.0-SNAPSHOT build.

Please make sure clean up your local cache folder:

rm -rf ~/.djl.ai/tokenizer

[priyadarshi9]

Hey @frankfliu , Thank you for a quick response. I could not find 0.19.0-SNAPSHOT version in maven repository. Could you please tell me how can I get this version?

[frankfliu]

You need add snapshot repo, you can find document from here

You can also see our example: https://github.com/deepjavalibrary/djl/blob/master/examples/pom.xml#L17-L22

[priyadarshi9]

Thanks, I could get the SNAPSHOT version now. Will try building on CentOS 7.

[frankfliu]

the nightly build is not started yet. Please wait for 2 hours and try again

[priyadarshi9]

Sure, thanks!