[SPARK-31655][BUILD] Upgrade snappy-java to 1.1.7.5

[AngersZhuuuu]

What changes were proposed in this pull request?

snappy-java have release v1.1.7.5, upgrade to latest version.

Fixed in v1.1.7.4

• Caching internal buffers for SnappyFramed streams Fix org.scala-lang: * inconsistent versions #234
• Fixed the native lib for ppc64le to work with glibc 2.17 (Previously it depended on 2.22)

Fixed in v1.1.7.5

• Fixes java.lang.NoClassDefFoundError: org/xerial/snappy/pool/DefaultPoolFactory in 1.1.7.4

xerial/snappy-java@1.1.7.3...1.1.7.5

v 1.1.7.5 release note:
xerial/snappy-java@edc4ec2

Why are the changes needed?

Fix bug

Does this PR introduce any user-facing change?

No

How was this patch tested?

No need

[SparkQA]

Test build #122387 has finished for PR 28472 at commit ccb1946.

• This patch fails build dependency tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #122389 has finished for PR 28472 at commit ff742f5.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[maropu]

retest this please

[SparkQA]

Test build #122393 has finished for PR 28472 at commit ff742f5.

• This patch fails due to an unknown error code, -9.
• This patch merges cleanly.
• This patch adds no public classes.

[HeartSaVioR]

retest this, please

[HeartSaVioR]

LGTM.

[yaooqinn]

Test build #122389 has finished for PR 28472 at commit ff742f5.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

cc @gaborgsomogyi, KafkaDelegationTokenSuite seems still flaky for setting up Kerberos

[SparkQA]

Test build #122396 has finished for PR 28472 at commit ff742f5.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AngersZhuuuu]

retest this please

[gaborgsomogyi]

Test build #122389 has finished for PR 28472 at commit ff742f5.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

cc @gaborgsomogyi, KafkaDelegationTokenSuite seems still flaky for setting up Kerberos

I'll check whether it's possible to move setup into the test itself...

[SparkQA]

Test build #122400 has finished for PR 28472 at commit ff742f5.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HeartSaVioR]

I've taken a look at the failure before - that's a weird situation, as it succeeded to create client/[email protected] principal and keytab, but failed to authenticate.

[dongjoon-hyun]

+1, LGTM. Merged to master for Apache Spark 3.1.0.
Thank you all.

[gaborgsomogyi]

Guys, I've taken a look at the possibilities how to solve the mentioned KafkaDelegationTokenSuite issue and found a new approach. Going to file a jira + PR probably next week...

[nblxa]

Hello. What is necessary to have this fix packported to 2.4.x?

[dongjoon-hyun]

Hi, @nblxa . Do you hit any snappy issue in Apache Spark 2.4.5?

[nblxa]

Hi @dongjoon-hyun I’m hitting the glibc-2.22 issue with 2.4.3 on RHEL7 on Power due to the snappy-java dependency. I guess the same version of snappy-java is used in 2.4.5?

[HeartSaVioR]

It may be a regression for such architecture (RHEL7 & IBM power) starting from Spark 2.4.0 where snappy-java 1.1.7.1 is being introduced. What I heard is that snappy-java 1.1.2.6 (Spark 2.3.x) doesn't have a reference of glibc-2.22.

I'm not sure the architecture is in official support range, hence can't say whether it's a regression or not. It'd be nice to have this in Spark 3.0.0 at least even it doesn't belong to the official support.

EDIT: forgot to say, I got some report on broken parquet in Spark 2.4.5 + a on same architecture where it was working well with Spark 2.3.x + a.

[maropu]

Hi, @nblxa Thanks for the report. The bug should have been fixed in snappy-java v1.1.7.4, but branch-2.4 still uses 1.1.7.3. https://github.com/apache/spark/blob/branch-2.4/pom.xml#L164 .
I think it might be worth bumping up the version. cc: @dongjoon-hyun @holdenk

[AngersZhuuuu]

Hi, @nblxa Thanks for the report. The bug should have been fixed in snappy-java v1.1.7.4, but branch-2.4 still uses 1.1.7.3. https://github.com/apache/spark/blob/branch-2.4/pom.xml#L164 .
I think it might be worth bumping up the version. cc: @dongjoon-hyun @holdenk

Raise a pr later.

[dongjoon-hyun]

Got it. Thanks.