-
Notifications
You must be signed in to change notification settings - Fork 434
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GLUTEN-5341]Fix test write parquet with compression codec #5424
Conversation
Run Gluten Clickhouse CI |
@JkSelf Could you please review this PR |
Run Gluten Clickhouse CI |
The tests won't pass till issue #5427 gets fixed as reading this codec is not supported in velox. I will disable these tests. |
Run Gluten Clickhouse CI |
Run Gluten Clickhouse CI |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch. Thanks.
===== Performance report for TPCH SF2000 with Velox backend, for reference only ====
|
[GLUTEN-5341] Fix test write parquet with compression codec
What changes were proposed in this pull request?
LZ4_RAW is currently not supported in velox, so fallback to non-native write when user specifies this compression codec.
It got introduced in spark 3.5 https://issues.apache.org/jira/browse/SPARK-43273
(Partially Fixes: #5341)
How was this patch tested?
Ran UTs