Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test case for ORC statistics test [databricks] #8995

Merged
merged 2 commits into from
Aug 11, 2023

Conversation

jlowe
Copy link
Member

@jlowe jlowe commented Aug 11, 2023

Fixes #8763. This restores the OrcSuite test that was reverted in #8983 by adding the new datagen module as a dependency and moving the test from the integration_tests module to the tests module.

Signed-off-by: Chong Gao <[email protected]>
Co-authored-by: Chong Gao <[email protected]>
Signed-off-by: Jason Lowe <[email protected]>
@jlowe jlowe added the test Only impacts tests label Aug 11, 2023
@jlowe jlowe requested a review from res-life August 11, 2023 00:24
@jlowe jlowe self-assigned this Aug 11, 2023
@jlowe jlowe changed the title Add test case for ORC statistics test (#8940) Add test case for ORC statistics test [databricks] Aug 11, 2023
@jlowe
Copy link
Member Author

jlowe commented Aug 11, 2023

build

@res-life
Copy link
Collaborator

Found build error:

Error: 8-11T01:02:46.563Z] [2023-08-11T01:02:35.710Z] [ERROR] /home/ubuntu/spark-rapids/tests/src/test/scala/com/nvidia/spark/rapids/OrcSuite.scala:183: Class org.apache.hadoop.hive.common.type.HiveDecimal not found - continuing with a stub.

Pushed a new commit to add hive-storage-api dependency.

@res-life
Copy link
Collaborator

build

@res-life
Copy link
Collaborator

The reason why I caused this issue is: I used -DallowConventionalDistJar argument to fast the compile.

mvn clean install -DallowConventionalDistJar=true -Dbuidver=330

So I compiled successfully locally.

@jlowe jlowe merged commit 47db3cb into NVIDIA:branch-23.10 Aug 11, 2023
27 checks passed
@jlowe jlowe deleted the restore-orcsuite branch August 11, 2023 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test Only impacts tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Statistics tests for ORC files written by GPU
3 participants