Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support table redirections from Hive to Hudi #15551

Merged
merged 2 commits into from
Jan 17, 2023

Conversation

mx123
Copy link
Contributor

@mx123 mx123 commented Dec 29, 2022

Description

Allow users to run queries referencing a single hive catalog to access both normal hive tables and hudi tables with new optimised+dedicated connector (provided an admin has configured hive.hudi-catalog-name to point to a hudi catalog).

Original pull request: #14349

Fixes #14750

@cla-bot cla-bot bot added the cla-signed label Dec 29, 2022
@mx123 mx123 changed the title Hive hudi redirections Support table redirections from Hive to Hudi Dec 29, 2022
@mx123 mx123 force-pushed the hive_hudi_redirections branch 5 times, most recently from 3e8f4ee to dffddb1 Compare December 30, 2022 14:40
@mx123 mx123 marked this pull request as ready for review December 30, 2022 14:43
@mx123 mx123 requested a review from findinpath December 30, 2022 14:43
@findinpath
Copy link
Contributor

Build is red during maven-checks

Error:  src/main/java/io/trino/plugin/hive/HiveMetadata.java:[280,1] (imports) ImportOrder: Wrong order for 'io.trino.plugin.hive.util.HiveUtil.isHudiTable' import.

Also please rebase on master because in the meantime a new version of Hudi 0.12.2 is being used.

@mx123 mx123 force-pushed the hive_hudi_redirections branch 4 times, most recently from e2a79a6 to 23e6292 Compare January 3, 2023 11:46
@mx123
Copy link
Contributor Author

mx123 commented Jan 3, 2023

Build is red during maven-checks

Error:  src/main/java/io/trino/plugin/hive/HiveMetadata.java:[280,1] (imports) ImportOrder: Wrong order for 'io.trino.plugin.hive.util.HiveUtil.isHudiTable' import.

fixed

Also please rebase on master because in the meantime a new version of Hudi 0.12.2 is being used.

done

@mx123 mx123 force-pushed the hive_hudi_redirections branch from 23e6292 to 2e79834 Compare January 3, 2023 15:51
@findinpath
Copy link
Contributor

Please squash the first 2 commits.

@findinpath findinpath requested a review from ebyhr January 4, 2023 09:00
@mx123 mx123 force-pushed the hive_hudi_redirections branch 7 times, most recently from 5d08bf3 to 981ad14 Compare January 4, 2023 12:15
@findinpath findinpath requested a review from codope January 9, 2023 17:00
@findinpath
Copy link
Contributor

@codope CPTAL whether we did cover all the possible cases for identifying whether we're dealing with a Hudi table in isHudiTable() ?

@mx123 mx123 force-pushed the hive_hudi_redirections branch 2 times, most recently from 1b85603 to b355636 Compare January 11, 2023 10:21
@mx123 mx123 force-pushed the hive_hudi_redirections branch 4 times, most recently from 6b99110 to ff454e5 Compare January 12, 2023 12:23
@mx123 mx123 force-pushed the hive_hudi_redirections branch 2 times, most recently from b70b33a to 2435a8c Compare January 13, 2023 12:16
tooptoop4 and others added 2 commits January 16, 2023 19:03
Added minimal config testing and modified to comply Trino code standards of original PR: trinodb#14349
@mx123 mx123 force-pushed the hive_hudi_redirections branch from 2435a8c to c2bfca3 Compare January 16, 2023 17:03
@findinpath findinpath requested a review from ebyhr January 17, 2023 04:37
@ebyhr ebyhr merged commit d5dc585 into trinodb:master Jan 17, 2023
@github-actions github-actions bot added this to the 406 milestone Jan 17, 2023
@ebyhr ebyhr mentioned this pull request Jan 17, 2023
@mx123 mx123 deleted the hive_hudi_redirections branch January 17, 2023 08:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Support table redirections from Hive to Hudi
6 participants