Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always allow unquoted keywords as column names #539

Merged
merged 2 commits into from
Sep 26, 2024

Conversation

KevinGe00
Copy link
Contributor

@KevinGe00 KevinGe00 commented Sep 26, 2024

What changes are proposed in this pull request, and why are they necessary?

This PR is a follow up to #503 and fixes #534. During integration testing with a Trino environment, the instantiation of the new HiveConf object threw failure to login: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name. This is due to the false assumption for the presence of Hive configuration files such as hive-site.xml in Trino environments, which new HiveConf() will try to look for. This PR fixes this false assumption by removing new HiveConf() and forces leniency for unquote keyword columns when parsing hive views. A long term solution will involve an engine-agnostic KV config map used ubiquitously across coral.

How was this patch tested?

  • clean build
  • testUnquotedKeywordAsColumnName passes when useSQL11ReservedKeywordsForIdentifier returns true and fails when useSQL11ReservedKeywordsForIdentifier returns false
  • tested translation of view with unquoted keyword column on spark-shell with these coral changes
  • set up local trino cluster with these coral changes, successfully ran translation for view with unquoted keyword column
  • regression tests for spark and trino passes

@KevinGe00 KevinGe00 merged commit b507761 into linkedin:master Sep 26, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unintended HiveConf overriding in Coral leading to query execution auth failures
2 participants