Add test for remote partition evolution in Delta Lake #13489
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
findinpath
reviewed
Aug 4, 2022
|
|
||
| onDelta().executeQuery("REPLACE TABLE default." + tableName + " USING DELTA PARTITIONED BY (new_part_col) AS SELECT * FROM " + tableName); | ||
|
|
||
| // This 2nd SELECT query caused NPE when the connector had cache for partitions and the column was changed remotely |
There was a problem hiding this comment.
The comment is not necessary because this compatibility test is a valid usecase.
There was a problem hiding this comment.
I suppose future readers can't quickly realize this test purpose without comment.
There was a problem hiding this comment.
I suppose that the name of the test could be centered around REPLACE which is a SQL command specific for Databricks.
testTrinoCanReadAndWriteIntoTableReplacedInDatabricks
There was a problem hiding this comment.
REPLACE statement is just a part of steps to reproduce and it's not purpose of this test. Let me keep as-is.
findinpath
approved these changes
Aug 4, 2022
findepi
approved these changes
Aug 4, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add test for remote partition evolution in Delta Lake. Before this connector was published from Starburst, we have cache for partitions and it caused NPE when the partition column is changed remotely.
Documentation
(x) No documentation is needed.
Release notes
(x) No release notes entries required.