Add hive hook configuration parameters for each hms #322
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
patduin
requested changes
Jul 4, 2024
jmnunezizu
reviewed
Jul 5, 2024
| @@ -133,13 +139,36 @@ public void unreachableMetastoreClient() { | |||
| } | |||
| } | |||
|
|
|||
| @Test | |||
| public void loadMetastoreFilterHook(){ | |||
|
|
|||
flaming-archer
approved these changes
Jul 11, 2024
|
|
||
| public PrefixingMetastoreFilter(HiveConf conf) { | ||
| prefix = conf.get(PREFIX_KEY,PREFIX_DEFAULT); |
There was a problem hiding this comment.
Is PREFIX_KEY meaningless? Is it ok just use hardcode to determine if it is the same as "prefix -".
patduin
reviewed
Jul 22, 2024
|
|
||
| String oldLocation=sd.getLocation(); | ||
| assertThat(filterHook.filterTable(table).getSd().getLocation(), equalTo("prefix-test-" + oldLocation ) ); | ||
| }catch (Exception e){ |
There was a problem hiding this comment.
please remove try catch. No point in swallowing exceptions in test best to just throw it and let it fail the test.
patduin
reviewed
Jul 22, 2024
| import static org.hamcrest.CoreMatchers.instanceOf; | ||
| import static org.hamcrest.CoreMatchers.is; | ||
| import static org.hamcrest.CoreMatchers.notNullValue; | ||
| import static org.hamcrest.CoreMatchers.*; |
patduin
reviewed
Jul 22, 2024
| try { | ||
| StorageDescriptor sd = new StorageDescriptor(); | ||
| File localWarehouseUri = temporaryFolder.newFolder("local-warehouse"); | ||
| sd.setLocation(new File(localWarehouseUri,"local_database/local_table").toURI().toString()); |
There was a problem hiding this comment.
Can you please try if a simple string also works, maybe we don't need to create a dummy folder just for string manupilations.:
sd.setLocation("file:///tmp/local_database/local_table");
patduin
requested changes
Jul 22, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
📝 Description
When configuring hive hooks in waggle-dance-server.yml, it will take effect on all primary hms or federate hms. It does not support specifying hive hooks for individual hms.
Now it is supported to add hive hook configuration for a single primary hms or federate hms in waggle-dance-federation.yml without affecting each other.
🔗 Related Issues
#320