Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DAE-133] Removing types-comparison when dropping a column #56

Merged
merged 2 commits into from
Mar 10, 2021

Conversation

felipemiquelim
Copy link
Contributor

@felipemiquelim felipemiquelim commented Mar 9, 2021

Why? 📖

When running an ALTER TABLE Hive stores the types of columns from table schema in sequential order to enforce the same types after the ALTER TABLE. However when performing our abstraction of DROP COLUMN, the types may change, since the schema will definitely change.

Currently, if we try to drop a column that changes this 'types order', the following error returns:

thrift_files.libraries.thrift_hive_metastore_client.ttypes.InvalidOperationException: InvalidOperationException(message='The following columns have types incompatible with the existing columns in their respective positions :\ncolumn_name')

What? 🔧

  • Adding a call to change metastore settings while in drop_columns_from_table:
    Allowing, in the current execution, the changing of types when comparing the table schema prior and after ALTER TABLE.
  • Adjusting tests

Type of change 🗄️

  • Bug fix (non-breaking change which fixes an issue)

How everything was tested? 📏

Ran the config command both alone and together with the drop_columns_from_table and it only succeed in the same 'transaction'.

Checklist 📝

  • I have added labels to distinguish the type of pull request.
  • My code follows the style guidelines of this project (docstrings, type hinting and linter compliance);
  • I have performed a self-review of my own code;
  • I have made corresponding changes to the documentation;
  • I have added tests that prove my fix is effective or that my feature works;
  • I have made sure that new and existing unit tests pass locally with my changes;

@felipemiquelim felipemiquelim added the bug Something isn't working label Mar 9, 2021
@felipemiquelim felipemiquelim requested a review from a team as a code owner March 9, 2021 20:06
@felipemiquelim felipemiquelim self-assigned this Mar 9, 2021
LucasMMota
LucasMMota previously approved these changes Mar 9, 2021
@sonarqubecloud
Copy link

sonarqubecloud bot commented Mar 9, 2021

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@felipemiquelim felipemiquelim merged commit a8158d7 into main Mar 10, 2021
@felipemiquelim felipemiquelim deleted the felipemiquelim/drop_columns_no_type_enforce branch March 10, 2021 13:30
@LucasMMota LucasMMota mentioned this pull request Mar 12, 2021
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants