Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for compacting small files for Hive tables #9398

Closed
wants to merge 42 commits into from
Closed

Add support for compacting small files for Hive tables #9398

wants to merge 42 commits into from

Commits on Oct 12, 2021

  1. Add SPI for table procedures

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    43977c1 View commit details
    Browse the repository at this point in the history

  2. fixup! Add SPI for table procedures

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    1d19c7f View commit details
    Browse the repository at this point in the history

  3. fixup! Add SPI for table procedures

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    2a6619c View commit details
    Browse the repository at this point in the history

  4. fixup! Add SPI for table procedures

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    6985dbd View commit details
    Browse the repository at this point in the history

  5. Add TableProceduresRegistry to Metadata

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    30f646b View commit details
    Browse the repository at this point in the history

  6. fixup! Add TableProceduresRegistry to Metadata

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    1db5b7d View commit details
    Browse the repository at this point in the history

  7. fixup! Add TableProceduresRegistry to Metadata

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    f153fbf View commit details
    Browse the repository at this point in the history

  8. Generalize AbstractPropertyManager

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    9a5b851 View commit details
    Browse the repository at this point in the history

  9. Add TableProceduresPropertyManager

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    f2bd0a1 View commit details
    Browse the repository at this point in the history

  10. Add support for table procedures SPI calls to Metadata

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    bf7790f View commit details
    Browse the repository at this point in the history

  11. fixup! Add support for table procedures SPI calls to Metadata

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    14ed0ab View commit details
    Browse the repository at this point in the history

  12. Add parser/analyzer support for ALTER TABLE ... EXECUTE

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    fd37a73 View commit details
    Browse the repository at this point in the history

  13. fixup! Add parser/analyzer support for ALTER TABLE ... EXECUTE

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    76ab884 View commit details
    Browse the repository at this point in the history

  14. fixup! Add parser/analyzer support for ALTER TABLE ... EXECUTE

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    d08e204 View commit details
    Browse the repository at this point in the history

  15. fixup! Add parser/analyzer support for ALTER TABLE ... EXECUTE

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    b83447f View commit details
    Browse the repository at this point in the history

  16. fixup! Add parser/analyzer support for ALTER TABLE ... EXECUTE

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    db68344 View commit details
    Browse the repository at this point in the history

  17. Add access control for table procedures

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    485f24d View commit details
    Browse the repository at this point in the history

  18. fixup! Add access control for table procedures

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    f407be1 View commit details
    Browse the repository at this point in the history

  19. Define QueryExecution for TableExecute statement

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    9597bb2 View commit details
    Browse the repository at this point in the history

  20. Add planner logic for TableExecute statement

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    1074941 View commit details
    Browse the repository at this point in the history

  21. fixup! Add planner logic for TableExecute statement

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    1a09c45 View commit details
    Browse the repository at this point in the history

  22. Do not output rows count from ALTER TABLE EXECUTE

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    63b98da View commit details
    Browse the repository at this point in the history

  23. Pass splits info to TableFinish operator for ALTER TABLE EXECUTE

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    3d63c92 View commit details
    Browse the repository at this point in the history

  24. Prefer toImmutableList in HiveMetadata

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    11d30ca View commit details
    Browse the repository at this point in the history

  25. Add Hive OPTIMIZE table procedure 

    Add support for compacting small files for non-transactional,
non-bucketed Hive tables.

ALTER TABLE xxxxx EXECUTE OPTIMIZE WITH(file_size_threshold = ...)
    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    898fa8c View commit details
    Browse the repository at this point in the history

  26. fixup! Add TableProceduresRegistry to Metadata

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    f56191c View commit details
    Browse the repository at this point in the history

  27. fixup! Add SPI for table procedures

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    77e91a3 View commit details
    Browse the repository at this point in the history

  28. fixup! Add parser/analyzer support for ALTER TABLE ... EXECUTE

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    5138fff View commit details
    Browse the repository at this point in the history

  29. fixup! Add planner logic for TableExecute statement

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    3f1586a View commit details
    Browse the repository at this point in the history

  30. fixup! Add TableProceduresPropertyManager

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    e77c2bc View commit details
    Browse the repository at this point in the history

  31. fixup! Add planner logic for TableExecute statement

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    62aa891 View commit details
    Browse the repository at this point in the history

  32. fixup! Pass splits info to TableFinish operator for ALTER TABLE EXECUTE

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    04ad808 View commit details
    Browse the repository at this point in the history

  33. fixup! Pass splits info to TableFinish operator for ALTER TABLE EXECUTE

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    d14457c View commit details
    Browse the repository at this point in the history

  34. fixup! Pass splits info to TableFinish operator for ALTER TABLE EXECUTE

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    b7c9b9b View commit details
    Browse the repository at this point in the history

  35. fixup! Pass splits info to TableFinish operator for ALTER TABLE EXECUTE

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    2724556 View commit details
    Browse the repository at this point in the history

  36. fixup! Add Hive OPTIMIZE table procedure

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    21bc91c View commit details
    Browse the repository at this point in the history

  37. fixup! Add SPI for table procedures

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    012bb1a View commit details
    Browse the repository at this point in the history

  38. Add tableExecuteSplitsInfo to FixedSplitSource

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    a047359 View commit details
    Browse the repository at this point in the history

  39. Allow filtering on partitions for Hive OPTIMIZE table procedure

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    210b395 View commit details
    Browse the repository at this point in the history

  40. fixup! Add parser/analyzer support for ALTER TABLE ... EXECUTE

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    5e43e3e View commit details
    Browse the repository at this point in the history

  41. Allow dropping declared write intentions in SemiTransactionalHiveMeta… 

    …store
    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    96d211f View commit details
    Browse the repository at this point in the history

  42. Add logic to limit chance for data loss in Hive OPTIMIZE table procedure

    @losipiuk
    losipiuk committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    262ebf3 View commit details
    Browse the repository at this point in the history