[columnar] Adds columnar.vacuum() and columnar.stats() #71
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds two new UDF's:
columnar.vacuum()
columnar.stats()
The vacuum does an in-place vacuum, combining stripes and moving them to the beginning of the table space. This builds off of work done by @mkaruza.
The stats are useful for testing and getting information about the current state of the table and its stripes.
Note, this includes fixing an issue that @mkaruza found, as it affected this as well. As such, it changed the results of the(edit: those changes have been merged separately)columnar_vacuum
test.Given that this makes modifications to stripe data and metadata, as well as a change to an existing test, it needs a thorough review.
The vacuum UDF can optionally include a count of the number of stripes to modify, allowing for incremental vacuuming. It returns the number of stripes modified. When this number hits
0
, then all vacuuming is complete.