Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[columnar] Adds columnar.vacuum() and columnar.stats() #71

Merged
merged 2 commits into from
May 16, 2023

Conversation

JerrySievert
Copy link
Contributor

@JerrySievert JerrySievert commented Apr 20, 2023

Adds two new UDF's:

  • columnar.vacuum()
  • columnar.stats()

The vacuum does an in-place vacuum, combining stripes and moving them to the beginning of the table space. This builds off of work done by @mkaruza.

The stats are useful for testing and getting information about the current state of the table and its stripes.

Note, this includes fixing an issue that @mkaruza found, as it affected this as well. As such, it changed the results of the columnar_vacuum test. (edit: those changes have been merged separately)

Given that this makes modifications to stripe data and metadata, as well as a change to an existing test, it needs a thorough review.

The vacuum UDF can optionally include a count of the number of stripes to modify, allowing for incremental vacuuming. It returns the number of stripes modified. When this number hits 0, then all vacuuming is complete.

CREATE TABLE t1 (a INT);
INSERT INTO t1 SELECT generate_series(1, 1000000, 1);
DELETE FROM t1 WHERE a % 3 = 0;
SELECT * FROM columnar.vacuum('test'::regclass, 1);

@JerrySievert JerrySievert requested review from wuputah and mkaruza April 20, 2023 22:41
@JerrySievert JerrySievert self-assigned this Apr 20, 2023
@wuputah
Copy link
Member

wuputah commented Apr 24, 2023

Deferring to @mkaruza for review.

@JerrySievert JerrySievert marked this pull request as draft April 24, 2023 19:37
@JerrySievert JerrySievert marked this pull request as ready for review April 25, 2023 18:48
@JerrySievert JerrySievert force-pushed the columnar-vacuum branch 2 times, most recently from 9e98a7f to 172bf3f Compare May 3, 2023 22:29
@wuputah wuputah requested a review from mkaruza May 12, 2023 16:02
Copy link
Contributor

@mkaruza mkaruza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@JerrySievert JerrySievert merged commit 90908f5 into main May 16, 2023
@JerrySievert JerrySievert deleted the columnar-vacuum branch May 16, 2023 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants