Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Gradual compaction #384

Merged
merged 4 commits into from
Nov 14, 2022
Merged

Gradual compaction #384

merged 4 commits into from
Nov 14, 2022

Conversation

uprendis
Copy link
Contributor

@uprendis uprendis commented Nov 2, 2022

  • Separate compaction into 256 prefixes, where each prefix is separated into 32 iterations, depending on range between first and last keys of the prefix. Previously compaction was separated only in 256 prefixes, which resulted into overly large compaction ranges, because data distribution isn't uniform and too much data may be stored behind the same prefix
  • Add forced DB compaction after genesis processing and DB transformation/migration, because Pebble DB has performance issues after inserting a lot of data without manual compaction

hadv
hadv approved these changes Nov 3, 2022
View reviewed changes
Copy link
Contributor

@hadv hadv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

hadv
hadv reviewed Nov 3, 2022
View reviewed changes
utils/compactdb/compactdb.go Outdated Show resolved Hide resolved
@uprendis uprendis merged commit 9e2374a into Fantom-foundation:develop Nov 14, 2022
@uprendis uprendis mentioned this pull request Dec 9, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hadv hadv hadv approved these changes

@rus-alex rus-alex Awaiting requested review from rus-alex

@andrecronje andrecronje Awaiting requested review from andrecronje andrecronje is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@uprendis @hadv