Support compacting files to different temperatures in FIFO compaction #11428
Conversation
cbi42
force-pushed
the
fifo-file-temperature
branch
from
0288af3 to
b8e4873
Compare
cbi42
changed the title
|
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
cbi42
force-pushed
the
fifo-file-temperature
branch
from
46f07d0 to
b18ebf2
Compare
|
@cbi42 has updated the pull request. You must reimport the pull request before landing. |
|
@cbi42 has updated the pull request. You must reimport the pull request before landing. |
|
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
include/rocksdb/advanced_options.h
Outdated
| // Dynamically changeable through SetOptions() API, e.g., | ||
| // SetOptions("compaction_options_fifo", | ||
| // "{file_temperature_age_thresholds={ | ||
| // {age=20;temperature=kCold}:{age=10;temperature=kWarm}}") |
There was a problem hiding this comment.
These seem out of order. Above it says "The elements should be in increasing order with respect to age field".
There was a problem hiding this comment.
Thanks for catching this. I've updated the comment and added related validation and unit test too.
|
@cbi42 has updated the pull request. You must reimport the pull request before landing. |
cbi42
force-pushed
the
fifo-file-temperature
branch
from
58e8f2a to
042c4cb
Compare
|
@cbi42 has updated the pull request. You must reimport the pull request before landing. |
|
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@cbi42 has updated the pull request. You must reimport the pull request before landing. |
|
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@cbi42 has updated the pull request. You must reimport the pull request before landing. |
1 similar comment
|
@cbi42 has updated the pull request. You must reimport the pull request before landing. |
|
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Summary:
CompactionOptionsFIFO::file_temperature_age_thresholdsthat allows user to specify age thresholds for compacting files to different temperatures. File temperature can be used to store files in different storage media. The new options allows specifying multiple temperature-age pairs. The option uses struct for a temperature-age pair to use the existing parsing functionality to make the option dynamically settable.age_for_warmthat was added for a similar purpose.FIFOCompactionPicker::PickTemperatureChangeCompaction.Test plan: adapted unit tests that were for
age_for_warmto this new option.