Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache deletion mark files together with meta files. #2457

Closed
wants to merge 14 commits into from
Closed

Cache deletion mark files together with meta files. #2457

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
3a2276a
Cache deletion mark files together with meta files.
pstibrany Apr 17, 2020
ad09efe
Recheck deletion mark files after hour of usage.
pstibrany Apr 20, 2020
448ff1d
Randomize recheck times
pstibrany Apr 20, 2020
6e41704
Unit tests for deletion markers.
pstibrany Apr 20, 2020
0c31a32
Added unit test for caching of deletion markers.
pstibrany Apr 20, 2020
24b0e13
Fixes spelling
pstibrany Apr 20, 2020
0381494
Fix sentences. Here we go again, Thanos Lint rules driving me crazy. …
pstibrany Apr 20, 2020
d54b377
Fix more lint warning. :lotus_position_man: :lotus_position_man: :lot…
pstibrany Apr 20, 2020
49065ff
Simplify loadDeletionMark to return only real errors.
pstibrany Apr 20, 2020
78d886c
Lint. 🐸 🐨 🐴 🐒 🐻 🐜 🐋
pstibrany Apr 20, 2020
977e1d9
Removed caching of non-existent deletion mark files.
pstibrany Apr 21, 2020
47f2a98
Fixed comment.
pstibrany Apr 21, 2020
9bcb1dc
No need to flush negative results anymore.
pstibrany Apr 21, 2020
95b6dff
Address review feedback.
pstibrany Apr 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/thanos/compact.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -293,7 +293,7 @@ func runCompact(
// While fetching blocks, we filter out blocks that were marked for deletion by using IgnoreDeletionMarkFilter.
// The delay of deleteDelay/2 is added to ensure we fetch blocks that are meant to be deleted but do not have a replacement yet.
// This is to make sure compactor will not accidentally perform compactions with gap instead.
ignoreDeletionMarkFilter := block.NewIgnoreDeletionMarkFilter(logger, bkt, deleteDelay/2)
ignoreDeletionMarkFilter := block.NewIgnoreDeletionMarkFilter(logger, deleteDelay/2)
duplicateBlocksFilter := block.NewDeduplicateFilter()

baseMetaFetcher, err := block.NewBaseFetcher(logger, 32, bkt, "", extprom.WrapRegistererWithPrefix("thanos_", reg))
Expand Down
2 changes: 1 addition & 1 deletion cmd/thanos/store.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ func runStore(
return errors.Wrap(err, "create index cache")
}

ignoreDeletionMarkFilter := block.NewIgnoreDeletionMarkFilter(logger, bkt, ignoreDeletionMarksDelay)
ignoreDeletionMarkFilter := block.NewIgnoreDeletionMarkFilter(logger, ignoreDeletionMarksDelay)
metaFetcher, err := block.NewMetaFetcher(logger, fetcherConcurrency, bkt, dataDir, extprom.WrapRegistererWithPrefix("thanos_", reg),
[]block.MetadataFilter{
block.NewTimePartitionMetaFilter(filterConf.MinTime, filterConf.MaxTime),
Expand Down
Loading