verify: Add separated blocks issue and fix. #280
Conversation
| // - another ovelapped block with range 20-50 | ||
| // - bigger block is missing exactly these sources from smaller block to fill its range. | ||
| // Various bugs could introduce these blocks, including broken compaction for Prometheus 2.2.0. | ||
| func SeparatedBlocks(ctx context.Context, logger log.Logger, bkt objstore.Bucket, backupBkt objstore.Bucket, repair bool) error { |
There was a problem hiding this comment.
Mh, can you elaborate a bit more what motivated this one. It seems like the starting position the example describes is actually valid, i.e. not an issue.
If the compactor does not pick that input up for compaction, that's probably just: prometheus-junkyard/tsdb#90
Though it can of course involve risk if the input blocks aren't truly disjoint in their actual data.
There was a problem hiding this comment.
The example states the broken state (overlapping), why you say it is not an issue?
Thanks for referencing this issue. It all makes sense, so maybe we want to add something like that for TSDB? I mean at least explicit VerticalCompact for now not used internally yet. To compact overlapped blocks.
Wonder however if compactor should do that automatically. For now any overlap case is invalid state. It can be introduced only by bug. It should not happen. Of course we could make it differently -> allow or don't care about overlaps and just "heal" them by something like vertical compact, true, but not sure if that's the right path. Not for now for sure.
For now I would really merge this for now here.
Signed-off-by: Bartek Plotka <[email protected]>
Signed-off-by: Bartek Plotka <[email protected]>
Signed-off-by: Bartek Plotka [email protected]