compactor: Consider upper bound of block size

[bwplotka]

Hi 👋

With vertical/offline deduplication we have a risk of oversized blocks. In the current design, this comes with the tradeoff of more difficult debug as well as at some point slower lookup.

From my experience chunk file size does not matter, but the index size matter, so we might want to split blocks at a certain size. Question is.. what size (: (most likely configurable). Thoughts? @brancz @SuperQ @pracucci @pstibrany

[pracucci]

I'm probably missing some context. Is your plan to just split the result of compaction to multiple blocks or to shard blocks before running the compaction? I guess the first one, but I would prefer to avoid any misunderstanding from my side.

[bwplotka]

I think actually something between. Do that during compaction (:

[bwplotka]

I am already exploring iterator interface on Prometheus side which could help with this

[pracucci]

This will leave still open the vertical scalability limit of a compactor, until blocks can't be sharded. What I mean is, if compacting blocks for the range X takes more time than X you have a scalability limit. Wouldn't blocks sharding in the compactor solve both?

Ignore this comment. Was a misunderstand from my side. This is blocks sharding agnostic.

[stale]

Hello 👋 Looks like there was no activity on this issue for last 30 days.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity for next week, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[bwplotka]

Still relevant.

…

On Wed, 29 Apr 2020 at 11:12, stale[bot] ***@***.***> wrote: Hello 👋 Looks like there was no activity on this issue for last 30 days. *Do you mind updating us on the status?* Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗 If there will be no activity for next week, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command <https://probot.github.io/apps/reminders/> if you wish to be reminded at some point in future. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2340 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVA3O5EBKGCCYR7CT4FAS3RO74QNANCNFSM4LWMW2HA> .

[stale]

Hello 👋 Looks like there was no activity on this issue for last 30 days.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity for next week, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗

[ipstatic]

I think this is still relevant.

[SuperQ]

Yes, I just had a 480GB compaction, which means I need 1T of local disk to handle these events. It would be nice to be able to set an upper bound on TSDB block sizes.

[squat]

still relevant

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[bwplotka]

@Biswajitghosh98 is on it (:

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗