Skip to content

Commit

Permalink
Add more deployment modes
Browse files Browse the repository at this point in the history
Signed-off-by: Filip Petkovski <[email protected]>
  • Loading branch information
fpetkovski committed Jan 5, 2023
1 parent f9676f7 commit 9443e0e
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 2 deletions.
2 changes: 1 addition & 1 deletion docs/components/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -815,7 +815,7 @@ Flags:
--rewrite.to-relabel-config-file=<file-path>
Path to YAML file that contains relabel configs
that will be applied to blocks
--tmp.dir="/tmp/thanos-rewrite"
--tmp.dir="/var/folders/0b/r42s6hv96hs2w84ycr2yf7qh0000gn/T/thanos-rewrite"
Working directory for temporary files
--tracing.config=<content>
Alternative to 'tracing.config-file' flag
Expand Down
Binary file added docs/img/distributed-execution-proposal-5.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
14 changes: 13 additions & 1 deletion docs/proposals-accepted/202301-distributed-query-execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,16 +121,28 @@ Each subquery would preserve the external `cluster` label which will allow the `

## Deployment models

With this approach, a Thanos admin can arrange remote Queriers in an arbitrary way, as long as TSDB replicas are always queried by only one remote Querier. The following deployment models can be used as examples:
With this approach, a Thanos admin can arrange remote Queriers in an arbitrary way, as long as TSDB replicas are always queried by only one remote Querier. The following deployment models can be used as examples:

#### Monitoring different environments with Prometheus pairs

In this deployment mode, remote queriers are attached to pairs of Prometheus instances. The central Querier delegates subqueries to them and performs a central aggregation of results.

<img src="../img/distributed-execution-proposal-4.png" alt="Distributed query execution" width="400"/>

#### Querying separate Store Gateways with disjoint datasets

Remote Queriers can be attached to Prometheus pairs and Store Gateways at the same time. The central querier delegates subqueries and deduplicates overlapping results before performing a central aggregation.

<img src="../img/distributed-execution-proposal-3.png" alt="Distributed query execution" width="400"/>

#### Running remote queriers as Store Gateway sidecars

Remote Queriers can be attached to disjoint groups of Store Gateways. They can even be attached to individual Store Gateways which have deduplicated TSDB blocks, or hold all replicas of a TSDB block. This will make sure penalty-based deduplication happens in the remote querier.

Store groups can be created by either partitioning TSDBs by time (time-based partitioning), or by external labels. Both of these techniques are documented in the [Store Gateway documentation](https://thanos.io/tip/components/store.md/#time-based-partitioning).

<img src="../img/distributed-execution-proposal-5.png" alt="Distributed query execution" width="400"/>

## 7 Alternatives

A viable alternative to the proposed method is to add support for Query Pushdown in the Thanos Querier. By extracting better as described in https://github.com/thanos-io/thanos/issues/5984, we can decide to execute a query in a local Querier, similar to how the sidecar does that against Prometheus.
Expand Down

0 comments on commit 9443e0e

Please sign in to comment.