receive: use async remote writing #7045
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GiedriusS
force-pushed
the
receive_worker_arch_upstream
branch
5 times, most recently
from
4ea9caf to
69a3429
Compare
2 tasks
There was a problem hiding this comment.
Left some questions.
Would be great if you could wait for the reorganization/refactor of the Receive handler in #6898.
Which that PR that code becomes much easier to follow and hopefully it'll be very easy to introduce these changes.
WDYT? Otherwise I'll have one more huge and difficult rebase to do, again :(
GiedriusS
force-pushed
the
receive_worker_arch_upstream
branch
3 times, most recently
from
266601f to
e4b6e99
Compare
Instead of spawning new goroutines for each peer that we want to remote write to, spawn a fixed number of worker goroutines and then schedule work on top of them. This has reduced the number of goroutines in our case about 10x-20x and the 99p of forwarding dropped from ~30s to just a few hundred milliseconds. Signed-off-by: Giedrius Statkevičius <[email protected]>
GiedriusS
force-pushed
the
receive_worker_arch_upstream
branch
from
e4b6e99 to
a177259
Compare
|
@douglascamata thanks for your review, I have made adjustments and rebased it on top of your recent nice cleanup/refactoring 🙇 please take a look when you have some time |
douglascamata
approved these changes
Jan 24, 2024
| @@ -888,6 +890,7 @@ func (rc *receiveConfig) registerFlag(cmd extkingpin.FlagClause) { | |||
|
|
|||
| cmd.Flag("receive.replica-header", "HTTP header specifying the replica number of a write request.").Default(receive.DefaultReplicaHeader).StringVar(&rc.replicaHeader) | |||
|
|
|||
| cmd.Flag("receive.forward.async-workers", "Number of concurrent workers processing forwarding of remote-write requests.").Default("5").UintVar(&rc.asyncForwardWorkerCount) | |||
There was a problem hiding this comment.
That might be breaking change to look out for if 5 is not sufficient for someone, right? I think we should mark it as "breaking" in changelog just so people can be careful in the rollout and know what to look for if 5 should not suffice. ( Even if its most likely enough for most people )
yeya24
approved these changes
Jan 26, 2024
GiedriusS
added a commit
to vinted/thanos
that referenced
this pull request
Jan 30, 2024
jnyi
pushed a commit
to jnyi/thanos
that referenced
this pull request
Apr 4, 2024
coleenquadros
added a commit
to stolostron/thanos
that referenced
this pull request
Jul 24, 2024
* CI: increase e2e test parallelism (#6875) Signed-off-by: Michael Hoffmann <[email protected]> * compact: hook nodownsamplemarkfilter into filters chain (#6893) We have a NoDownsampleMarkFilter that we were not using before in the compactor for some reason. Hook it into the filters chain if downsampling is enabled and then trim matching ULIDs from the downsampling process. Add a test to cover this scenario. Fixes https://github.com/thanos-io/thanos/issues/6179. Signed-off-by: Giedrius Statkevičius <[email protected]> * e2e/compact: fix assertion values (#6909) I think the original values were simply too small there - if the runner is slow then it can see smaller metrics values. If the runner works well then the values can go to these bigger values and then the test fails. Signed-off-by: Giedrius Statkevičius <[email protected]> * exposes asyncOperationProcessor (#6908) expose struct fix lint Signed-off-by: Ben Ye <[email protected]> * Removed the gitpod from the contribution.md (#6907) Signed-off-by: MeenuyD <[email protected]> * .circleci: cache tool deps (#6910) I have noticed that downloading tool deps takes ~5 min in CI each time. We should be able to also cache those tool deps. Signed-off-by: Giedrius Statkevičius <[email protected]> * MAINTAINERS.md: add myself as maintainer (#6911) Signed-off-by: Michael Hoffmann <[email protected]> * blog: Introducing ThanosCon in Paris KubeCon EU 2024 (#6912) * blog: Introducing ThanosCon in Paris KubeCon EU 2024 Signed-off-by: Saswata Mukherjee <[email protected]> * Add image Signed-off-by: Saswata Mukherjee <[email protected]> * make docs Signed-off-by: Saswata Mukherjee <[email protected]> * Apply suggestions Co-authored-by: Matej Gera <[email protected]> Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Saswata Mukherjee <[email protected]> * Apply suggestions Co-authored-by: Bartlomiej Plotka <[email protected]> Signed-off-by: Saswata Mukherjee <[email protected]> * Add link to slack channel Signed-off-by: Saswata Mukherjee <[email protected]> --------- Signed-off-by: Saswata Mukherjee <[email protected]> Co-authored-by: Matej Gera <[email protected]> Co-authored-by: Bartlomiej Plotka <[email protected]> * Update 2023-20-11-thanoscon.md (typo) (#6915) Signed-off-by: Bartlomiej Plotka <[email protected]> * Added info about ThanosCon to website and Readme. (#6917) * Added info about ThanosCon to website and Readme. Signed-off-by: bwplotka <[email protected]> * Update website/layouts/index.html Co-authored-by: Saswata Mukherjee <[email protected]> Signed-off-by: Bartlomiej Plotka <[email protected]> --------- Signed-off-by: bwplotka <[email protected]> Signed-off-by: Bartlomiej Plotka <[email protected]> Co-authored-by: Saswata Mukherjee <[email protected]> * Objstore: Bump Objstore for Azure Workload Identity support (#6891) * Bump objstore for Azure Workload Identity support Signed-off-by: Rikhil Shah <[email protected]> * make check-docs Signed-off-by: Rikhil Shah <[email protected]> * Add changelog entry Signed-off-by: Rikhil Shah <[email protected]> * Update Azure client docs Signed-off-by: Rikhil Shah <[email protected]> * make check-docs Signed-off-by: Rikhil Shah <[email protected]> * Move changelog entry to 0.33.0 release Signed-off-by: Rikhil Shah <[email protected]> * Move changelog entry Signed-off-by: Rikhil Shah <[email protected]> --------- Signed-off-by: Rikhil Shah <[email protected]> Signed-off-by: Michael Hoffmann <[email protected]> * update compactor backlog doc for checking halt (#6906) Signed-off-by: Ben Ye <[email protected]> * Update LabyrintLabs logo Signed-off-by: Martin Hauskrecht <[email protected]> * Support reload using signal (#6453) * Support reload using signal Signed-off-by: Simon Pasquier <[email protected]> * Add --reloader.method option to sidecar This option allows to tell the sidecar to send a SIGHUP signal to the monitored process to reload its configuration instead of the default HTTP-based method. Signed-off-by: Simon Pasquier <[email protected]> * Update docs and CHANGELOG.md Signed-off-by: Simon Pasquier <[email protected]> --------- Signed-off-by: Simon Pasquier <[email protected]> * Support float histogram in store gateway (#6925) * support float histogram in store gateway Signed-off-by: Ben Ye <[email protected]> * fix lint Signed-off-by: Ben Ye <[email protected]> * fix e2e test Signed-off-by: Ben Ye <[email protected]> * update prompb Signed-off-by: Ben Ye <[email protected]> * add changelog Signed-off-by: Ben Ye <[email protected]> --------- Signed-off-by: Ben Ye <[email protected]> * chore: add truelayer to the adopters (#6933) Signed-off-by: Matteo Martellini <[email protected]> * Fetcher: Add a BlockIDsFetcher Interface to BaseFetcher (#6902) * add BlockIDsFetcher to BaseFetcher Signed-off-by: Wen Xu <[email protected]> * fix lint Signed-off-by: Wen Xu <[email protected]> * use chan in the interface method to accept active block ids Signed-off-by: Wen Xu <[email protected]> * fix comments Signed-off-by: Wen Xu <[email protected]> * fix lint Signed-off-by: Wen Xu <[email protected]> * add description of active and parital blocks and modify changelog Signed-off-by: Wen Xu <[email protected]> * fix interface description Signed-off-by: Wen Xu <[email protected]> * remove entry in changelog Signed-off-by: Wen Xu <[email protected]> --------- Signed-off-by: Wen Xu <[email protected]> * Update hugo file (#6927) * Update hugo file #6889 Signed-off-by: Kartikay <[email protected]> * Version 0.33 removed Signed-off-by: Kartikay <[email protected]> --------- Signed-off-by: Kartikay <[email protected]> * Updates busybox SHA (#6897) (#6937) Signed-off-by: GitHub <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: fpetkovski <[email protected]> * Fix hotstar logo (#6938) * Fix hotstar logo Signed-off-by: Kartikay <[email protected]> * Delete website/.hugo_build.lock Signed-off-by: Kartikay <[email protected]> --------- Signed-off-by: Kartikay <[email protected]> Signed-off-by: Kartikay <[email protected]> * errutil: fix deadlock Fix deadlock in the Add() function - it accidentally was calling itself recursively. This struct wraps a `MultiError` so we need to call `Add()` on that `MultiError` inside. This deadlock can manifest in Receive - million+ leaking goroutines, all stuck on the MultiTSDB lock. Signed-off-by: Giedrius Statkevičius <[email protected]> * update runtime/CloseWithErrCapture comment (#6952) * update runtime/CloseWithErrCapture comment Signed-off-by: Sagnik Das <[email protected]> * fix lint Signed-off-by: Sagnik Das <[email protected]> --------- Signed-off-by: Sagnik Das <[email protected]> * Added a new flag for maximum retention bytes for thanos (#6944) * changes Signed-off-by: Kartikay <[email protected]> * Changed Int64 to BytesVar Signed-off-by: Kartikay <[email protected]> * Updated Changelog for new flag in receive Signed-off-by: Kartikay <[email protected]> --------- Signed-off-by: Kartikay <[email protected]> Signed-off-by: Kartikay <[email protected]> * fix store gateway span name (#6953) Signed-off-by: Ben Ye <[email protected]> * Upgrade prometheus to latest main (#6950) * upgrade prometheus to latest main Signed-off-by: Ben Ye <[email protected]> * upgrade again Signed-off-by: Ben Ye <[email protected]> * fix test Signed-off-by: Ben Ye <[email protected]> --------- Signed-off-by: Ben Ye <[email protected]> * Improve filtered index cache filtering (#6955) * improve filtered index cache filtering Signed-off-by: Ben Ye <[email protected]> * make inline Signed-off-by: Ben Ye <[email protected]> --------- Signed-off-by: Ben Ye <[email protected]> * store: fix misc label_value issues related to external labels (#6879) Signed-off-by: Michael Hoffmann <[email protected]> * receive: fix adding fields to logger Fix adding fields to the logger because otherwise it looks like "[tenant foo]: (UNDEFINED)". Signed-off-by: Giedrius Statkevičius <[email protected]> * Copy labels coming from remote engines (#6957) * Copy labels coming from remote engines When running in distributed mode, the remote engine will use an unsafe cast from ZLabels to Prometheus labels to avoid making new allocations. This makes it hard to use the new gRPC shared buffer pool for receiving and decompressing messages since memory gets retained beyond the scope of a Recv() call. This commit removes the unsafe cast and makes an explicit memory copy of received series labels. Since remote queries are already aggregated series, the amount of data we receive should be small anyway, and the copies on average should have a small impact. Signed-off-by: Filip Petkovski <[email protected]> * Use clone on strings Signed-off-by: Filip Petkovski <[email protected]> --------- Signed-off-by: Filip Petkovski <[email protected]> * Add tracing index cache (#6954) * add tracing index cache Signed-off-by: Ben Ye <[email protected]> * changelog Signed-off-by: Ben Ye <[email protected]> --------- Signed-off-by: Ben Ye <[email protected]> * Change order of log for finished compacting blocks (#6966) Signed-off-by: Daniel Deluiggi <[email protected]> * Update MAINTAINERS.md (#6967) Signed-off-by: Matej Gera <[email protected]> * Add `keep_firing_for` support to alerting rule (#6943) * Add keep_firing_for to alert rules Signed-off-by: JHeil <[email protected]> * Updating changelog Signed-off-by: JHeil <[email protected]> * Re-order keepFiringFor at the end of the struct Signed-off-by: JHeil <[email protected]> --------- Signed-off-by: JHeil <[email protected]> Signed-off-by: JHeil <[email protected]> * Use registerer for gRPC opts and endpoints (#6971) The EndpointSet and StoreGRPC options accept concrete *prometheus.Registry type. This makes it hard to create multiple instances of them because they will try to register the same metrics and the registry will panic. This commit changes the type to a prometheus.Registerer which allows clients to inject a prefixed registry and get metrics for each instance. Signed-off-by: Filip Petkovski <[email protected]> * receive: upload compacted blocks if OOO enabled (#6974) * Add streaming series limit at block series client (#6972) * add series limit that is applied when streaming using block series client Signed-off-by: Ben Ye <[email protected]> * changelog Signed-off-by: Ben Ye <[email protected]> * add unit tests Signed-off-by: Ben Ye <[email protected]> * address comments Signed-off-by: Ben Ye <[email protected]> * fix comment Signed-off-by: Ben Ye <[email protected]> --------- Signed-off-by: Ben Ye <[email protected]> * Updated the defaultChecked property in checkbox element (#6976) Signed-off-by: lpreethvika-samsung <[email protected]> * Update prometheus to d0c2d9c (#6978) * Update prometheus to d0c2d9c This commit updates Prometheus to commit d0c2d9c which contains the loser-tree based postings merge. Note that we cannot update to latest main since there is currently a bug in PromQL when querying native histograms introduced by https://github.com/prometheus/prometheus/pull/13276. This issue should be fixed by https://github.com/prometheus/prometheus/pull/13289. This commit also updates the Thanos PromQL engine to latest main. Signed-off-by: Filip Petkovski <[email protected]> * Fix handler_test.go Signed-off-by: Filip Petkovski <[email protected]> * Fix manager_test.go Signed-off-by: Filip Petkovski <[email protected]> * Use empty registry for file discovery Signed-off-by: Filip Petkovski <[email protected]> --------- Signed-off-by: Filip Petkovski <[email protected]> * Updates busybox SHA (#6982) Signed-off-by: GitHub <[email protected]> Co-authored-by: fpetkovski <[email protected]> * Updates busybox SHA (#6982) Signed-off-by: GitHub <[email protected]> Co-authored-by: fpetkovski <[email protected]> * VERSION: Bump version for release Signed-off-by: Michael Hoffmann <[email protected]> * rule: native histogram support (#6390) * Added native histogram support for ruler Signed-off-by: Sebastian Rabenhorst <[email protected]> Formatted imports Signed-off-by: Sebastian Rabenhorst <[email protected]> Fixed imports Signed-off-by: Sebastian Rabenhorst <[email protected]> Formated imports Signed-off-by: Sebastian Rabenhorst <[email protected]> * Fixed native histogram tests Signed-off-by: Sebastian Rabenhorst <[email protected]> Fixed receiver type Signed-off-by: Sebastian Rabenhorst <[email protected]> * Fix for rebase Signed-off-by: Sebastian Rabenhorst <[email protected]> * Added docs for query endpoints differences Signed-off-by: Sebastian Rabenhorst <[email protected]> * Fixed comments and naming Signed-off-by: Sebastian Rabenhorst <[email protected]> * made HTTPConfig optional Signed-off-by: Sebastian Rabenhorst <[email protected]> * made HTTPConfig optional Signed-off-by: Sebastian Rabenhorst <[email protected]> * Reverted and added check Signed-off-by: Sebastian Rabenhorst <[email protected]> * Fixes from comments Signed-off-by: Sebastian Rabenhorst <[email protected]> * renamed queryconfig to clientconfig Signed-off-by: Sebastian Rabenhorst <[email protected]> * common prepareEndpointSet Signed-off-by: Sebastian Rabenhorst <[email protected]> * fixed lint Signed-off-by: Sebastian Rabenhorst <[email protected]> * Fixed sidecar Signed-off-by: Sebastian Rabenhorst <[email protected]> * Fixed tests Signed-off-by: Sebastian Rabenhorst <[email protected]> --------- Signed-off-by: Sebastian Rabenhorst <[email protected]> * mdox ignore checking twitter urls (#7001) Signed-off-by: Ben Ye <[email protected]> * Updated Grofers logo (#7006) Signed-off-by: Kartikay <[email protected]> * TraceID : Fetching TraceID (#6973) * docs: add promcon 2023 thanos talks Signed-off-by: Michael Hoffmann <[email protected]> * Added website page for companies who offer consultancy and enterprise… (#7000) * Added website page for companies who offer consultancy and enterprise support for Thanos Signed-off-by: Kartikay <[email protected]> * adopters.yml revert Signed-off-by: Kartikay <[email protected]> * retrigger checks Signed-off-by: Kartikay <[email protected]> * added a new line in welcome.md Signed-off-by: Kartikay <[email protected]> * retrigger checks Signed-off-by: Kartikay <[email protected]> --------- Signed-off-by: Kartikay <[email protected]> * Lazy downloaded index header (#6984) * lazy downloaded index header Signed-off-by: Ben Ye <[email protected]> * update tests Signed-off-by: Ben Ye <[email protected]> * address comments Signed-off-by: Ben Ye <[email protected]> * address comments Signed-off-by: Ben Ye <[email protected]> * changelog Signed-off-by: Ben Ye <[email protected]> --------- Signed-off-by: Ben Ye <[email protected]> * tests: use remote write in query frontend tests (#6998) * query-frontend: Added support of auto_discovery for memcached (#7004) * query-frontend: Added support of auto_discovery for memcached Signed-off-by: Vasiliy Rumyantsev <[email protected]> * adjustments to build on main branch Signed-off-by: Vasiliy Rumyantsev <[email protected]> * CHANGELOG.md Signed-off-by: Vasiliy Rumyantsev <[email protected]> * typo fixed Signed-off-by: Vasiliy Rumyantsev <[email protected]> * minor fixex after review Signed-off-by: Vasiliy Rumyantsev <[email protected]> --------- Signed-off-by: Vasiliy Rumyantsev <[email protected]> * Fix URI encoding of strings Signed-off-by: Kartikay <[email protected]> * ui: enable partial response strategy by default Rebuild Signed-off-by: Vanshika <[email protected]> * Added negative offset check for caching queries (#7011) Signed-off-by: pawarpranav83 <[email protected]> Co-authored-by: pawarpranav83 <[email protected]> * Update MAINTAINERS.md (Bartek's company) Should be done in January last year... Signed-off-by: Bartlomiej Plotka <[email protected]> * store: add chunksize tests to acceptance tests * add chunk size tests to acceptance tests * refactor acceptance tests slightly Signed-off-by: Michael Hoffmann <[email protected]> * added tasrie it services as support partner (#7023) * added tasrie it services as support partner Signed-off-by: Tasrie IT Services <[email protected]> * added the link in the list.html page Signed-off-by: Tasrie IT Services <[email protected]> * fixed the logo resolution to 190x70 Signed-off-by: Tasrie IT Services <[email protected]> * Remove extraneous text Signed-off-by: Saswata Mukherjee <[email protected]> --------- Signed-off-by: Tasrie IT Services <[email protected]> Signed-off-by: Saswata Mukherjee <[email protected]> Co-authored-by: Saswata Mukherjee <[email protected]> * Query Frontend: Add tenant label to metrics (#6887) This commit adds a tenant label to the HTTP metrics which are exported by the Query Frontend. Signed-off-by: Jacob Baungard Hansen <[email protected]> * tests: use remote write in query frontend tests (#7017) * resolved issue #7024 Signed-off-by: Pratham Agarwal <[email protected]> * QueryFrontend|Query: Create new arg to enable extended functions (#7028) * Adding new parameter for extended functions in querier Signed-off-by: Pedro Tanaka <[email protected]> * Adding new flag for QFE Signed-off-by: Pedro Tanaka <[email protected]> * improve argument passing in query side Signed-off-by: Pedro Tanaka <[email protected]> * Adding changelog Signed-off-by: Pedro Tanaka <[email protected]> * Adding e2e test for query Signed-off-by: Pedro Tanaka <[email protected]> * undoing uneeded changes Signed-off-by: Pedro Tanaka <[email protected]> * fixing docs Signed-off-by: Pedro Tanaka <[email protected]> * fixing e2e tests Signed-off-by: Pedro Tanaka <[email protected]> * Fixing backward compat test Signed-off-by: Pedro Tanaka <[email protected]> * changes from CR Signed-off-by: Pedro Tanaka <[email protected]> --------- Signed-off-by: Pedro Tanaka <[email protected]> Signed-off-by: Pedro Tanaka <[email protected]> * receive: respect forward timeout in http handler buckets (#7030) Forward timeout is what ultimately decides how long a HTTP request might take so it doesn't make sense to hard-code max value of 5. In this pull request I propose respecting the configured forward timeout and adding extra buckets if the default buckets don't cover it completely. Signed-off-by: Giedrius Statkevičius <[email protected]> * receive: do not leak grpc connections (#7031) * receive: do not leak grpc connections Prevent a leak in gRPC connections by garbage collecting old ones when the hashring changes. For that purpose, I propose adding a `Nodes() string` method so that it would be possible to know what nodes do not exist in the hashring anymore. Signed-off-by: Giedrius Statkevičius <[email protected]> * receive: change order of operations Signed-off-by: Giedrius Statkevičius <[email protected]> --------- Signed-off-by: Giedrius Statkevičius <[email protected]> * fix multi-tenancy link (#7032) Signed-off-by: Alec Rajeev <[email protected]> * removing todo comments from query docs Signed-off-by: Harsh Pratap Singh <[email protected]> * Query: add optional tenancy enforcement (#6756) * Query: add optional tenancy enforcement With this commit it's now possible to enable enforcement of tenancy. If tenancy is enabled, a tenant label will be added to queries based on the tenant information provided by the tenant header, and the tenant-label-name. The implementation for query APIs are done by using prom-label-proxy as library, while the implementation for non-query APIs are written from scratch. Signed-off-by: Jacob Baungard Hansen <[email protected]> * Add changelog entry Signed-off-by: Jacob Baungard Hansen <[email protected]> * Query: Add non-default tenant testcase Signed-off-by: Jacob Baungard Hansen <[email protected]> * Test: make query a constant to make linter happy Signed-off-by: Jacob Baungard Hansen <[email protected]> * Address review comments - Remove empty lines - If multiple tenant matchers are found in the original query, we only replace the first one with the header provided tenant, and remove any subsequent ones. Signed-off-by: Jacob Baungard Hansen <[email protected]> * Address review comments - `--enable-tenancy` -> `--enforce-tenancy` - Create `RewritePromQL` and `RewriteLabelMatchers` to clean up code in query api. Also move getLabelMatchers to tenancy pkg. - Use prom-label-proxys `EnforceMatchers` to rewrite labels on non-query APIs instead of own solution - Don't specifically handle `illegalLabelMatcherError` Signed-off-by: Jacob Baungard Hansen <[email protected]> * Re-arrage go.mod to make linter happy. Signed-off-by: Jacob Baungard Hansen <[email protected]> * Address review comments Minor changes to CLI docs, code-comments and changelog. Signed-off-by: Jacob Baungard Hansen <[email protected]> * Query: Add tenancy docs This commit adds documentation for the tenancy features. Signed-off-by: Jacob Baungard Hansen <[email protected]> * Update docs/components/query.md Review comment Co-authored-by: Saswata Mukherjee <[email protected]> Signed-off-by: Jacob Baungård Hansen <[email protected]> --------- Signed-off-by: Jacob Baungard Hansen <[email protected]> Signed-off-by: Jacob Baungård Hansen <[email protected]> Co-authored-by: Saswata Mukherjee <[email protected]> * Tests: unique docker env name for tenant tests (#7038) The e2e tests would occasionally fail due to non-unqiue docker environment names. With this commit the tests are environments are given unique names to avoid these failures. Signed-off-by: Jacob Baungard Hansen <[email protected]> * changed default 3m to 15m Signed-off-by: Player256 <[email protected]> * Fix docs (#7039) Fix docs post #6539 merge. Signed-off-by: Filip Petkovski <[email protected]> * Revert "receive: upload compacted blocks if OOO enabled (#6974)" (#7053) This reverts commit 7b8eb86c0ff3b1144aa1cea392806afa40a4cdf8. Proper way to handle this is to disable vertical compaction. I am trying to add this functionality here: https://github.com/prometheus/prometheus/pull/13393 Signed-off-by: Giedrius Statkevičius <[email protected]> * [CHORE] adding thanos upload-snapshot command (#6884) Signed-off-by: Nicolas Takashi <[email protected]> * fix docs error Signed-off-by: Kartikay <[email protected]> * cmd: make bucket upload command take lset from flags (#7059) Signed-off-by: Michael Hoffmann <[email protected]> * CHANGELOG: mark 0.34 as in progress Signed-off-by: Michael Hoffmann <[email protected]> * VERSION: cut release 0.34.0-rc.0 Signed-off-by: Michael Hoffmann <[email protected]> * UI: Don't always force tracing (#7062) Forced tracing was.. Forced true always, even if the checkbox in the UI to enable tracing was not actually checked. Signed-off-by: Jacob Baungard Hansen <[email protected]> * go.mod: update Prometheus version (#7047) Update Prometheus version to include https://github.com/prometheus/prometheus/pull/13242 which is important for me - it unblocks further postings work. Signed-off-by: Giedrius Statkevičius <[email protected]> * Make RetryError and HaltError able to be fetched for root cause (#7043) * Make RetryError and HaltError able to be fetched for root cause Signed-off-by: Alex Le <[email protected]> * Added unit test Signed-off-by: Alex Le <[email protected]> * fix lint Signed-off-by: Alex Le <[email protected]> * fixed IsRetryError and IsHaltError functions Signed-off-by: Alex Le <[email protected]> --------- Signed-off-by: Alex Le <[email protected]> * receive: disable overlapping compaction Use the new TSDB flag to disable overlapping compaction to fix OOO samples handling in the Receive component. Signed-off-by: Giedrius Statkevičius <[email protected]> * CI: Ensure static react-app is checked in (#7063) * CI: Ensure static react-app is checked in With this commit the CI system should fail if changes to the react-app has been made without checking in the changes. Signed-off-by: Jacob Baungard Hansen <[email protected]> * Add `react-app` as dependency `check-react-app` To ensure the react-app is rebuilt before checking for changes. Signed-off-by: Jacob Baungard Hansen <[email protected]> --------- Signed-off-by: Jacob Baungard Hansen <[email protected]> * Receive: refactor handler for improved readability and organization (#6898) * [wip] First checkpoint Signed-off-by: Douglas Camata <[email protected]> * [wip] Second checkpoint All tests passing, unit and e2e. Signed-off-by: Douglas Camata <[email protected]> * Small random refactors Signed-off-by: Douglas Camata <[email protected]> * Add some useful trace tags Signed-off-by: Douglas Camata <[email protected]> * Concurrent and traced local writes Signed-off-by: Douglas Camata <[email protected]> * Improve variable names in remote writes Signed-off-by: Douglas Camata <[email protected]> * Rename `newFanoutForward` function Signed-off-by: Douglas Camata <[email protected]> * More refactors Signed-off-by: Douglas Camata <[email protected]> * Fix linting issue Signed-off-by: Douglas Camata <[email protected]> * Add a quorum test with sloppy quorum Signed-off-by: Douglas Camata <[email protected]> * [wip] Try to make retries work Signed-off-by: Douglas Camata <[email protected]> * [wip] Checkpoint: wait group still hanging Signed-off-by: Douglas Camata <[email protected]> * Some refactors Signed-off-by: Douglas Camata <[email protected]> * Add some commented code so I don't lose it Signed-off-by: Douglas Camata <[email protected]> * Adapt tests Signed-off-by: Douglas Camata <[email protected]> * Remove sloppy quorum code Signed-off-by: Douglas Camata <[email protected]> * Move some code around Signed-off-by: Douglas Camata <[email protected]> * Remove even more leftover of sloppy quorum Signed-off-by: Douglas Camata <[email protected]> * Extract a type to hold function params Signed-off-by: Douglas Camata <[email protected]> * Remove unused struct field Signed-off-by: Douglas Camata <[email protected]> * Remove useless variable Signed-off-by: Douglas Camata <[email protected]> * Remove type that wasn't used enough Signed-off-by: Douglas Camata <[email protected]> * Delete function to tighten up max buffered responses Signed-off-by: Douglas Camata <[email protected]> * Add comments to some functions Signed-off-by: Douglas Camata <[email protected]> * Fix peer up check Signed-off-by: Douglas Camata <[email protected]> * Fix size of replication tracking slices Signed-off-by: Douglas Camata <[email protected]> * Rename context Signed-off-by: Douglas Camata <[email protected]> * Don't do local writes concurrently Signed-off-by: Douglas Camata <[email protected]> * Remove extra error logging Signed-off-by: Douglas Camata <[email protected]> * Fix syntax after merge Signed-off-by: Douglas Camata <[email protected]> * Add missing methods to peersContainer Signed-off-by: Douglas Camata <[email protected]> * Fix handler test Signed-off-by: Douglas Camata <[email protected]> * Reset peers state on hashring changes Signed-off-by: Douglas Camata <[email protected]> * Handle PR comment regarding waitgroup Signed-off-by: Douglas Camata <[email protected]> * Set span tags to help debug Signed-off-by: Douglas Camata <[email protected]> * Fix concurrency issue We close the request as soon as quorum is reached and leave a few Go routines running to finish replication and so cleanups. This means that the context from the HTTP request is cancelled... which ends up also cancelling the pending replication requests. Signed-off-by: Douglas Camata <[email protected]> * Fix request ID middleware Signed-off-by: Douglas Camata <[email protected]> * Fix `distributeTimeseriesToReplicas` comment Signed-off-by: Douglas Camata <[email protected]> * Extract var with 1-indexed replication index Signed-off-by: Douglas Camata <[email protected]> * Rename methods in peersContainer interface Signed-off-by: Douglas Camata <[email protected]> * Make peerGroup `getConnection` check if peers are up Signed-off-by: Douglas Camata <[email protected]> * Remove yet one more not useful log Signed-off-by: Douglas Camata <[email protected]> * Remove logger from `h.sendWrites` Signed-off-by: Douglas Camata <[email protected]> --------- Signed-off-by: Douglas Camata <[email protected]> * Upgrade grpc to 1.57.2 (#7078) 1、In the replace of go.mod, due to weaveworks/common#239, The grpc version is 1.45.0, but there are vulnerabilities in this version. In order to fix CVE-2023-44478, the grpc version needs to be upgraded to 1.57.2 2、In order to upgrade GRPC, the version of weaveworks/common also needs to be upgraded, otherwise the build will fail Signed-off-by: hanyuting8 <[email protected]> * Store: acceptance test for proxy store (#7084) * Add basic acceptance tests for proxy store * Fix bug where invalid requests got ignored because of partial response strategy Signed-off-by: Michael Hoffmann <[email protected]> * Fix lazy postings with zero length (#7083) * fix lazy postings with zero length Signed-off-by: Ben Ye <[email protected]> * changelog Signed-off-by: Ben Ye <[email protected]> * unit tests Signed-off-by: Ben Ye <[email protected]> * fix doc Signed-off-by: Ben Ye <[email protected]> --------- Signed-off-by: Ben Ye <[email protected]> * Store: fix label values edge case (#7082) If the requested label is an external label and we have series matchers we should only return results if the series matchers actually match a series. Signed-off-by: Michael Hoffmann <[email protected]> * VERSION: cut release 0.34.0-rc.1 Signed-off-by: Michael Hoffmann <[email protected]> * receive: race condition in handler Close() when stopped early (#7087) Receiver hangs waiting for the HTTP Hander to shutdown if an error occurs before Handler is initialized. This might happen, for example, if the hashring is too small for a given replication factor. Signed-off-by: Mikhail Nozdrachev <[email protected]> * receive: use async remote writing (#7045) * CHANGELOG: cut release 0.34 (#7095) Signed-off-by: Michael Hoffmann <[email protected]> * Stores: convert tests to not rely on slice labels (#7098) Signed-off-by: Michael Hoffmann <[email protected]> * all: get rid of query pushdown to simplify query path (#7014) Signed-off-by: Michael Hoffmann <[email protected]> * Store: dont rely on slice labels continued Signed-off-by: Michael Hoffmann <[email protected]> * Update prometheus/prometheus (#7096) * Update prometheus/prometheus This commit updates prometheus/prometheus to latest main (60b6266e). Signed-off-by: Filip Petkovski <[email protected]> * Fix file discovery Signed-off-by: Filip Petkovski <[email protected]> --------- Signed-off-by: Filip Petkovski <[email protected]> * receive/handler: fix locking twice (#7112) Fix bug introduced in https://github.com/thanos-io/thanos/pull/6898: we were RLock()ing twice. This leads to a deadlock in some situations. Signed-off-by: Giedrius Statkevičius <[email protected]> * fix minio store gateway err (#7114) Signed-off-by: Kartikay <[email protected]> * receive/handler: do not double lock (#7124) markPeerUnavailable was always taking a lock and in one case we were calling it with a lock already taken. Fix this. Signed-off-by: Giedrius Statkevičius <[email protected]> * default to alertmanager v2 api (#7123) Signed-off-by: Jake Keeys <[email protected]> * Receive: dont rely on slice labels (#7100) Signed-off-by: Michael Hoffmann <[email protected]> * Upgrade grpc to 1.57.2 (#7078) 1、In the replace of go.mod, due to weaveworks/common#239, The grpc version is 1.45.0, but there are vulnerabilities in this version. In order to fix CVE-2023-44478, the grpc version needs to be upgraded to 1.57.2 2、In order to upgrade GRPC, the version of weaveworks/common also needs to be upgraded, otherwise the build will fail Signed-off-by: hanyuting8 <[email protected]> Signed-off-by: Michael Hoffmann <[email protected]> * VERSION: cut release 0.34.1 Signed-off-by: Michael Hoffmann <[email protected]> * docs: update helm installation instruction the prometheus helm chart is a community maintained chart since a few years. With that, the old example pointed to an old chart and the provided example values aren't also working anymore. This update the documentation. Signed-off-by: Mario Constanti <[email protected]> * docs: fix link (#7129) The link has moved to another since Cisco bought Banzai Cloud. Signed-off-by: Giedrius Statkevičius <[email protected]> * docs: run make docs for helm installation instruction Signed-off-by: Mario Constanti <[email protected]> * Fixing log line for remote engine in debug mode (#7133) Signed-off-by: Pedro Tanaka <[email protected]> * Adding new method on BucketedBytes to expose used memory (#7137) * Adding new method on bucketed bytes to expose used Signed-off-by: Pedro Tanaka <[email protected]> * Removing interface, using RWMutex Signed-off-by: Pedro Tanaka <[email protected]> --------- Signed-off-by: Pedro Tanaka <[email protected]> * Adding InfraCloud as Enterprise support partner (#7141) * adding InfraCloud as Enterprise support partner Signed-off-by: Chetan Deshmukh <[email protected]> * replaced svg file to match layout Signed-off-by: Chetan Deshmukh <[email protected]> * added alt-text and horizontal image Signed-off-by: Chetan Deshmukh <[email protected]> --------- Signed-off-by: Chetan Deshmukh <[email protected]> * docs: fix link (#7129) The link has moved to another since Cisco bought Banzai Cloud. Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: Michael Hoffmann <[email protected]> * cache: attach object storage hash to iter key (#6880) Attach object storage hash to the iter key so that it would be possible to reuse the same cache storage e.g. Redis for different buckets. Without this, the results are funny to say the least if you accidentally attempt to do that. Thus, let's add the hash to reduce the possibility of an accident for our users. Signed-off-by: Giedrius Statkevičius <[email protected]> * Add support for extended promql functions in rule (#7105) Adds a flag to register the extended promql functions supported by the thanos query engine when running the rule component. This will allow rule config files containing query expressions with (xrate / xincrease / xdelta) to pass validation. This will only work if the query endpoint in use is running the thanos engine. Signed-off-by: Samuel Dufel <[email protected]> * Extended func support - doc update (#7161) * Add support for extended promql functions in rule Adds a flag to register the extended promql functions supported by the thanos query engine when running the rule component. This will allow rule config files containing query expressions with (xrate / xincrease / xdelta) to pass validation. This will only work if the query endpoint in use is running the thanos engine. Signed-off-by: Samuel Dufel <[email protected]> * Update rendered docs with added flag Signed-off-by: Samuel Dufel <[email protected]> --------- Signed-off-by: Samuel Dufel <[email protected]> * Copy labels from remote instant queries (#7151) Similar to https://github.com/thanos-io/thanos/pull/6957, we should copy labels from remote instant queries so that memory does not get overwritten when processing series in a central engine. Signed-off-by: Filip Petkovski <[email protected]> * e2e/query_frontend: add tests for explain/analyze (#7160) Adding tests for explain/analyze with QFE. Will add fixes as separate PR. Signed-off-by: Giedrius Statkevičius <[email protected]> * cache/caching_bucket: add path to hash (#7158) Add path to the hash. This allows identifying difference instances by different config paths. Signed-off-by: Giedrius Statkevičius <[email protected]> * Allow using different listing strategies (#7134) * Allow using different listing strategies Signed-off-by: Filip Petkovski <[email protected]> * Expose flags for block list strategy Signed-off-by: Filip Petkovski <[email protected]> * Run make docs Signed-off-by: Filip Petkovski <[email protected]> * Fix whitespace Signed-off-by: Filip Petkovski <[email protected]> * Add CHANGELOG entry Signed-off-by: Filip Petkovski <[email protected]> --------- Signed-off-by: Filip Petkovski <[email protected]> * cache: implement the circuit breaker pattern for asynchronous set operations in the cache client (#7010) * Implement the circuit breaker pattern for asynchronous set operations in the cache client Signed-off-by: Xiaochao Dong (@damnever) <[email protected]> * Add feature flag for circuitbreaker Signed-off-by: Xiaochao Dong (@damnever) <[email protected]> * Sync docs Signed-off-by: Xiaochao Dong (@damnever) <[email protected]> * Skip configuration validation if the circuit breaker is disabled Signed-off-by: Xiaochao Dong (@damnever) <[email protected]> * Make lint happy Signed-off-by: Xiaochao Dong (@damnever) <[email protected]> * Abstract the logic of the circuit breaker Signed-off-by: Xiaochao Dong (@damnever) <[email protected]> --------- Signed-off-by: Xiaochao Dong (@damnever) <[email protected]> * queryfrontend: fix analysis after API changes Fix the analysis functionality with query-frontend after the recent changes. Added tests for this. Signed-off-by: Giedrius Statkevičius <[email protected]> * bugfix: lazy posting optimization with wrong cardinality for estimation (#7122) * bugfix: catch lazy posting optimization using wrong cardinality for estimation Signed-off-by: Ben Ye <[email protected]> * update changelog Signed-off-by: Ben Ye <[email protected]> --------- Signed-off-by: Ben Ye <[email protected]> * *: properly treat native histogram deduplication in chunk series merger We have detected a problem in the chunk seriers merger where it will panic in case it encounters native histogram chunks. I am using thanos as a library for a project and wanted to use the penalty function to dedup blocks from Prometheus instances. Signed-off-by: Pedro Tanaka <[email protected]> * Query UI: Add tenant box (#6867) * Query UI: Add tenant box With this commit as tenant box is added to the query UI. It can be used to specify which tenant to use when making a query. Signed-off-by: Jacob Baungard Hansen <[email protected]> * Re-compile static react app Recompiles the static react app as now needed following: https://github.com/thanos-io/thanos/pull/6900 Signed-off-by: Jacob Baungard Hansen <[email protected]> * Move changelog item to appropiate future release After merging it was under the 0.34 release. Signed-off-by: Jacob Baungard Hansen <[email protected]> * Move query path tenancy proposal to done Signed-off-by: Jacob Baungard Hansen <[email protected]> --------- Signed-off-by: Jacob Baungard Hansen <[email protected]> * receive: add support for globbing tenant specifiers (#7155) We want to be able to route all tenants which begin with certain letters to some receivers so we need to have some kind of globbing/regex support in the hashring. This PR adds that functionality. We've been using this in prod successfully. Signed-off-by: Giedrius Statkevičius <[email protected]> * .circleci: bump setup_docker_version version (#7177) The current image is deprecated. See https://discuss.circleci.com/t/remote-docker-image-deprecations-and-eol-for-2024/50176. Signed-off-by: Giedrius Statkevičius <[email protected]> * Unhide distributed execution mode This commit exposes the distributed query execution mode to end-users by unhiding the flag used to toggle this feature. The commit also adds documentation on when the mode is appropriate to be used. Signed-off-by: Filip Petkovski <[email protected]> * Add changelog entry Signed-off-by: Filip Petkovski <[email protected]> * cortex/querier: fix analysis merging (#7179) We were not merging analysis properly - mergo was overwriting data. Instead of using a whole library for this, just write two small functions and use them. Add test to cover this. Signed-off-by: Giedrius Statkevičius <[email protected]> * Fix docs Fixes docs formatting and updates the distributed execution link to the done proposal. Signed-off-by: Filip Petkovski <[email protected]> * Added platformengineers Signed-off-by: Munir Khakhi <[email protected]> * Added platformengineers (#7181) Signed-off-by: Munir Khakhi <[email protected]> * Bump google.golang.org/protobuf to v1.33.0 (#7191) This PR bumps the version of google.golang.org/protobuf to v1.33.0 fix a potential vulnerability in the protojson.Unmarhsl function [1] that can occure when unmarshaling a message with a protobuf value. Even if the function isn't used directly in Thanos it would be safer to just bump it directly. [1] https://pkg.go.dev/vuln/GO-2024-2611 Signed-off-by: Daniel Mellado <[email protected]> * fix: make responsive support page and change size of platform-engineer logo fix: add anchor tag to all images Signed-off-by: Payal17122000 <[email protected]> * downsample: retry objstore related errors Signed-off-by: Vasiliy Rumyantsev <[email protected]> * downsample: retry objstore related errors Signed-off-by: Vasiliy Rumyantsev <[email protected]> * rule: do not turn off if resolving fails (#7192) Do not turn off Ruler if resolving fails. We can still (try to) evaluate rules even if Alertmanager is not available. Signed-off-by: Giedrius Statkevičius <[email protected]> * Query UI: Only show tenant box with enforcement on (#7186) With this commit we only show the tenant-ui box when enforcement of tenancy is on, as it is not needed otherwise. Signed-off-by: Jacob Baungard Hansen <[email protected]> * Reloader: Add support for watching and decompressing Prometheus configuration directories (#7199) Signed-off-by: Daniel Hrabovcak <[email protected]> * [FEAT] Adding blog post (#7202) Signed-off-by: Helia Barroso <[email protected]> Co-authored-by: Helia Barroso <[email protected]> * storepb: make ServerAsClient channels unbuffered Signed-off-by: Michael Hoffmann <[email protected]> * Add support for TSDB selector in querier (#7200) * Add support for TSDB selector in querier This PR allows using the query distributed mode against a set of multi-tenant receivers as described in https://github.com/thanos-io/thanos/blob/main/docs/proposals-done/202301-distributed-query-execution.md#distributed-execution-against-receive-components. The feature is enabled by a selector.relabel-config flag in the Query component which allows it to select a subset of TSDBs to query based on their external labels. Signed-off-by: Filip Petkovski <[email protected]> * Add CHANGELOG entry and fix docs Signed-off-by: Filip Petkovski <[email protected]> * Fix tests Signed-off-by: Filip Petkovski <[email protected]> * Add comments Signed-off-by: Filip Petkovski <[email protected]> * Add test case for MatchersForLabelSets Signed-off-by: Filip Petkovski <[email protected]> * Fix failing test Signed-off-by: Filip Petkovski <[email protected]> * Use an unbuffered channel Signed-off-by: Filip Petkovski <[email protected]> * Change flag description Signed-off-by: Filip Petkovski <[email protected]> * Remove parameter from ServerAsClient Signed-off-by: Filip Petkovski <[email protected]> --------- Signed-off-by: Filip Petkovski <[email protected]> * Update thanos-io/promql-engine (#7215) * Update thanos-io/promql-engine This commit updates the promql-engine module to latest main and modifies to remote engine based on the breaking change. Signed-off-by: Filip Petkovski <[email protected]> * Fix lint Signed-off-by: Filip Petkovski <[email protected]> --------- Signed-off-by: Filip Petkovski <[email protected]> * expose NewPromSeriesSet (#7214) Signed-off-by: Ben Ye <[email protected]> * [CHORE] adding auto GOMEMLIMIT flag (#7223) Co-authored-by: Filip Petkovski <[email protected]> * bump objstore package version to latest main (#7228) Signed-off-by: Ben Ye <[email protected]> * Query-frontend: fix missing redis username config (#7224) * add username cfg to rueidis client Signed-off-by: Thibault Mange <[email protected]> * update changelog Signed-off-by: Thibault Mange <[email protected]> --------- Signed-off-by: Thibault Mange <[email protected]> * UI: Showing Block Size Stats (#7233) * feat(ui): added BlockSizeStats calculation to blocks page A block can have a list of contained files set in `.thanos.files`. If the `files` array is set, all referenced files with `size_bytes` set are counted: - sum of all `chunk/*` file sizes - size of index file - total size (sum of both) Shows statistics about the selected block in the block details view: - Total size of block - Size of index (and percentage of total) - Size of all chunks (and percentage of total) - Daily growth, based on total size and block duration Output is humanized up to Pebibytes and fixed to two decimal places; raw bytes are accessible through mouse over / title text. Signed-off-by: Markus Möslinger <[email protected]> * feat(ui): added aggregated BlockSizeStats to blocks row title Added total size of all blocks from a source to the row title, beneath the source name. The shown total size is humanized up to pebibytes and fixed to two decimal places; raw bytes value is accessible through mouse over / title text. The shown value will refresh with selected compaction levels, but doesn't take block filter into account. I thought about showing daily growth as well, but just summing all milliseconds of all blocks doesn't work with overlapping blocks / multiple resolutions. Signed-off-by: Markus Möslinger <[email protected]> * chore(docs): added UI block size PR to CHANGELOG.md Signed-off-by: Markus Möslinger <[email protected]> * chore(ui): removed comments Automatic code formatting duplicated some comments near import statements. Signed-off-by: Markus Möslinger <[email protected]> --------- Signed-off-by: Markus Möslinger <[email protected]> * Fix lazy expanded postings cache and bug of non equal matcher (#7220) * fix lazy expanded postings cache and bug of non equal matcher with non existent values Signed-off-by: Ben Ye <[email protected]> * test case for remove keys noop Signed-off-by: Ben Ye <[email protected]> * add promqlsmith fuzz test Signed-off-by: Ben Ye <[email protected]> * update Signed-off-by: Ben Ye <[email protected]> * changelog Signed-off-by: Ben Ye <[email protected]> * fix go mod Signed-off-by: Ben Ye <[email protected]> * rename test Signed-off-by: Ben Ye <[email protected]> * fix series request timestamp Signed-off-by: Ben Ye <[email protected]> * skip e2e test Signed-off-by: Ben Ye <[email protected]> * handle non lazy expanded case Signed-off-by: Ben Ye <[email protected]> * update comment Signed-off-by: Ben Ye <[email protected]> --------- Signed-off-by: Ben Ye <[email protected]> * Bump Prometheus to include new label regex optimization (#7232) * bump Prometheus version to include new label matcher regex value optimization Signed-off-by: Ben Ye <[email protected]> * update Signed-off-by: Ben Ye <[email protected]> * fix again Signed-off-by: Ben Ye <[email protected]> * include latest fix Signed-off-by: Ben Ye <[email protected]> * update go mod Signed-off-by: Ben Ye <[email protected]> * fix explain test Signed-off-by: Ben Ye <[email protected]> * fix test again Signed-off-by: Ben Ye <[email protected]> * update again Signed-off-by: Ben Ye <[email protected]> * update Signed-off-by: Ben Ye <[email protected]> * fix tests so far Signed-off-by: Ben Ye <[email protected]> * fix compactor tests Signed-off-by: Ben Ye <[email protected]> * use own out of order chunk index Signed-off-by: Ben Ye <[email protected]> --------- Signed-off-by: Ben Ye <[email protected]> * update docs for receive routing only with limits (#7241) Signed-off-by: Alec Rajeev <[email protected]> * docs: add thanoscon 2024 talks (#7243) Signed-off-by: Michael Hoffmann <[email protected]> * remove write method from Compactor interface (#7246) Signed-off-by: Ben Ye <[email protected]> * change shipper to not overwrite all external labels (#7247) Signed-off-by: Ben Ye <[email protected]> * fix(changelog): fix GOMEMLIMIT pull request reference Signed-off-by: roth-wine <[email protected]> * Update kakkoyun's affiliation (#7251) * Added Shield in adopters (#7254) * Added Shield in adopters Signed-off-by: suhas.chikkanna.shield <[email protected]> * Upload compatible image Signed-off-by: suhas-chikkanna <[email protected]> --------- Signed-off-by: suhas.chikkanna.shield <[email protected]> Signed-off-by: suhas-chikkanna <[email protected]> Co-authored-by: suhas.chikkanna.shield <[email protected]> * Tracing: added missing sampler types (#7231) * added missing sampler types Signed-off-by: Neeraj Nagure <[email protected]> * added changelog entry Signed-off-by: Neeraj Nagure <[email protected]> * fixed changelog entry Signed-off-by: Neeraj Nagure <[email protected]> * Fixed changelog entry conflict Signed-off-by: Neeraj Nagure <[email protected]> --------- Signed-off-by: Neeraj Nagure <[email protected]> * fix query_test when --race enabled (#7258) Signed-off-by: Yi Jin <[email protected]> * go.mod: bump promql-engine (#7263) Bump promql-engine version to include samples counting. Signed-off-by: Giedrius Statkevičius <[email protected]> * Fix 7244 error targets page (#7245) * added UNKNOWN to TargetHealth_value at targets proto Signed-off-by: Tidhar Klein Orbach <[email protected]> * added TargetHealth_value UNKNOWN to rpc.pb.go Signed-off-by: Tidhar Klein Orbach <[email protected]> --------- Signed-off-by: Tidhar Klein Orbach <[email protected]> * clarify documentation around selecor.relabel-config option Signed-off-by: notafile <[email protected]> * clarify writing around sharding Signed-off-by: notafile <[email protected]> * don't halt compaction due to overlapping sources when vertical compaction is enabled (#7225) Signed-off-by: Ben Ye <[email protected]> * Propagate the query plan * Serialize the plan for remote executions latest engine Signed-off-by: Pedro Tanaka <[email protected]> Propagate marshaled plan and introduce optimizer Propagating the query plan in the remote engine requests and introduce new SetProjectionColumns optimizer Signed-off-by: Pedro Tanaka <[email protected]> * Fixing passing down of plan Signed-off-by: Pedro Tanaka <[email protected]> * go mod tidy Signed-off-by: Pedro Tanaka <[email protected]> * avoid panics Signed-off-by: Pedro Tanaka <[email protected]> * delete dev file Signed-off-by: Pedro Tanaka <[email protected]> * undo small refactor Signed-off-by: Pedro Tanaka <[email protected]> * improve test Signed-off-by: Pedro Tanaka <[email protected]> --------- Signed-off-by: Pedro Tanaka <[email protected]> generating protos Signed-off-by: Pedro Tanaka <[email protected]> fixing v1 Signed-off-by: Pedro Tanaka <[email protected]> delete unused method Signed-off-by: Pedro Tanaka <[email protected]> Set projection labels after distributing queries * removing second precision engine, upstream already truncates Signed-off-by: Pedro Tanaka <[email protected]> * Passing the plan along as the query in remote executions Signed-off-by: Pedro Tanaka <[email protected]> * Using proper constructors passing the query plan Signed-off-by: Pedro Tanaka <[email protected]> * Refactor query creation from plan Signed-off-by: Pedro Tanaka <[email protected]> * fallback in case we cant use plan Signed-off-by: Pedro Tanaka <[email protected]> * refactor, add tests Signed-off-by: Pedro Tanaka <[email protected]> * Refactor to method Signed-off-by: Pedro Tanaka <[email protected]> * last nits Signed-off-by: Pedro Tanaka <[email protected]> * api/ui: show peak/total samples in analyze (#7269) Show the new peak/total fields in analyze output next to each operator. Add tooltips to explain what is the meaning of each field. Signed-off-by: Giedrius Statkevičius <[email protected]> * receive/handler: implement tenant label splitting (#7256) * receive/handler: implement tenant label splitting Implement splitting incoming HTTP requests along some label inside of the timeseries themselves. This functionality is useful when you have one big application exposing lots of series and, for instance, you have a label `team` that identifies different owners of metrics in that application. Then using this you can use that `team` label to have different tenants in Thanos. Only negative thing that I could spot is that if after splitting one of the requests fails then that code is used for all tenants and that skews the Receiver metrics a little bit. I think that can be left as a TODO task. Signed-off-by: Giedrius Statkevičius <[email protected]> * test/e2e: add more receiver tests Signed-off-by: Giedrius Statkevičius <[email protected]> * thanos/receive: note that splitting takes precendence over HTTP Signed-off-by: Giedrius Statkevičius <[email protected]> * thanos/receive: fix typo Signed-off-by: Giedrius Statkevičius <[email protected]> --------- Signed-off-by: Giedrius Statkevičius <[email protected]> Signed-off-by: Giedrius Statkevičius <[email protected]> * query: fixing dedup iterator when working on mixed sample types (#7271) * query: fixing dedup iterator when working on mixed sample types There was a panic in case the dedupiterator worked on two chunks with both Native Histograms and Float (XOR encoded). Co-authored-by: Sebastian Rabenhorst <[email protected]> Signed-off-by: Pedro Tanaka <[email protected]> * Adding changelog Signed-off-by: Pedro Tanaka <[email protected]> * fixing lint Signed-off-by: Pedro Tanaka <[email protected]> * removing comments Signed-off-by: Pedro Tanaka <[email protected]> * Fixing repro test case Signed-off-by: Pedro Tanaka <[email protected]> * fixing initialization Signed-off-by: Pedro Tanaka <[email protected]> * fixing changelog Signed-off-by: Pedro Tanaka <[email protected]> * adding header to new file Signed-off-by: Pedro Tanaka <[email protected]> * using t.Run Signed-off-by: Pedro Tanaka <[email protected]> * fixing ordering of samples in tests Signed-off-by: Pedro Tanaka <[email protected]> --------- Signed-off-by: Pedro Tanaka <[email protected]> Co-authored-by: Sebastian Rabenhorst <[email protected]> * change the reflect package to an unsafe package (#7143) - as 'reflect.String.Header' is deprecated, it is replaced with an unsafe package. Signed-off-by: Youngjun <[email protected]> * Receive: fix issue-7248 with parallel receive_forward (#7267) * Receive: fix issue-7248 by introducing a worker pool Signed-off-by: Yi Jin <[email protected]> * fix unit test bug Signed-off-by: Yi Jin <[email protected]> * fix CLI flags not pass into the receive handler Signed-off-by: Yi Jin <[email protected]> * address comments Signed-off-by: Yi Jin <[email protected]> * init context in constructor Signed-off-by: Yi Jin <[email protected]> --------- Signed-off-by: Yi Jin <[email protected]> * Fix incorrect comments (#7268) Signed-off-by: Magiceses <[email protected]> * [CHORE] adding user agent (#7281) Signed-off-by: Nicolas Takashi <[email protected]> * receive/multitsdb: do not delete not uploaded blocks (#7166) * receive/multitsdb: do not delete not uploaded blocks If a block hasn't been uploaded yet then tell the TSDB layer not to delete them. This prevents a nasty race where the TSDB layer can delete a block before the shipper gets to it. I saw this happen with a very small block. Signed-off-by: Giedrius Statkevičius <[email protected]> * receive/multitsdb: change order Signed-off-by: Giedrius Statkevičius <[email protected]> * shipper/receive: just use a single lock Signed-off-by: Giedrius Statkevičius <[email protected]> --------- Signed-off-by: Giedrius Statkevičius <[email protected]> * Compact: Replace group with resolution in ownsample metrics (#7283) Compaction dowsnample metrics have too high a cardinality, causing metric bloat on large installations. The group information is better suited to logs. * Replace with a resolution label to reduce cardinality. Fixes: #5841 Signed-off-by: SuperQ <[email protected]> * Propagate warnings from instant queries Warnings from remote instant queries get turned into errors, which is a bug. It should be up to the root client to decide whether warnings should be show as such, or converted to errors. Signed-off-by: Filip Petkovski <[email protected]> * Add CHANGELOG entry Signed-off-by: Filip Petkovski <[email protected]> * Receive: stop relying on grpc server config to set grpc client secure/skipVerify Signed-off-by: Guillaume Lecerf <[email protected]> * Show warnings in query frontend (#7289) * Show warnings in query frontend QFE currently does not parse warnings from downstream queriers. This commit fixes that by adding the field to proto messages and modifies the merge function to take warnings into account. Signed-off-by: Filip Petkovski <[email protected]> * Add CHANGELOG entry Signed-off-by: Filip Petkovski <[email protected]> * Omit empty warnings Signed-off-by: Filip Petkovski <[email protected]> --------- Signed-off-by: Filip Petkovski <[email protected]> * Optimize empty posting check in lazy posting (#7298) * change lazy postings empty posting check to use cardinality Signed-off-by: Ben Ye <[email protected]> * update lazy posting test Signed-off-by: Ben Ye <[email protected]> --------- Signed-off-by: Ben Ye <[email protected]> * Sidecar: mark as unqueryable if prometheus is down (#7297) If the prometheus that belongs to a sidecar is down we dont need to query the sidecar. This PR makes it so that we take the sidecar out of the endpoint set then. We do the same for all other store APIs by retuning an error in the info/Info gRPC call if they are marked as not ready. Signed-off-by: Michael Hoffmann <[email protected]> * [CHORE] considering X-Forwarded-For on HTTP Logging (#7303) Signed-off-by: Nicolas Takashi <[email protected]> * Query|Receiver: Do not log full request on ProxyStore by default (#7305) * Query|Receiver|Store: Do not log full request on ProxyStore by default We had a problem on our production where a sudden increase in requests with long matchers was putting our receivers under a lot of pressure. Upon checking profiles we saw that the problem was calls to Log() S…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Instead of spawning new goroutines for each peer that we want to remote write to, spawn a fixed number of worker goroutines and then schedule work on top of them.
This has reduced the number of goroutines in our case about 10x-20x and the 99p of forwarding dropped from ~30s to just a few hundred milliseconds.