Signed-off-by: Ben Ye <[email protected]>

Signed-off-by: Ben Ye <[email protected]>

* Return Query Analysis in API

A param  is added to QueryAPI, if true then query analysis is
returned by the  method of the query having structure
 is returned in response.

Signed-off-by: nishchay-veer <[email protected]>

* Added analyze checkbox in Thanos UI

A analyze checkbox is added to the thanos query api, that requests for operator telemetry which includes CPU Time

Signed-off-by: nishchay-veer <[email protected]>

* Return Query Analysis in API

A param  is added to QueryAPI, if true then query analysis is
returned by the  method of the query having structure
 is returned in response.

Signed-off-by: nishchay-veer <[email protected]>

* Added analyze checkbox in Thanos UI

A analyze checkbox is added to the thanos query api, that requests for operator telemetry which includes CPU Time

Signed-off-by: nishchay-veer <[email protected]>

* Add query explain API

Signed-off-by: Saswata Mukherjee <[email protected]>

* Hooked queryTelemetry data into UI

Signed-off-by: nishchay-veer <[email protected]>

* /query_explain and /query_range_explain for explain-tree

Signed-off-by: nishchay-veer <[email protected]>

* update promql-engine

Signed-off-by: nishchay-veer <[email protected]>

* Execution time shows 0s

Signed-off-by: nishchay-veer <[email protected]>

* Show execution time of operators

Signed-off-by: nishchay-veer <[email protected]>

* Removing QueryExplainParam from query api

Signed-off-by: nishchay-veer <[email protected]>

* bad request format in Explain

Signed-off-by: nishchay-veer <[email protected]>

* Showing Expalin and Analyze Output

Signed-off-by: nishchay-veer <[email protected]>

* Added tooltip and different enpoints for table and graph queries

Signed-off-by: nishchay-veer <[email protected]>

* Linters pass

Signed-off-by: nishchay-veer <[email protected]>

* disable Explain when engine is 'prometheus'

Signed-off-by: nishchay-veer <[email protected]>

* passing query params to explain endpoints

Signed-off-by: nishchay-veer <[email protected]>

* fixed react test case failing

Signed-off-by: nishchay-veer <[email protected]>

* fix ui tests

Signed-off-by: nishchay-veer <[email protected]>

* fix some e2e test fails

Signed-off-by: nishchay-veer <[email protected]>

* added customised tooltip in place of Tooltip component

Signed-off-by: nishchay-veer <[email protected]>

* removed Tooltip from Panel

Signed-off-by: nishchay-veer <[email protected]>

* Linters pass

Signed-off-by: nishchay-veer <[email protected]>

* 4 arguments in QueryInstant

Signed-off-by: nishchay-veer <[email protected]>

* resolving conflicts -2

Signed-off-by: nishchay-veer <[email protected]>

* resolving conflicts in Panel.tsx

Signed-off-by: nishchay-veer <[email protected]>

* adding checkbox

Signed-off-by: nishchay-veer <[email protected]>

* fixing linters fail

Signed-off-by: nishchay-veer <[email protected]>

---------

Signed-off-by: nishchay-veer <[email protected]>
Signed-off-by: Saswata Mukherjee <[email protected]>
Signed-off-by: Nishchay Veer <[email protected]>
Co-authored-by: Saswata Mukherjee <[email protected]>

…6789)

No need to show the symbol if analyze is disabled. It looks weird. Let's
not do that.

Signed-off-by: Giedrius Statkevičius <[email protected]>

Two of the same names are used in e2e environment names. Fix this name
clash.

Signed-off-by: Giedrius Statkevičius <[email protected]>

* set dialer timeout to 5s in NewRoundTripperFromConfig

Signed-off-by: Walther Lee <[email protected]>

* add dialer_timeout field to HTTP TransportConfig

Signed-off-by: Walther Lee <[email protected]>

---------

Signed-off-by: Walther Lee <[email protected]>
Co-authored-by: Walther Lee <[email protected]>

Running tests with -race shows that there is a race between
bapi.blocks() and bapi.SetLoaded/SetGlobal() because the latter is
called continuously and asynchronously in a different thread. blocks()
is called through the HTTP API. Since block info is immutable, it is
enough to add a lock here to fix this problem.

Signed-off-by: Giedrius Statkevičius <[email protected]>

…hanos-io#6787)

* initialize new query stats struct at each goroutine

Signed-off-by: Ben Ye <[email protected]>

* remove comment

Signed-off-by: Ben Ye <[email protected]>

* address feedback

Signed-off-by: Ben Ye <[email protected]>

* fix lint

Signed-off-by: Ben Ye <[email protected]>

---------

Signed-off-by: Ben Ye <[email protected]>

…ies metric (thanos-io#6792)

Signed-off-by: Ben Ye <[email protected]>

Fix a race where GetPrometheusEngine or GetThanosEngine is called twice
at the same time from multiple HTTP requests. This fixes the race:

```
10:29:50 querier-query: ==================
10:29:50 querier-query: WARNING: DATA RACE
10:29:50 querier-query: Write at 0x00c0005fa0f8 by goroutine 285:
10:29:50 querier-query: github.com/thanos-io/thanos/pkg/api/query.(*QueryEngineFactory).GetPrometheusEngine()
10:29:50 querier-query: /go/src/github.com/thanos-io/thanos/pkg/api/query/v1.go:105 +0x1f9
10:29:50 querier-query: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).parseEngineParam()
10:29:50 querier-query: /go/src/github.com/thanos-io/thanos/pkg/api/query/v1.go:325 +0x109
10:29:50 querier-query: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).query()
10:29:50 querier-query: /go/src/github.com/thanos-io/thanos/pkg/api/query/v1.go:626 +0x605
10:29:50 querier-query: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).query-fm()
...
10:29:50 querier-query: Previous read at 0x00c0005fa0f8 by goroutine 287:
10:29:50 querier-query: github.com/thanos-io/thanos/pkg/api/query.(*QueryEngineFactory).GetPrometheusEngine()
10:29:50 querier-query: /go/src/github.com/thanos-io/thanos/pkg/api/query/v1.go:101 +0x13d
10:29:50 querier-query: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).parseEngineParam()
10:29:50 querier-query: /go/src/github.com/thanos-io/thanos/pkg/api/query/v1.go:325 +0x109
10:29:50 querier-query: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).query()
10:29:50 querier-query: /go/src/github.com/thanos-io/thanos/pkg/api/query/v1.go:626 +0x605
10:29:50 querier-query: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).query-fm()
...
```

Signed-off-by: Giedrius Statkevičius <[email protected]>

Each tracing.StartSpan() writes a value into the given context so
there's a race if we keep reusing the same context. Fix this by starting
a new span in each goroutine. This also makes logical sense. Fixes the
following race:

```
15:21:13 querier-1: WARNING: DATA RACE
15:21:13 querier-1: Read at 0x00c0009c5050 by goroutine 328:
15:21:13 querier-1: context.(*valueCtx).Value()
15:21:13 querier-1: /usr/local/go/src/context/context.go:751 +0x76
15:21:13 querier-1: github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/tracing.newClientSpanFromContext()
15:21:13 querier-1: /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/[email protected]/interceptors/tracing/client.go:87 +0x241
15:21:13 querier-1: github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/tracing.(*opentracingClientReportable).ClientReporter()
15:21:13 querier-1: /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/[email protected]/interceptors/tracing/client.go:51 +0x195
15:21:13 querier-1: github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/tracing.UnaryClientInterceptor.UnaryClientInterceptor.func1()
15:21:13 querier-1: /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/[email protected]/interceptors/client.go:19 +0x1a9
15:21:13 querier-1: github.com/thanos-io/thanos/pkg/extgrpc.StoreClientGRPCOpts.ChainUnaryClient.func4.1.1()
15:21:13 querier-1: /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/[email protected]/chain.go:74 +0x10a
15:21:13 querier-1: github.com/thanos-io/thanos/pkg/extgrpc.StoreClientGRPCOpts.(*ClientMetrics).UnaryClientInterceptor.func3()
15:21:13 querier-1: /go/pkg/mod/github.com/grpc-ecosystem/[email protected]/client_metrics.go:112 +0x126
15:21:13 querier-1: github.com/thanos-io/thanos/pkg/extgrpc.StoreClientGRPCOpts.ChainUnaryClient.func4.1.1()
15:21:13 querier-1: /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/[email protected]/chain.go:74 +0x10a
15:21:13 querier-1: github.com/thanos-io/thanos/pkg/extgrpc.StoreClientGRPCOpts.ChainUnaryClient.func4()
15:21:13 querier-1: /go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/[email protected]/chain.go:83 +0x17b
15:21:13 querier-1: google.golang.org/grpc.(*ClientConn).Invoke()
15:21:13 querier-1: /go/pkg/mod/google.golang.org/[email protected]/call.go:35 +0x25d
15:21:13 querier-1: github.com/thanos-io/thanos/pkg/store/storepb.(*storeClient).LabelValues()
15:21:13 querier-1: /go/src/github.com/thanos-io/thanos/pkg/store/storepb/rpc.pb.go:1034 +0xe5
15:21:13 querier-1: github.com/thanos-io/thanos/pkg/query.(*endpointRef).LabelValues()
15:21:13 querier-1: <autogenerated>:1 +0xa1                                                                                                                                        15:21:13 querier-1: github.com/thanos-io/thanos/pkg/store.(*ProxyStore).LabelValues.func1()
15:21:13 querier-1: /go/src/github.com/thanos-io/thanos/pkg/store/proxy.go:586 +0x323
15:21:13 querier-1: golang.org/x/sync/errgroup.(*Group).Go.func1()
15:21:13 querier-1: /go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x76
15:21:13 querier-1: Previous write at 0x00c0009c5050 by goroutine 325:
15:21:13 querier-1: context.WithValue()
15:21:13 querier-1: /usr/local/go/src/context/context.go:718 +0xce
15:21:13 querier-1: github.com/opentracing/opentracing-go.ContextWithSpan()
15:21:13 querier-1: /go/pkg/mod/github.com/opentracing/[email protected]/gocontext.go:17 +0xec
15:21:13 querier-1: github.com/thanos-io/thanos/pkg/tracing.StartSpan()
15:21:13 querier-1: /go/src/github.com/thanos-io/thanos/pkg/tracing/tracing.go:73 +0x238
15:21:13 querier-1: github.com/thanos-io/thanos/pkg/store.(*ProxyStore).LabelValues()
15:21:13 querier-1: /go/src/github.com/thanos-io/thanos/pkg/store/proxy.go:567 +0xb25
15:21:13 querier-1: github.com/thanos-io/thanos/pkg/query.(*querier).LabelValues()
15:21:13 querier-1: /go/src/github.com/thanos-io/thanos/pkg/query/querier.go:422 +0x3f5
15:21:13 querier-1: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).labelValues()
15:21:13 querier-1: /go/src/github.com/thanos-io/thanos/pkg/api/query/v1.go:1092 +0x17d1
15:21:13 querier-1: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).labelValues-fm()
15:21:13 querier-1: <autogenerated>:1 +0x45
15:21:13 querier-1: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).Register.GetInstr.func1.1()
```

Signed-off-by: Giedrius Statkevičius <[email protected]>

Return copy of the map because the compactor runs garbage collector
concurrently that deletes entries from the original map. Fixes race:

```
10:55:35 compact-working-dedup: ==================
10:55:35 compact-working-dedup: WARNING: DATA RACE
10:55:35 compact-working-dedup: Write at 0x00c001822150 by goroutine 220:
10:55:35 compact-working-dedup: runtime.mapdelete()
10:55:35 compact-working-dedup: /usr/local/go/src/runtime/map.go:696 +0x0
10:55:35 compact-working-dedup: github.com/thanos-io/thanos/pkg/compact.(*Syncer).GarbageCollect()
10:55:35 compact-working-dedup: /go/src/github.com/thanos-io/thanos/pkg/compact/compact.go:201 +0x324
10:55:35 compact-working-dedup: github.com/thanos-io/thanos/pkg/compact.(*BucketCompactor).Compact()
10:55:35 compact-working-dedup: /go/src/github.com/thanos-io/thanos/pkg/compact/compact.go:1422 +0x60f
10:55:35 compact-working-dedup: main.runCompact.func7()
10:55:35 compact-working-dedup: /go/src/github.com/thanos-io/thanos/cmd/thanos/compact.go:426 +0xfa
10:55:35 compact-working-dedup: main.runCompact.func8.1()
10:55:35 compact-working-dedup: /go/src/github.com/thanos-io/thanos/cmd/thanos/compact.go:481 +0x69
10:55:35 compact-working-dedup: github.com/thanos-io/thanos/pkg/runutil.Repeat()
10:55:35 compact-working-dedup: /go/src/github.com/thanos-io/thanos/pkg/runutil/runutil.go:74 +0xc3
10:55:35 compact-working-dedup: main.runCompact.func8()
10:55:35 compact-working-dedup: /go/src/github.com/thanos-io/thanos/cmd/thanos/compact.go:480 +0x224
10:55:35 compact-working-dedup: github.com/oklog/run.(*Group).Run.func1()
10:55:35 compact-working-dedup: /go/pkg/mod/github.com/oklog/[email protected]/group.go:38 +0x39
10:55:35 compact-working-dedup: github.com/oklog/run.(*Group).Run.func2()
10:55:35 compact-working-dedup: /go/pkg/mod/github.com/oklog/[email protected]/group.go:39 +0x4f
10:55:35 compact-working-dedup: Previous read at 0x00c001822150 by goroutine 223:
10:55:35 compact-working-dedup: runtime.mapiternext()
10:55:35 compact-working-dedup: /usr/local/go/src/runtime/map.go:867 +0x0
10:55:35 compact-working-dedup: github.com/thanos-io/thanos/pkg/compact.(*DefaultGrouper).Groups()
10:55:35 compact-working-dedup: /go/src/github.com/thanos-io/thanos/pkg/compact/compact.go:289 +0xfd
10:55:35 compact-working-dedup: main.runCompact.func16.1()
10:55:35 compact-working-dedup: /go/src/github.com/thanos-io/thanos/cmd/thanos/compact.go:626 +0x4ae
10:55:35 compact-working-dedup: github.com/thanos-io/thanos/pkg/runutil.Repeat()
10:55:35 compact-working-dedup: /go/src/github.com/thanos-io/thanos/pkg/runutil/runutil.go:74 +0xc3
10:55:35 compact-working-dedup: main.runCompact.func16()
10:55:35 compact-working-dedup: /go/src/github.com/thanos-io/thanos/cmd/thanos/compact.go:591 +0x3f9
10:55:35 compact-working-dedup: github.com/oklog/run.(*Group).Run.func1()
10:55:35 compact-working-dedup: /go/pkg/mod/github.com/oklog/[email protected]/group.go:38 +0x39
10:55:35 compact-working-dedup: github.com/oklog/run.(*Group).Run.func2()
10:55:35 compact-working-dedup: /go/pkg/mod/github.com/oklog/[email protected]/group.go:39 +0x4f
10:55:35 compact-working-dedup: Goroutine 220 (running) created at:
10:55:35 compact-working-dedup: github.com/oklog/run.(*Group).Run()
10:55:35 compact-working-dedup: /go/pkg/mod/github.com/oklog/[email protected]/group.go:37 +0xad
10:55:35 compact-working-dedup: main.main()
10:55:35 compact-working-dedup: /go/src/github.com/thanos-io/thanos/cmd/thanos/main.go:159 +0x2964
```

Signed-off-by: Giedrius Statkevičius <[email protected]>

)

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.14.0 to 0.17.0.
- [Commits](golang/net@v0.14.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Signed-off-by: GitHub <[email protected]>
Co-authored-by: fpetkovski <[email protected]>

Signed-off-by: Thibault Mange <[email protected]>

Signed-off-by: Ben Ye <[email protected]>

* fix matchersToPostingGroups vals variable shadow bug

Signed-off-by: Ben Ye <[email protected]>

* update changelog

Signed-off-by: Ben Ye <[email protected]>

---------

Signed-off-by: Ben Ye <[email protected]>

…ls (thanos-io#6816)

External Labels should also be tested for matches against the matchers.

Signed-off-by: Michael Hoffmann <[email protected]>

Signed-off-by: Ben Ye <[email protected]>

Signed-off-by: Walther Lee <[email protected]>

* Build with Go 1.21 (thanos-io#6615)

* Build with Go 1.21



* Update tools



---------



* update go alpine image to 3.18 (thanos-io#6750)



* build(deps): bump golang.org/x/net from 0.14.0 to 0.17.0 (thanos-io#6805)

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.14.0 to 0.17.0.
- [Commits](golang/net@v0.14.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...




* Updates busybox SHA (thanos-io#6808)




* Fix matchersToPostingGroups vals variable shadow bug (thanos-io#6817)

* fix matchersToPostingGroups vals variable shadow bug



* update changelog



---------



* fix head series limiter trigger (thanos-io#6802)



* Store: fix prometheus store label values for matches on external labels (thanos-io#6816)

External Labels should also be tested for matches against the matchers.



* Cut patch release v0.32.5



* Revert "Fix matchersToPostingGroups vals variable shadow bug (thanos-io#6817)"

This reverts commit 4ed9bb0.



---------

Signed-off-by: Saswata Mukherjee <[email protected]>
Signed-off-by: Coleen Iona Quadros <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: GitHub <[email protected]>
Signed-off-by: Ben Ye <[email protected]>
Signed-off-by: Thibault Mange <[email protected]>
Signed-off-by: Michael Hoffmann <[email protected]>
Co-authored-by: Coleen Iona Quadros <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: fpetkovski <[email protected]>
Co-authored-by: Ben Ye <[email protected]>
Co-authored-by: Thibault Mange <[email protected]>
Co-authored-by: Michael Hoffmann <[email protected]>

Bring thanos-io/promql-engine#320 into Thanos.
Fixes thanos-io/promql-engine#312.

Signed-off-by: Giedrius Statkevičius <[email protected]>

* receive/handler: fix label names/values race

There is a label name/value race in the current loop because
`labelpb.ReAllocZLabelsStrings(&t.Labels, r.opts.Intern)` might be
called which overwrites the original labels. At the same time, we might
also be forwarding the same request through gRPC to other Receive nodes.

Fixes the following race:

<details>
<summary>Trace of the race</summary>

10:53:51 receive-1: WARNING: DATA RACE
10:53:51 receive-1: Read at 0x00c001097b90 by goroutine 361:
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/store/labelpb.(*ZLabel).Size()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/store/labelpb/label.go:273 +0x35
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/store/storepb/prompb.(*TimeSeries).MarshalToSizedBuffer()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/store/storepb/prompb/types.pb.go:1499 +0x7c4
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/store/storepb.(*WriteRequest).MarshalToSizedBuffer()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/store/storepb/rpc.pb.go:1318 +0x409
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/store/storepb.(*WriteRequest).Marshal()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/store/storepb/rpc.pb.go:1286 +0x64
10:53:51 receive-1: google.golang.org/protobuf/internal/impl.legacyMarshal()
10:53:51 receive-1: /go/pkg/mod/google.golang.org/[email protected]/internal/impl/legacy_message.go:402 +0xb1
10:53:51 receive-1: google.golang.org/protobuf/proto.MarshalOptions.marshal()
10:53:51 receive-1: /go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:166 +0x3a2
10:53:51 receive-1: google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend()
10:53:51 receive-1: /go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:125 +0x96
10:53:51 receive-1: github.com/golang/protobuf/proto.marshalAppend()
10:53:51 receive-1: /go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:40 +0xce
10:53:51 receive-1: github.com/golang/protobuf/proto.Marshal()
10:53:51 receive-1: /go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:23 +0x65
10:53:51 receive-1: google.golang.org/grpc/encoding/proto.codec.Marshal()
10:53:51 receive-1: /go/pkg/mod/google.golang.org/[email protected]/encoding/proto/proto.go:45 +0x66
10:53:51 receive-1: google.golang.org/grpc/encoding/proto.(*codec).Marshal()
10:53:51 receive-1: <autogenerated>:1 +0x53
10:53:51 receive-1: google.golang.org/grpc.encode()
10:53:51 receive-1: /go/pkg/mod/google.golang.org/[email protected]/rpc_util.go:594 +0x64
10:53:51 receive-1: google.golang.org/grpc.prepareMsg()
10:53:51 receive-1: /go/pkg/mod/google.golang.org/[email protected]/stream.go:1610 +0x1a8
10:53:51 receive-1: google.golang.org/grpc.(*clientStream).SendMsg()
10:53:51 receive-1: /go/pkg/mod/google.golang.org/[email protected]/stream.go:791 +0x284
10:53:51 receive-1: google.golang.org/grpc.invoke()
10:53:51 receive-1: /go/pkg/mod/google.golang.org/[email protected]/call.go:70 +0xf2

...
10:53:51 receive-1: Previous write at 0x00c001097b90 by goroutine 357:
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/store/labelpb.ReAllocZLabelsStrings()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/store/labelpb/label.go:69 +0x25e
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/receive.(*Writer).Write()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/receive/writer.go:144 +0x13e4
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/receive.(*Handler).fanoutForward.func2.1()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/receive/handler.go:672 +0x153
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/tracing.DoInSpan()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/tracing/tracing.go:95 +0x125
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/receive.(*Handler).fanoutForward.func2()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/receive/handler.go:671 +0x1fd
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/receive.(*Handler).fanoutForward.func6()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/receive/handler.go:682 +0x61
10:53:51 receive-1: Goroutine 361 (running) created at:
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/receive.(*Handler).fanoutForward()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/receive/handler.go:688 +0x9c7
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/receive.(*Handler).forward()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/receive/handler.go:612 +0x53a
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/receive.(*Handler).handleRequest()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/receive/handler.go:417 +0xca8
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/receive.(*Handler).receiveHTTP()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/receive/handler.go:539 +0x1d89
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/receive.(*Handler).receiveHTTP-fm()
10:53:51 receive-1: <autogenerated>:1 +0x51
10:53:51 receive-1: net/http.HandlerFunc.ServeHTTP()
10:53:51 receive-1: /usr/local/go/src/net/http/server.go:2136 +0x47
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/receive.NewHandler.RequestID.func2()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/server/http/middleware/request_id.go:40 +0x191
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/receive.(*Handler).testReady-fm.(*Handler).testReady.func1()
10:53:51 receive-1: /go/src/github.com/thanos-io/thanos/pkg/receive/handler.go:263 +0x249
10:53:51 receive-1: net/http.HandlerFunc.ServeHTTP()
10:53:51 receive-1: /usr/local/go/src/net/http/server.go:2136 +0x47
10:53:51 receive-1: github.com/thanos-io/thanos/pkg/extprom/http.httpInstrumentationHandler.func1()

</details>

Signed-off-by: Giedrius Statkevičius <[email protected]>

* receive/handler: remove break

Signed-off-by: Giedrius Statkevičius <[email protected]>

---------

Signed-off-by: Giedrius Statkevičius <[email protected]>

Signed-off-by: Ben Ye <[email protected]>

…projects (thanos-io#6827)

* Expose fetcher and syncer metrics to be provided by depending projects.

Signed-off-by: Alex Le <[email protected]>

* Updated CHANGELOG

Signed-off-by: Alex Le <[email protected]>

* Remove CHANGELOG change

Signed-off-by: Alex Le <[email protected]>

---------

Signed-off-by: Alex Le <[email protected]>

We are re-reading the limits configuration periodically and also reading
it at the same time hence we need a lock around it. Thus, let's make
that struct member private and add a getter that returns the limiter
under a mutex lock.

Fixes:

```
17:14:45 receive-i3: WARNING: DATA RACE
17:14:45 receive-i3: Read at 0x00c00090aec0 by goroutine 131:
17:14:45 receive-i3: github.com/thanos-io/thanos/pkg/receive.(*headSeriesLimit).QueryMetaMonitoring()
17:14:45 receive-i3: /go/src/github.com/thanos-io/thanos/pkg/receive/head_series_limiter.go:109 +0x2fb
17:14:45 receive-i3: main.runReceive.func9.1()
17:14:45 receive-i3: /go/src/github.com/thanos-io/thanos/cmd/thanos/receive.go:402 +0x9b
17:14:45 receive-i3: github.com/thanos-io/thanos/pkg/runutil.Repeat()
17:14:45 receive-i3: /go/src/github.com/thanos-io/thanos/pkg/runutil/runutil.go:74 +0xc3
17:14:45 receive-i3: Previous write at 0x00c00090aec0 by goroutine 138:
17:14:45 receive-i3: github.com/thanos-io/thanos/pkg/receive.NewHeadSeriesLimit()
17:14:45 receive-i3: /go/src/github.com/thanos-io/thanos/pkg/receive/head_series_limiter.go:41 +0x316
17:14:45 receive-i3: github.com/thanos-io/thanos/pkg/receive.(*Limiter).loadConfig()
17:14:45 receive-i3: /go/src/github.com/thanos-io/thanos/pkg/receive/limiter.go:168 +0xd0d
17:14:45 receive-i3: github.com/thanos-io/thanos/pkg/receive.(*Limiter).StartConfigReloader.func1()
17:14:45 receive-i3: /go/src/github.com/thanos-io/thanos/pkg/receive/limiter.go:111 +0x207
17:14:45 receive-i3: github.com/thanos-io/thanos/pkg/extkingpin.(*pollingEngine).start.func1()
```

Signed-off-by: Giedrius Statkevičius <[email protected]>

Fix the following race:

```
12:36:39 querier-1: ==================
12:36:39 querier-1: WARNING: DATA RACE
12:36:39 querier-1: Read at 0x00c000159540 by goroutine 341:
12:36:39 querier-1: reflect.Value.String()
12:36:39 querier-1: /usr/local/go/src/reflect/value.go:2589 +0xd76
12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).writeAny()
12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:563 +0xd86
12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).writeStruct()
12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:325 +0x19db
12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).writeAny()
12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:606 +0xb2a
12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).writeStruct()
12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:453 +0xdd6
12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).writeAny()
12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:606 +0xb2a
12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).writeStruct()
12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:453 +0xdd6
12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).Marshal()
12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:896 +0x5c8
12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).Text()
12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:908 +0x92
12:36:39 querier-1: github.com/gogo/protobuf/proto.CompactTextString()
12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:930 +0x8e
12:36:39 querier-1: github.com/thanos-io/thanos/pkg/store/storepb.(*SeriesRequest).String()
12:36:39 querier-1: /go/src/github.com/thanos-io/thanos/pkg/store/storepb/rpc.pb.go:316 +0x7b
12:36:39 querier-1: github.com/thanos-io/thanos/pkg/store.(*ProxyStore).Series()
12:36:39 querier-1: /go/src/github.com/thanos-io/thanos/pkg/store/proxy.go:277 +0x8f
12:36:39 querier-1: github.com/thanos-io/thanos/pkg/query.(*querier).selectFn()

12:36:39 querier-1: Previous write at 0x00c000159540 by goroutine 339:
12:36:39 querier-1: golang.org/x/exp/slices.insertionSortOrdered[go.shape.string]()
12:36:39 querier-1: /go/pkg/mod/golang.org/x/[email protected]/slices/zsortordered.go:15 +0x357
12:36:39 querier-1: golang.org/x/exp/slices.pdqsortOrdered[go.shape.string]()
12:36:39 querier-1: /go/pkg/mod/golang.org/x/[email protected]/slices/zsortordered.go:75 +0x72f
12:36:39 querier-1: golang.org/x/exp/slices.Sort[go.shape.[]string,go.shape.string]()
12:36:39 querier-1: /go/pkg/mod/golang.org/x/[email protected]/slices/sort.go:19 +0x45a
12:36:39 querier-1: github.com/prometheus/prometheus/promql.(*evaluator).eval()
12:36:39 querier-1: /go/pkg/mod/github.com/prometheus/[email protected]/promql/engine.go:1352 +0x432
12:36:39 querier-1: github.com/prometheus/prometheus/promql.(*evaluator).Eval()
12:36:39 querier-1: /go/pkg/mod/github.com/prometheus/[email protected]/promql/engine.go:1052 +0x105
12:36:39 querier-1: github.com/prometheus/prometheus/promql.(*Engine).execEvalStmt()
12:36:39 querier-1: /go/pkg/mod/github.com/prometheus/[email protected]/promql/engine.go:708 +0xb15
12:36:39 querier-1: github.com/prometheus/prometheus/promql.(*Engine).exec()
12:36:39 querier-1: /go/pkg/mod/github.com/prometheus/[email protected]/promql/engine.go:646 +0x4c8
12:36:39 querier-1: github.com/prometheus/prometheus/promql.(*query).Exec()
12:36:39 querier-1: /go/pkg/mod/github.com/prometheus/[email protected]/promql/engine.go:235 +0x232
12:36:39 querier-1: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).query()
12:36:39 querier-1: /go/src/github.com/thanos-io/thanos/pkg/api/query/v1.go:681 +0xdfd
12:36:39 querier-1: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).query-fm()
12:36:39 querier-1: <autogenerated>:1 +0x45
12:36:39 querier-1: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).Register.GetInstr.func1.1()
12:36:39 querier-1: /go/src/github.com/thanos-io/thanos/pkg/api/api.go:212 +0x62
12:36:39 querier-1: net/http.HandlerFunc.ServeHTTP()
12:36:39 querier-1: /usr/local/go/src/net/http/server.go:2136 +0x47
12:36:39 querier-1: github.com/thanos-io/thanos/pkg/logging.(*HTTPServerMiddleware).HTTPMiddleware.func1()
```

Problem is that the same slice is sorted in the PromQL engine whereas
the same hints slice could still be used in other Select() calls where
String() is called and then it reads those hints.

Signed-off-by: Giedrius Statkevičius <[email protected]>

* Adding Grupo Olx as user

Signed-off-by: Nelson Almeida <[email protected]>

* Adding Grupo OLX logo

Signed-off-by: Nelson Almeida <[email protected]>

---------

Signed-off-by: Nelson Almeida <[email protected]>

* Receive: Add default tenant to HTTP metrics

Previously, if the tenant header was empty/not supplied, the exported
metrics would have an empty string as tenant. With this commit we
instead use the default tenant as can be configured with:
`--receive.default-tenant-id`.

Signed-off-by: Jacob Baungard Hansen <[email protected]>

* Query: Add tenant label to exported metrics

With this commit we now add the tenant label to relevant metrics
exported by the query component.

This includes the HTTP metrics handled by the InstrumentationMiddleware
and the query latency metrics.

Signed-off-by: Jacob Baungard Hansen <[email protected]>

---------

Signed-off-by: Jacob Baungard Hansen <[email protected]>

…#6819)

* Fix: Removes Deprecated ioutil

In Go, io/ioutil has been recently deprecated in favor of the drop in replacements "io" and "os". With the exception of the generated code in the file marked "DO NOT EDIT", this commit addresses those instances of ioutil with the respective function replacements.

Happy Hacktoberfest! Thank you for taking a moment to review my PR!

Signed-off-by: donuts-are-good <[email protected]>

* Adds Changelog entry

Completing the request for a changelog entry.

Signed-off-by: donuts-are-good <[email protected]>

* Removes Changelog Entry

This commit removes the ioutil changes in this PR, as they are not user-facing issues

Signed-off-by: donuts-are-good <[email protected]>

---------

Signed-off-by: donuts-are-good <[email protected]>

…nos-io#6809)

Signed-off-by: Ben Ye <[email protected]>

…nos-io#6836)

* Expose more overridable metrics from fetcher and default grouper

Signed-off-by: Alex Le <[email protected]>

* fix test

Signed-off-by: Alex Le <[email protected]>

* rename new functions

Signed-off-by: Alex Le <[email protected]>

---------

Signed-off-by: Alex Le <[email protected]>

* Make query frontend forward tenant info downstream

Signed-off-by: Douglas Camata <[email protected]>

* Add TODO for client certificate

Signed-off-by: Douglas Camata <[email protected]>

* Finish qfe tenant forward features

* Add tenant forwarding based on tenant cert.
* Fix bugs in the header management.

Signed-off-by: Douglas Camata <[email protected]>

* Test more scenarios

Signed-off-by: Douglas Camata <[email protected]>

* Goimports files

Signed-off-by: Douglas Camata <[email protected]>

* Rerun CI

Signed-off-by: Douglas Camata <[email protected]>

* Rerun goimports without any line breaks

Signed-off-by: Douglas Camata <[email protected]>

* Resort imports in groups (std, 3rd-party, local)

Signed-off-by: Douglas Camata <[email protected]>

* No need to forward a custom tenant header downstream

We should always and only forward the default (internal) one.

Signed-off-by: Douglas Camata <[email protected]>

* Make query frontend tenant cert config hidden

Signed-off-by: Douglas Camata <[email protected]>

* Error out when org ig and tenant header are provided at the same time

Signed-off-by: Douglas Camata <[email protected]>

* Fix typo

Signed-off-by: Douglas Camata <[email protected]>

* Update docs

Signed-off-by: Douglas Camata <[email protected]>

* Improve comments around forward and org id headers for tenancy

Signed-off-by: Douglas Camata <[email protected]>

* Add some test comments

Signed-off-by: Douglas Camata <[email protected]>

* Improve logic for detecting tenant and org id headers were provided

Signed-off-by: Douglas Camata <[email protected]>

* Fix order of check for org id headers

Signed-off-by: Douglas Camata <[email protected]>

* Add some TODO comments for the future

Signed-off-by: Douglas Camata <[email protected]>

* Rerun CI

Signed-off-by: Douglas Camata <[email protected]>

* Make default tenant flag in qfe match receive's

Signed-off-by: Douglas Camata <[email protected]>

* Remove unnecessary struct fields

Signed-off-by: Douglas Camata <[email protected]>

* Add instant query to tenant forward tests

Signed-off-by: Douglas Camata <[email protected]>

* Rerun build

Signed-off-by: Douglas Camata <[email protected]>

---------

Signed-off-by: Douglas Camata <[email protected]>

* expose fields for index cache

Signed-off-by: Ben Ye <[email protected]>

* fix lint

Signed-off-by: Ben Ye <[email protected]>

---------

Signed-off-by: Ben Ye <[email protected]>

Signed-off-by: lpreethvika-samsung <[email protected]>

Signed-off-by: Michael Hoffmann <[email protected]>

Signed-off-by: Michael Hoffmann <[email protected]>

* Create aligned blocks when flushing TSDBs

The receiver forces a flush on several occasions, such as when shutting down
or when pruning a tenant. The flush involves compacting the entire head,
which can sometimes be longer than the max block duration. Since each
receiver will flush at different times, these unaligned blocks trigger
vertical compaction with other align blocks in object storage and create
a new block which whose duration is outside of the pre-defined compaction levels.

This commit tries to resolve that issue by simulating a regular head compaction
to create a block that fits the max duration, followed by flushing the remainder
of the head which creates a separate, smaller sized block.

Signed-off-by: Filip Petkovski <[email protected]>

* Add more tests

Signed-off-by: Filip Petkovski <[email protected]>

* Fix test for one sample tsdb

Signed-off-by: Filip Petkovski <[email protected]>

---------

Signed-off-by: Filip Petkovski <[email protected]>

Fix the following race:
```

WARNING: DATA RACE
Read at 0x00c000614f98 by goroutine 521:
  github.com/thanos-io/thanos/pkg/queryfrontend.TestRoundTripQueryCacheWithShardingMiddleware.func1()
      /home/giedrius/dev/thanos/pkg/queryfrontend/roundtrip_test.go:612 +0x28e
  testing.tRunner()
      /usr/lib/go-1.21/src/testing/testing.go:1595 +0x238
  testing.(*T).Run.func1()
      /usr/lib/go-1.21/src/testing/testing.go:1648 +0x44

Previous write at 0x00c000614f98 by goroutine 532:
  github.com/thanos-io/thanos/pkg/queryfrontend.promqlResultsWithFailures.func1()
      /home/giedrius/dev/thanos/pkg/queryfrontend/roundtrip_test.go:917 +0x26f
  net/http.HandlerFunc.ServeHTTP()
      /usr/lib/go-1.21/src/net/http/server.go:2136 +0x47
  net/http.serverHandler.ServeHTTP()
      /usr/lib/go-1.21/src/net/http/server.go:2938 +0x2a1
  net/http.(*conn).serve()
      /usr/lib/go-1.21/src/net/http/server.go:2009 +0xc24
  net/http.(*Server).Serve.func3()
      /usr/lib/go-1.21/src/net/http/server.go:3086 +0x4f
```

Signed-off-by: Giedrius Statkevičius <[email protected]>

…s-io#6849)

* Move to `e2eobs.Observable` API to make profiling tests easier
and enable profiling on the interactive test.

Signed-off-by: Saswata Mukherjee <[email protected]>

* Address some e2e test flakes

Signed-off-by: Saswata Mukherjee <[email protected]>

---------

Signed-off-by: Saswata Mukherjee <[email protected]>

Signed-off-by: Daniel Deluiggi <[email protected]>

Fix race in the test by using an atomic value. The current variable is
shared between goroutines.

Signed-off-by: Giedrius Statkevičius <[email protected]>

Appending to the hints slice modifies the original request and we have
multiple of goroutines doing that at the same time. Hence, make a copy
of the hints and append only then.

Signed-off-by: Giedrius Statkevičius <[email protected]>

* Improve binary reader metrics

Signed-off-by: 🌲 Harry 🌊 John 🏔 <[email protected]>

* Update buckets

Signed-off-by: 🌲 Harry 🌊 John 🏔 <[email protected]>

* Update CHANGELOG

Signed-off-by: 🌲 Harry 🌊 John 🏔 <[email protected]>

---------

Signed-off-by: 🌲 Harry 🌊 John 🏔 <[email protected]>

Prepare for eventually building with stringlabels by converting
code that depends on slice labels to use the labels APIs.

Signed-off-by: Michael Hoffmann <[email protected]>

Signed-off-by: Michael Hoffmann <[email protected]>

Fix the following race:
```
Read at 0x00c000645ac0 by goroutine 43:
  github.com/prometheus/client_golang/prometheus.(*MetricVec).hashLabelValues()
      /home/giedrius/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/vec.go:279 +0x3a4
  github.com/prometheus/client_golang/prometheus.(*MetricVec).GetMetricWithLabelValues()
      /home/giedrius/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/vec.go:228 +0x109
  github.com/prometheus/client_golang/prometheus.(*GaugeVec).GetMetricWithLabelValues()
      /home/giedrius/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/gauge.go:203 +0x57
  github.com/prometheus/client_golang/prometheus.(*GaugeVec).WithLabelValues()
      /home/giedrius/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/gauge.go:236 +0xe4
  github.com/thanos-io/thanos/pkg/extprom.(*TxGaugeVec).WithLabelValues()
      /home/giedrius/dev/thanos/pkg/extprom/tx_gauge.go:105 +0xb8
  github.com/thanos-io/thanos/pkg/compact.(*GatherNoCompactionMarkFilter).Filter.func1()
      /home/giedrius/dev/thanos/pkg/compact/compact.go:1603 +0x78d
  golang.org/x/sync/errgroup.(*Group).Go.func1()
      /home/giedrius/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x76

Previous write at 0x00c000645ac0 by goroutine 41:
  github.com/prometheus/client_golang/prometheus.NewMetricVec()
      /home/giedrius/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/vec.go:66 +0xfe
  github.com/prometheus/client_golang/prometheus.v2.NewGaugeVec()
      /home/giedrius/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/gauge.go:168 +0x80
  github.com/prometheus/client_golang/prometheus.NewGaugeVec()
      /home/giedrius/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/gauge.go:153 +0x197
  github.com/prometheus/client_golang/prometheus/promauto.Factory.NewGaugeVec()
      /home/giedrius/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/promauto/auto.go:306 +0x5c
  github.com/thanos-io/thanos/pkg/extprom.NewTxGaugeVec.func1()
      /home/giedrius/dev/thanos/pkg/extprom/tx_gauge.go:31 +0x124
  github.com/thanos-io/thanos/pkg/extprom.(*TxGaugeVec).ResetTx()
      /home/giedrius/dev/thanos/pkg/extprom/tx_gauge.go:49 +0x72
  github.com/thanos-io/thanos/pkg/extprom.(*TxGaugeVec).WithLabelValues()
      /home/giedrius/dev/thanos/pkg/extprom/tx_gauge.go:103 +0x58
  github.com/thanos-io/thanos/pkg/compact.(*GatherNoCompactionMarkFilter).Filter.func1()
      /home/giedrius/dev/thanos/pkg/compact/compact.go:1603 +0x78d
  golang.org/x/sync/errgroup.(*Group).Go.func1()
      /home/giedrius/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x76
```

During `TestNoMarkFilterAtomic`, Filter() is called concurrently thus
leading to this situation.

Signed-off-by: Giedrius Statkevičius <[email protected]>

Signed-off-by: Sylvain Rabot <[email protected]>

…io#6861)

Signed-off-by: Michael Hoffmann <[email protected]>

* testutil/prometheus: fix start/stop race

Fix the following race between start and stop:

```
WARNING: DATA RACE
Read at 0x00c0001940a0 by goroutine 35:
  github.com/thanos-io/thanos/pkg/testutil/e2eutil.(*Prometheus).Stop()
      /home/giedrius/dev/thanos/pkg/testutil/e2eutil/prometheus.go:298 +0x94
  github.com/thanos-io/thanos/pkg/metadata.TestPrometheus_Metadata_e2e.func1()
      /home/giedrius/dev/thanos/pkg/metadata/prometheus_test.go:32 +0x33
  runtime.deferreturn()
      /usr/lib/go-1.21/src/runtime/panic.go:477 +0x30
  testing.tRunner()
      /usr/lib/go-1.21/src/testing/testing.go:1595 +0x238
  testing.(*T).Run.func1()
      /usr/lib/go-1.21/src/testing/testing.go:1648 +0x44

Previous write at 0x00c0001940a0 by goroutine 73:
  os/exec.(*Cmd).Start()
      /usr/lib/go-1.21/src/os/exec/exec.go:693 +0x977
  os/exec.(*Cmd).Run()
      /usr/lib/go-1.21/src/os/exec/exec.go:587 +0x26
  os/exec.(*Cmd).CombinedOutput()
      /usr/lib/go-1.21/src/os/exec/exec.go:1005 +0x1f6
  github.com/thanos-io/thanos/pkg/testutil/e2eutil.(*Prometheus).start.func1()
      /home/giedrius/dev/thanos/pkg/testutil/e2eutil/prometheus.go:226 +0x52
```

While fixing this I have also noticed some pretty bad code like
hardcoding durations in random time.Sleep. Also, some Start() calls were
without waiting for Prometheus to be up. Calling start but without
waiting for Prometheus to be up doesn't make much sense so I have taken
the liberty to refactor this place. Also, this should make tests faster
because we won't have random sleeps too.

Signed-off-by: Giedrius Statkevičius <[email protected]>

* e2eutil: pass prefix too

Signed-off-by: Giedrius Statkevičius <[email protected]>

---------

Signed-off-by: Giedrius Statkevičius <[email protected]>

…om (thanos-io#6863)

0.42.0 to 0.44.0

Signed-off-by: Daniel Mellado <[email protected]>

Signed-off-by: Alex Le <[email protected]>

…om (thanos-io#6864)

0.42.0 to 0.44.0

Signed-off-by: Daniel Mellado <[email protected]>

Signed-off-by: Ben Ye <[email protected]>

Changed fetcher and syncer Histogram metric to Observer

Got this during `TestLimiter_StartConfigReloader`:

```
Read at 0x00c0005123f0 by goroutine 8711:
  github.com/thanos-io/thanos/pkg/extkingpin.(*staticPathContent).Content()
      /home/giedrius/dev/thanos/pkg/extkingpin/path_content_reloader.go:90 +0x28
  github.com/thanos-io/thanos/pkg/receive.ParseLimitConfigContent()
      /home/giedrius/dev/thanos/pkg/receive/limiter.go:203 +0x3e
  github.com/thanos-io/thanos/pkg/receive.(*Limiter).loadConfig()
      /home/giedrius/dev/thanos/pkg/receive/limiter.go:143 +0x65
  github.com/thanos-io/thanos/pkg/receive.(*Limiter).StartConfigReloader.func1()
      /home/giedrius/dev/thanos/pkg/receive/limiter.go:119 +0x207
  github.com/thanos-io/thanos/pkg/extkingpin.(*pollingEngine).start.func1()
      /home/giedrius/dev/thanos/pkg/extkingpin/path_content_reloader.go:64 +0x6b8
  github.com/thanos-io/thanos/pkg/extkingpin.(*pollingEngine).start.func2()
      /home/giedrius/dev/thanos/pkg/extkingpin/path_content_reloader.go:74 +0x56

Previous write at 0x00c0005123f0 by goroutine 8710:
  github.com/thanos-io/thanos/pkg/extkingpin.(*staticPathContent).Rewrite()
      /home/giedrius/dev/thanos/pkg/extkingpin/path_content_reloader.go:111 +0x529
  github.com/thanos-io/thanos/pkg/receive.TestLimiter_StartConfigReloader()
      /home/giedrius/dev/thanos/pkg/receive/limiter_test.go:44 +0x5aa
  testing.tRunner()
      /usr/lib/go-1.21/src/testing/testing.go:1595 +0x238
  testing.(*T).Run.func1()
      /usr/lib/go-1.21/src/testing/testing.go:1648 +0x44
```

Fix it by protecting `t.content` with a mutex and by copying the content's
slice so that the caller wouldn't have access to the original slice.

Signed-off-by: Giedrius Statkevičius <[email protected]>

Let's enable the race detection for unit tests. Closes
thanos-io#4664.

Signed-off-by: Giedrius Statkevičius <[email protected]>

…hunks (thanos-io#6874)

Signed-off-by: Michael Hoffmann <[email protected]>

* *: mark as 0.33-rc.0

Signed-off-by: Michael Hoffmann <[email protected]>

* Bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp from (thanos-io#6863)

0.42.0 to 0.44.0

Signed-off-by: Daniel Mellado <[email protected]>
Signed-off-by: Michael Hoffmann <[email protected]>

---------

Signed-off-by: Michael Hoffmann <[email protected]>
Signed-off-by: Daniel Mellado <[email protected]>
Co-authored-by: Daniel Mellado <[email protected]>

Signed-off-by: Douglas Camata <[email protected]>

Spotted some randomness in the compactor test -
`thanos_blocks_meta_synced` includes cached blocks. Depending on luck,
Store might or might not see those blocks leading to inconsistent
results. Thus, only check the loaded metas count because that's what
matters.

Signed-off-by: Giedrius Statkevičius <[email protected]>

Signed-off-by: GitHub <[email protected]>
Co-authored-by: fpetkovski <[email protected]>

…o#6891)

* Bump objstore for Azure Workload Identity support

Signed-off-by: Rikhil Shah <[email protected]>

* make check-docs

Signed-off-by: Rikhil Shah <[email protected]>

* Add changelog entry

Signed-off-by: Rikhil Shah <[email protected]>

* Update Azure client docs

Signed-off-by: Rikhil Shah <[email protected]>

* make check-docs

Signed-off-by: Rikhil Shah <[email protected]>

* Move changelog entry to 0.33.0 release

Signed-off-by: Rikhil Shah <[email protected]>

* Move changelog entry

Signed-off-by: Rikhil Shah <[email protected]>

---------

Signed-off-by: Rikhil Shah <[email protected]>

Currently, the shipper metadata file is always called `thanos.shipper.json`
in the Prometheus data directory. This precludes running multiple sidecars
that upload to different object stores, as they will overwrite each other's
metadata file.

This commit allows the metadata file name to be customized via a flag. The
default is unchanged, but it can be overridden with the `--shipper.meta-file-name`
flag.

As part of this, we update the signatures of `WriteMetaFile` and `ReadMetaFile`
to take the full path of the metadata file, rather than just the directory,
and updates the tests that go along with this.

Signed-off-by: sinkingpoint <[email protected]>

…om-shipper-paths

sidecar: Allow customizing the shipper metadata file name

Looks like Banzai Cloud has migrated their site, and this article has
been lost.

Signed-off-by: Jacob Baungard Hansen <[email protected]>

* tutorial to integrate thanos with cilium

Signed-off-by: Coleen Iona Quadros <[email protected]>

* remove new line

Signed-off-by: Coleen Iona Quadros <[email protected]>

* add link in quick tutorial

Signed-off-by: Coleen Iona Quadros <[email protected]>

---------

Signed-off-by: Coleen Iona Quadros <[email protected]>

* UI: Use native go:embed instead of go-bindata

With this commit we migrate from the no longer actively maintained
go-bindata project, to go:embed, for embedding the UI app into the go
binary.

Signed-off-by: Jacob Baungard Hansen <[email protected]>

* UI: Remove static react app from .dockerignore

Otherwise we cannot build in e2e tests for example.

Signed-off-by: Jacob Baungard Hansen <[email protected]>

---------

Signed-off-by: Jacob Baungard Hansen <[email protected]>

Signed-off-by: Michael Hoffmann <[email protected]>

We have a NoDownsampleMarkFilter that we were not using before in the
compactor for some reason. Hook it into the filters chain if
downsampling is enabled and then trim matching ULIDs from the
downsampling process. Add a test to cover this scenario.

Fixes thanos-io#6179.

Signed-off-by: Giedrius Statkevičius <[email protected]>

I think the original values were simply too small there - if the runner
is slow then it can see smaller metrics values. If the runner works well
then the values can go to these bigger values and then the test fails.

Signed-off-by: Giedrius Statkevičius <[email protected]>

expose struct



fix lint

Signed-off-by: Ben Ye <[email protected]>

Signed-off-by: MeenuyD <[email protected]>

I have noticed that downloading tool deps takes ~5 min in CI each time.
We should be able to also cache those tool deps.

Signed-off-by: Giedrius Statkevičius <[email protected]>

Signed-off-by: Michael Hoffmann <[email protected]>

* blog: Introducing ThanosCon in Paris KubeCon EU 2024

Signed-off-by: Saswata Mukherjee <[email protected]>

* Add image

Signed-off-by: Saswata Mukherjee <[email protected]>

* make docs

Signed-off-by: Saswata Mukherjee <[email protected]>

* Apply suggestions

Co-authored-by: Matej Gera <[email protected]>
Co-authored-by: Bartlomiej Plotka <[email protected]>
Signed-off-by: Saswata Mukherjee <[email protected]>

* Apply suggestions

Co-authored-by: Bartlomiej Plotka <[email protected]>
Signed-off-by: Saswata Mukherjee <[email protected]>

* Add link to slack channel

Signed-off-by: Saswata Mukherjee <[email protected]>

---------

Signed-off-by: Saswata Mukherjee <[email protected]>
Co-authored-by: Matej Gera <[email protected]>
Co-authored-by: Bartlomiej Plotka <[email protected]>

Signed-off-by: Bartlomiej Plotka <[email protected]>

* Added info about ThanosCon to website and Readme.

Signed-off-by: bwplotka <[email protected]>

* Update website/layouts/index.html

Co-authored-by: Saswata Mukherjee <[email protected]>
Signed-off-by: Bartlomiej Plotka <[email protected]>

---------

Signed-off-by: bwplotka <[email protected]>
Signed-off-by: Bartlomiej Plotka <[email protected]>
Co-authored-by: Saswata Mukherjee <[email protected]>

…o#6891)

* Bump objstore for Azure Workload Identity support

Signed-off-by: Rikhil Shah <[email protected]>

* make check-docs

Signed-off-by: Rikhil Shah <[email protected]>

* Add changelog entry

Signed-off-by: Rikhil Shah <[email protected]>

* Update Azure client docs

Signed-off-by: Rikhil Shah <[email protected]>

* make check-docs

Signed-off-by: Rikhil Shah <[email protected]>

* Move changelog entry to 0.33.0 release

Signed-off-by: Rikhil Shah <[email protected]>

* Move changelog entry

Signed-off-by: Rikhil Shah <[email protected]>

---------

Signed-off-by: Rikhil Shah <[email protected]>
Signed-off-by: Michael Hoffmann <[email protected]>

Signed-off-by: Ben Ye <[email protected]>

Signed-off-by: Martin Hauskrecht <[email protected]>

Update LabyrinthLabs logo

…-33-cherry-pick-objstore-bump

Objstore: Bump Objstore for Azure Workload Identity support (thanos-io#6891)

* Support reload using signal

Signed-off-by: Simon Pasquier <[email protected]>

* Add --reloader.method option to sidecar

This option allows to tell the sidecar to send a SIGHUP signal to the
monitored process to reload its configuration instead of the default
HTTP-based method.

Signed-off-by: Simon Pasquier <[email protected]>

* Update docs and CHANGELOG.md

Signed-off-by: Simon Pasquier <[email protected]>

---------

Signed-off-by: Simon Pasquier <[email protected]>

* support float histogram in store gateway

Signed-off-by: Ben Ye <[email protected]>

* fix lint

Signed-off-by: Ben Ye <[email protected]>

* fix e2e test

Signed-off-by: Ben Ye <[email protected]>

* update prompb

Signed-off-by: Ben Ye <[email protected]>

* add changelog

Signed-off-by: Ben Ye <[email protected]>

---------

Signed-off-by: Ben Ye <[email protected]>

Signed-off-by: Matteo Martellini <[email protected]>