Merge branch 'main' into active-limit-cfg

Signed-off-by: Saswata Mukherjee <[email protected]>

.circleci/config.yml

@@ -35,7 +35,7 @@ jobs:
       - run:
           name: "Run unit tests."
           environment:
-            THANOS_TEST_OBJSTORE_SKIP: AZURE,COS,ALIYUNOSS,BOS
+            THANOS_TEST_OBJSTORE_SKIP: GCS,S3,AZURE,COS,ALIYUNOSS,BOS,OCI
             # Variables for Swift testing.
             OS_AUTH_URL: http://127.0.0.1:5000/v2.0
             OS_PASSWORD: s3cr3t
@@ -45,7 +45,7 @@ jobs:
           # taskset sets CPU affinity to 2 (current CPU limit).
           command: |
             if [ -z ${GCP_PROJECT} ]; then
-              export THANOS_TEST_OBJSTORE_SKIP=${THANOS_TEST_OBJSTORE_SKIP},GCS,S3
+              export THANOS_TEST_OBJSTORE_SKIP=${THANOS_TEST_OBJSTORE_SKIP}
             fi
             echo "Skipping tests for object storages: ${THANOS_TEST_OBJSTORE_SKIP}"
             taskset 2 make test

.github/workflows/codeql-analysis.yml

@@ -37,6 +37,11 @@ jobs:
     - name: Checkout repository
       uses: actions/checkout@v2
 
+    - name: Set up Go
+      uses: actions/setup-go@v2
+      with:
+        go-version: 1.18
+
     # Initializes the CodeQL tools for scanning.
     - name: Initialize CodeQL
       uses: github/codeql-action/init@v1

CHANGELOG.md

@@ -13,6 +13,7 @@ We use *breaking :warning:* to mark changes that are not backward compatible (re
 ### Fixed
 - [#5642](https://github.com/thanos-io/thanos/pull/5642) Receive: Log labels correctly in writer debug messages.
 - [#5655](https://github.com/thanos-io/thanos/pull/5655) Receive: Fix recreating already pruned tenants.
+- [#5702](https://github.com/thanos-io/thanos/pull/5702) Store: Upgrade minio-go/v7 to fix panic caused by leaked goroutines.
 
 ### Added
 * [#5654](https://github.com/thanos-io/thanos/pull/5654) Query: add `--grpc-compression` flag that controls the compression used in gRPC client. With the flag it is now possible to compress the traffic between Query and StoreAPI nodes - you get lower network usage in exchange for a bit higher CPU/RAM usage.
@@ -24,6 +25,7 @@ We use *breaking :warning:* to mark changes that are not backward compatible (re
 - [#5255](https://github.com/thanos-io/thanos/pull/5296) Query: Use k-way merging for the proxying logic. The proxying sub-system now uses much less resources (~25-80% less CPU usage, ~30-50% less RAM usage according to our benchmarks). Reduces query duration by a few percent on queries with lots of series.
 - [#5690](https://github.com/thanos-io/thanos/pull/5690) Compact: update `--debug.accept-malformed-index` flag to apply to downsampling. Previously the flag only applied to compaction, and fatal errors would still occur when downsampling was attempted.
 - [#5685](https://github.com/thanos-io/thanos/pull/5685) Receive: Make active/head series limiting configuration per tenant by adding it to new limiting config.
+- [#5707](https://github.com/thanos-io/thanos/pull/5707) Objstore: Update objstore to latest version which includes a refactored Azure Storage Account implementation with a new SDK.
 
 ### Removed
 

CONTRIBUTING.md

@@ -67,7 +67,7 @@ The following section explains various suggestions and procedures to note during
 ### Prerequisites
 
 * It is strongly recommended that you use Linux distributions systems or OSX for development.
-* Go 1.17.x or higher.
+* Go 1.18.x or higher.
 * For React UI, you will need a working NodeJS environment and the npm package manager to compile the Web UI assets.
 
 ### First Steps

MAINTAINERS.md

@@ -1,16 +1,17 @@
 # Core Maintainers of this repository
 
-| Name                  | Email                  | Slack                    | GitHub                                         | Company       |
-|-----------------------|------------------------|--------------------------|------------------------------------------------|---------------|
-| Bartłomiej Płotka     | [email protected]     | `@bwplotka`              | [@bwplotka](https://github.com/bwplotka)       | Red Hat       |
-| Frederic Branczyk     | [email protected]    | `@brancz`                | [@brancz](https://github.com/brancz)           | Polar Signals |
-| Giedrius Statkevičius | [email protected] | `@Giedrius Statkevičius` | [@GiedriusS](https://github.com/GiedriusS)     | Vinted        |
-| Kemal Akkoyun         | [email protected]     | `@kakkoyun`              | [@kakkoyun](https://github.com/kakkoyun)       | Polar Signals |
-| Lucas Servén Marín    | [email protected]      | `@squat`                 | [@squat](https://github.com/squat)             | Red Hat       |
-| Prem Saraswat         | [email protected]     | `@Prem Saraswat`         | [@onprem](https://github.com/onprem)           | Red Hat       |
-| Matthias Loibl        | [email protected] | `@metalmatze`            | [@metalmatze](https://github.com/metalmatze)   | Polar Signals |
-| Ben Ye                | [email protected]  | `@yeya24`                | [@yeya24](https://github.com/yeya24)           | ByteDance     |
-| Wiard van Rij         | [email protected]      | `@wiard van Rij`         | [@wiardvanrij](https://github.com/wiardvanrij) | Roku          |
+| Name                  | Email                  | Slack                    | GitHub                                         | Company             |
+|-----------------------|------------------------|--------------------------|------------------------------------------------|---------------------|
+| Bartłomiej Płotka     | [email protected]     | `@bwplotka`              | [@bwplotka](https://github.com/bwplotka)       | Red Hat             |
+| Frederic Branczyk     | [email protected]    | `@brancz`                | [@brancz](https://github.com/brancz)           | Polar Signals       |
+| Giedrius Statkevičius | [email protected] | `@Giedrius Statkevičius` | [@GiedriusS](https://github.com/GiedriusS)     | Vinted              |
+| Kemal Akkoyun         | [email protected]     | `@kakkoyun`              | [@kakkoyun](https://github.com/kakkoyun)       | Polar Signals       |
+| Lucas Servén Marín    | [email protected]      | `@squat`                 | [@squat](https://github.com/squat)             | Red Hat             |
+| Prem Saraswat         | [email protected]     | `@Prem Saraswat`         | [@onprem](https://github.com/onprem)           | Red Hat             |
+| Matthias Loibl        | [email protected] | `@metalmatze`            | [@metalmatze](https://github.com/metalmatze)   | Polar Signals       |
+| Ben Ye                | [email protected]  | `@yeya24`                | [@yeya24](https://github.com/yeya24)           | Amazon Web Services |
+| Wiard van Rij         | [email protected]      | `@wiard van Rij`         | [@wiardvanrij](https://github.com/wiardvanrij) | Roku                |
+| Matej Gera            | [email protected]    | `@Matej Gera`            | [@matej-g](https://github.com/matej-g)         | Red Hat             |
 
 We are bunch of people from different companies with various interests and skills. We are from different parts of the world: Germany, Italy, Lithuania, Poland, UK, India and China. We have something in common though: We all share the love for OpenSource, Go, Prometheus, :coffee: and Observability topics.
 
@@ -36,8 +37,7 @@ Full list of triage persons is displayed below:
 | Michael Dai     | `@jojohappy`       | [@jojohappy](https://github.com/jojohappy)   |         |
 | Xiang Dai       | `@daixiang0`       | [@daixiang0](https://github.com/daixiang0)   |         |
 | Jimmie Han      | `@hanjm`           | [@hanjm](https://github.com/hanjm)           | Tencent |
-| Matej Gera      | `@Matej Gera`      | [@matej-g](https://github.com/matej-g)       | Red Hat |
-| Filip Petkovski | `@Filip Petkovski` | [@fpetkovski](https://github.com/fpetkovski) |         |
+| Filip Petkovski | `@Filip Petkovski` | [@fpetkovski](https://github.com/fpetkovski) | Shopify |
 
 Please reach any of the maintainer on slack or email if you want to help as well.
 

cmd/thanos/main.go

@@ -10,6 +10,7 @@ import (
 	"os"
 	"os/signal"
 	"path/filepath"
+	"regexp"
 	"runtime"
 	"runtime/debug"
 	"syscall"
@@ -73,7 +74,9 @@ func main() {
 	metrics := prometheus.NewRegistry()
 	metrics.MustRegister(
 		version.NewCollector("thanos"),
-		collectors.NewGoCollector(),
+		collectors.NewGoCollector(
+			collectors.WithGoCollectorRuntimeMetrics(collectors.GoRuntimeMetricsRule{Matcher: regexp.MustCompile("/.*")}),
+		),
 		collectors.NewProcessCollector(collectors.ProcessCollectorOpts{}),
 	)
 

cmd/thanos/query.go

@@ -25,8 +25,8 @@ import (
 	"github.com/prometheus/prometheus/discovery/targetgroup"
 	"github.com/prometheus/prometheus/model/labels"
 	"github.com/prometheus/prometheus/promql"
-	"google.golang.org/grpc"
-
+	v1 "github.com/prometheus/prometheus/web/api/v1"
+	"github.com/thanos-community/promql-engine/engine"
 	apiv1 "github.com/thanos-io/thanos/pkg/api/query"
 	"github.com/thanos-io/thanos/pkg/compact/downsample"
 	"github.com/thanos-io/thanos/pkg/component"
@@ -54,6 +54,7 @@ import (
 	"github.com/thanos-io/thanos/pkg/targets"
 	"github.com/thanos-io/thanos/pkg/tls"
 	"github.com/thanos-io/thanos/pkg/ui"
+	"google.golang.org/grpc"
 )
 
 const (
@@ -62,6 +63,13 @@ const (
 	queryPushdown        = "query-pushdown"
 )
 
+type promqlEngineType string
+
+const (
+	promqlEnginePrometheus promqlEngineType = "prometheus"
+	promqlEngineThanos     promqlEngineType = "thanos"
+)
+
 // registerQuery registers a query command.
 func registerQuery(app *extkingpin.App) {
 	comp := component.Query
@@ -88,6 +96,8 @@ func registerQuery(app *extkingpin.App) {
 
 	queryTimeout := extkingpin.ModelDuration(cmd.Flag("query.timeout", "Maximum time to process query by query node.").
 		Default("2m"))
+	promqlEngine := cmd.Flag("query.promql-engine", "PromQL engine to use.").Default(string(promqlEnginePrometheus)).Hidden().
+		Enum(string(promqlEnginePrometheus), string(promqlEngineThanos))
 
 	maxConcurrentQueries := cmd.Flag("query.max-concurrent", "Maximum number of queries processed concurrently by query node.").
 		Default("20").Int()
@@ -295,6 +305,7 @@ func registerQuery(app *extkingpin.App) {
 			*alertQueryURL,
 			*grpcProxyStrategy,
 			component.Query,
+			promqlEngineType(*promqlEngine),
 		)
 	})
 }
@@ -366,6 +377,7 @@ func runQuery(
 	alertQueryURL string,
 	grpcProxyStrategy string,
 	comp component.Component,
+	promqlEngine promqlEngineType,
 ) error {
 	if alertQueryURL == "" {
 		lastColon := strings.LastIndex(httpBindAddr, ":")
@@ -487,20 +499,6 @@ func runQuery(
 			maxConcurrentSelects,
 			queryTimeout,
 		)
-
-		engineOpts = promql.EngineOpts{
-			Logger: logger,
-			Reg:    reg,
-			// TODO(bwplotka): Expose this as a flag: https://github.com/thanos-io/thanos/issues/703.
-			MaxSamples:    math.MaxInt32,
-			Timeout:       queryTimeout,
-			LookbackDelta: lookbackDelta,
-			NoStepSubqueryIntervalFn: func(int64) int64 {
-				return defaultEvaluationInterval.Milliseconds()
-			},
-			EnableNegativeOffset: true,
-			EnableAtModifier:     true,
-		}
 	)
 
 	// Periodically update the store set with the addresses we see in our cluster.
@@ -597,12 +595,36 @@ func runQuery(
 		prober.NewInstrumentation(comp, logger, extprom.WrapRegistererWithPrefix("thanos_", reg)),
 	)
 
+	engineOpts := promql.EngineOpts{
+		Logger: logger,
+		Reg:    reg,
+		// TODO(bwplotka): Expose this as a flag: https://github.com/thanos-io/thanos/issues/703.
+		MaxSamples:    math.MaxInt32,
+		Timeout:       queryTimeout,
+		LookbackDelta: lookbackDelta,
+		NoStepSubqueryIntervalFn: func(int64) int64 {
+			return defaultEvaluationInterval.Milliseconds()
+		},
+		EnableNegativeOffset: true,
+		EnableAtModifier:     true,
+	}
+
 	// An active query tracker will be added only if the user specifies a non-default path.
 	// Otherwise, the nil active query tracker from existing engine options will be used.
 	if activeQueryDir != "" {
 		engineOpts.ActiveQueryTracker = promql.NewActiveQueryTracker(activeQueryDir, maxConcurrentQueries, logger)
 	}
-	engine := promql.NewEngine(engineOpts)
+
+	var queryEngine v1.QueryEngine
+	switch promqlEngine {
+	case promqlEnginePrometheus:
+		queryEngine = promql.NewEngine(engineOpts)
+	case promqlEngineThanos:
+		queryEngine = engine.New(engine.Opts{EngineOpts: engineOpts})
+	default:
+		return errors.Errorf("unknown query.promql-engine type %v", promqlEngine)
+	}
+
 	lookbackDeltaCreator := LookbackDeltaFactory(engineOpts, dynamicLookbackDelta)
 
 	// Start query API + UI HTTP server.
@@ -633,7 +655,7 @@ func runQuery(
 		api := apiv1.NewQueryAPI(
 			logger,
 			endpoints.GetEndpointStatus,
-			engine,
+			queryEngine,
 			lookbackDeltaCreator,
 			queryableCreator,
 			// NOTE: Will share the same replica label as the query for now.
@@ -695,9 +717,10 @@ func runQuery(
 				if httpProbe.IsReady() {
 					mint, maxt := proxy.TimeRange()
 					return &infopb.StoreInfo{
-						MinTime:          mint,
-						MaxTime:          maxt,
-						SupportsSharding: true,
+						MinTime:           mint,
+						MaxTime:           maxt,
+						SupportsSharding:  true,
+						SendsSortedSeries: true,
 					}
 				}
 				return nil
@@ -709,7 +732,7 @@ func runQuery(
 			info.WithQueryAPIInfoFunc(),
 		)
 
-		grpcAPI := apiv1.NewGRPCAPI(time.Now, queryReplicaLabels, queryableCreator, engine, lookbackDeltaCreator, instantDefaultMaxSourceResolution)
+		grpcAPI := apiv1.NewGRPCAPI(time.Now, queryReplicaLabels, queryableCreator, queryEngine, lookbackDeltaCreator, instantDefaultMaxSourceResolution)
 		s := grpcserver.New(logger, reg, tracer, grpcLogOpts, tagOpts, comp, grpcProbe,
 			grpcserver.WithServer(apiv1.RegisterQueryServer(grpcAPI)),
 			grpcserver.WithServer(store.RegisterStoreServer(proxy)),

cmd/thanos/receive.go

@@ -316,9 +316,10 @@ func runReceive(
 				if httpProbe.IsReady() {
 					minTime, maxTime := mts.TimeRange()
 					return &infopb.StoreInfo{
-						MinTime:          minTime,
-						MaxTime:          maxTime,
-						SupportsSharding: true,
+						MinTime:           minTime,
+						MaxTime:           maxTime,
+						SupportsSharding:  true,
+						SendsSortedSeries: true,
 					}
 				}
 				return nil

cmd/thanos/sidecar.go

@@ -268,9 +268,10 @@ func runSidecar(
 				if httpProbe.IsReady() {
 					mint, maxt := promStore.Timestamps()
 					return &infopb.StoreInfo{
-						MinTime:          mint,
-						MaxTime:          maxt,
-						SupportsSharding: true,
+						MinTime:           mint,
+						MaxTime:           maxt,
+						SupportsSharding:  true,
+						SendsSortedSeries: true,
 					}
 				}
 				return nil

cmd/thanos/store.go

@@ -393,9 +393,10 @@ func runStore(
 			if httpProbe.IsReady() {
 				mint, maxt := bs.TimeRange()
 				return &infopb.StoreInfo{
-					MinTime:          mint,
-					MaxTime:          maxt,
-					SupportsSharding: true,
+					MinTime:           mint,
+					MaxTime:           maxt,
+					SupportsSharding:  true,
+					SendsSortedSeries: true,
 				}
 			}
 			return nil

docs/getting-started.md

@@ -40,7 +40,7 @@ Thanos can **not** be downloaded nor installed via the `go get` or `go install`
 
 Thanos uses the directive *replace*. The reason is to provide a way to unblock ourselves promptly while also being flexible in the packages that we (re)use. Support for `go install` is not likely at this point.
 
-If you want to build Thanos from source you would need a working installation of the Go 1.17+ [toolchain](https://github.com/golang/tools) (`GOPATH`, `PATH=${GOPATH}/bin:${PATH}`). Next one should make a clone of our repository:
+If you want to build Thanos from source you would need a working installation of the Go 1.18+ [toolchain](https://github.com/golang/tools) (`GOPATH`, `PATH=${GOPATH}/bin:${PATH}`). Next one should make a clone of our repository:
 
 ```
 git clone [email protected]:thanos-io/thanos.git