Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

query: fix hints race #6831

Merged
merged 1 commit into from
Oct 20, 2023
Merged

query: fix hints race #6831

merged 1 commit into from
Oct 20, 2023

Commits on Oct 20, 2023

  1. query: fix hints race

    Fix the following race:
    
    ```
    12:36:39 querier-1: ==================
    12:36:39 querier-1: WARNING: DATA RACE
    12:36:39 querier-1: Read at 0x00c000159540 by goroutine 341:
    12:36:39 querier-1: reflect.Value.String()
    12:36:39 querier-1: /usr/local/go/src/reflect/value.go:2589 +0xd76
    12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).writeAny()
    12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:563 +0xd86
    12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).writeStruct()
    12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:325 +0x19db
    12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).writeAny()
    12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:606 +0xb2a
    12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).writeStruct()
    12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:453 +0xdd6
    12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).writeAny()
    12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:606 +0xb2a
    12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).writeStruct()
    12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:453 +0xdd6
    12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).Marshal()
    12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:896 +0x5c8
    12:36:39 querier-1: github.com/gogo/protobuf/proto.(*TextMarshaler).Text()
    12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:908 +0x92
    12:36:39 querier-1: github.com/gogo/protobuf/proto.CompactTextString()
    12:36:39 querier-1: /go/pkg/mod/github.com/gogo/[email protected]/proto/text.go:930 +0x8e
    12:36:39 querier-1: github.com/thanos-io/thanos/pkg/store/storepb.(*SeriesRequest).String()
    12:36:39 querier-1: /go/src/github.com/thanos-io/thanos/pkg/store/storepb/rpc.pb.go:316 +0x7b
    12:36:39 querier-1: github.com/thanos-io/thanos/pkg/store.(*ProxyStore).Series()
    12:36:39 querier-1: /go/src/github.com/thanos-io/thanos/pkg/store/proxy.go:277 +0x8f
    12:36:39 querier-1: github.com/thanos-io/thanos/pkg/query.(*querier).selectFn()
    
    12:36:39 querier-1: Previous write at 0x00c000159540 by goroutine 339:
    12:36:39 querier-1: golang.org/x/exp/slices.insertionSortOrdered[go.shape.string]()
    12:36:39 querier-1: /go/pkg/mod/golang.org/x/[email protected]/slices/zsortordered.go:15 +0x357
    12:36:39 querier-1: golang.org/x/exp/slices.pdqsortOrdered[go.shape.string]()
    12:36:39 querier-1: /go/pkg/mod/golang.org/x/[email protected]/slices/zsortordered.go:75 +0x72f
    12:36:39 querier-1: golang.org/x/exp/slices.Sort[go.shape.[]string,go.shape.string]()
    12:36:39 querier-1: /go/pkg/mod/golang.org/x/[email protected]/slices/sort.go:19 +0x45a
    12:36:39 querier-1: github.com/prometheus/prometheus/promql.(*evaluator).eval()
    12:36:39 querier-1: /go/pkg/mod/github.com/prometheus/[email protected]/promql/engine.go:1352 +0x432
    12:36:39 querier-1: github.com/prometheus/prometheus/promql.(*evaluator).Eval()
    12:36:39 querier-1: /go/pkg/mod/github.com/prometheus/[email protected]/promql/engine.go:1052 +0x105
    12:36:39 querier-1: github.com/prometheus/prometheus/promql.(*Engine).execEvalStmt()
    12:36:39 querier-1: /go/pkg/mod/github.com/prometheus/[email protected]/promql/engine.go:708 +0xb15
    12:36:39 querier-1: github.com/prometheus/prometheus/promql.(*Engine).exec()
    12:36:39 querier-1: /go/pkg/mod/github.com/prometheus/[email protected]/promql/engine.go:646 +0x4c8
    12:36:39 querier-1: github.com/prometheus/prometheus/promql.(*query).Exec()
    12:36:39 querier-1: /go/pkg/mod/github.com/prometheus/[email protected]/promql/engine.go:235 +0x232
    12:36:39 querier-1: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).query()
    12:36:39 querier-1: /go/src/github.com/thanos-io/thanos/pkg/api/query/v1.go:681 +0xdfd
    12:36:39 querier-1: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).query-fm()
    12:36:39 querier-1: <autogenerated>:1 +0x45
    12:36:39 querier-1: github.com/thanos-io/thanos/pkg/api/query.(*QueryAPI).Register.GetInstr.func1.1()
    12:36:39 querier-1: /go/src/github.com/thanos-io/thanos/pkg/api/api.go:212 +0x62
    12:36:39 querier-1: net/http.HandlerFunc.ServeHTTP()
    12:36:39 querier-1: /usr/local/go/src/net/http/server.go:2136 +0x47
    12:36:39 querier-1: github.com/thanos-io/thanos/pkg/logging.(*HTTPServerMiddleware).HTTPMiddleware.func1()
    ```
    
    Problem is that the same slice is sorted in the PromQL engine whereas
    the same hints slice could still be used in other Select() calls where
    String() is called and then it reads those hints.
    
    Signed-off-by: Giedrius Statkevičius <[email protected]>
    GiedriusS committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    3cec018 View commit details
    Browse the repository at this point in the history