Skip to content

Commit

Permalink
Merge release 0.18 (#3809)
Browse files Browse the repository at this point in the history
* CHANGELOG.md: release v0.18.0

Signed-off-by: Lucas Servén Marín <[email protected]>

* VERSION,tutorials: bump versions

Signed-off-by: Lucas Servén Marín <[email protected]>

* Revert "VERSION,tutorials: bump versions"

This reverts commit 5a27c70.
The "thanos:" prefix for the container images was accidentally removed.

Signed-off-by: Lucas Servén Marín <[email protected]>

* VERSION,tutorials: bump version for all images

Signed-off-by: Lucas Servén Marín <[email protected]>

* CHANGELOG: bump release date

Signed-off-by: Lucas Servén Marín <[email protected]>

* pkg/rules/proxy: fix hotlooping when receiving client errors

Currently, if we receive an error from the underlying client stream,
we continue with trying to receive additional data.
This causes a hotloop as we will receive the same error again.
This fixes it by returning in the error case and adds a unit test for the proxy logic.

Fixes #3717

Signed-off-by: Sergiusz Urbaniak <[email protected]>

* CHANGELOG.md: fix changelog to incorporate new fix (#3737)

This commit fixes the order of the changelog to properly reflect a
recent change.

Signed-off-by: Lucas Servén Marín <[email protected]>

* Fixed website and added new step for release process (tmp). (#3738)

Signed-off-by: Bartlomiej Plotka <[email protected]>

* CHANGELOG.md: update v0.18.0 release date

Signed-off-by: Lucas Servén Marín <[email protected]>

Co-authored-by: Sergiusz Urbaniak <[email protected]>
Co-authored-by: Bartlomiej Plotka <[email protected]>
  • Loading branch information
3 people authored Feb 17, 2021
1 parent 7c5afc6 commit e4aa01a
Show file tree
Hide file tree
Showing 21 changed files with 316 additions and 52 deletions.
32 changes: 19 additions & 13 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,33 +27,39 @@ We use _breaking :warning:_ to mark changes that are not backward compatible (re

- [#3705](https://github.com/thanos-io/thanos/pull/3705) Store: Fix race condition leading to failing queries or possibly incorrect query results.

## [v0.18.0](https://github.com/thanos-io/thanos/releases) - Release in progress
## [v0.18.0](https://github.com/thanos-io/thanos/releases/tag/v0.18.0) - 2021.01.27

### Added

- [#3579](https://github.com/thanos-io/thanos/pull/3579) Cache: Added inmemory cache for caching bucket.
- [#3469](https://github.com/thanos-io/thanos/pull/3469) StoreAPI: Added `hints` field to `LabelNamesRequest` and `LabelValuesRequest`. Hints in an opaque data structure that can be used to carry additional information from the store and its content is implementation specific.
- [#3380](https://github.com/thanos-io/thanos/pull/3380) Mixin: Add block deletion panels for compactor dashboards.
- [#3568](https://github.com/thanos-io/thanos/pull/3568) Store: Optimized inject label stage of index lookup.
- [#3566](https://github.com/thanos-io/thanos/pull/3566) StoreAPI: Support label matchers in labels API.
- [#3531](https://github.com/thanos-io/thanos/pull/3531) Store: Optimized common cases for time selecting smaller amount of series by avoiding looking up symbols.
- [#3469](https://github.com/thanos-io/thanos/pull/3469) StoreAPI: Added `hints` field to `LabelNamesRequest` and `LabelValuesRequest`. Hints are an opaque data structure that can be used to carry additional information from the store and its content is implementation-specific.
- [#3421](https://github.com/thanos-io/thanos/pull/3421) Tools: Added `thanos tools bucket rewrite` command allowing to delete series from given block.
- [#3509](https://github.com/thanos-io/thanos/pull/3509) Store: Added touch series limit
- [#3388](https://github.com/thanos-io/thanos/pull/3378) Tools: Bucket replicator now can specify block IDs to copy.
- [#3121](https://github.com/thanos-io/thanos/pull/3121) Receive: Added `--receive.hashrings` alternative to `receive.hashrings-file` flag (lower priority). Content of JSON file that contains the hashring configuration.
- [#3509](https://github.com/thanos-io/thanos/pull/3509) Store: Added a CLI flag to limit the number of series that are touched.
- [#3444](https://github.com/thanos-io/thanos/pull/3444) Query Frontend: Make POST request to downstream URL for labels and series API endpoints.
- [#3388](https://github.com/thanos-io/thanos/pull/3388) Tools: Bucket replicator now can specify block IDs to copy.
- [#3385](https://github.com/thanos-io/thanos/pull/3385) Tools: Bucket prints extra statistics for block index with debug log-level.
- [#3121](https://github.com/thanos-io/thanos/pull/3121) Receive: Added `--receive.hashrings` alternative to `receive.hashrings-file` flag (lower priority). The flag expects the literal hashring configuration in JSON format.

### Fixed

- [#3567](https://github.com/thanos-io/thanos/pull/3567) Mixin: Reintroduce `thanos_objstore_bucket_operation_failures_total` alert.
- [#3527](https://github.com/thanos-io/thanos/pull/3527) Query Frontend: Fix query_range behavior when start/end times are the same
- [#3560](https://github.com/thanos-io/thanos/pull/3560) query-frontend: Allow separate label cache
- [#3672](https://github.com/thanos-io/thanos/pull/3672) rule: prevent rule crash from no such host error when using `dnssrv+` or `dnssrvnoa+`.
- [#3760](https://github.com/thanos-io/thanos/pull/3760) Store: Fix panic caused by a race condition happening on concurrent index-header reader usage and unload, when `--store.enable-index-header-lazy-reader` is enabled.
- [#3759](https://github.com/thanos-io/thanos/pull/3759) Store: Fix panic caused by a race condition happening on concurrent index-header lazy load and unload, when `--store.enable-index-header-lazy-reader` is enabled.
- [#3560](https://github.com/thanos-io/thanos/pull/3560) Query Frontend: Allow separate label cache
- [#3672](https://github.com/thanos-io/thanos/pull/3672) Rule: Prevent crashing due to `no such host error` when using `dnssrv+` or `dnssrvnoa+`.
- [#3461](https://github.com/thanos-io/thanos/pull/3461) Compact, Shipper, Store: Fixed panic when no external labels are set in block metadata.

### Changed

- [#3496](https://github.com/thanos-io/thanos/pull/3496) s3: Respect SignatureV2 flag for all credential providers.
- [#3496](https://github.com/thanos-io/thanos/pull/3496) S3: Respect SignatureV2 flag for all credential providers.
- [#2732](https://github.com/thanos-io/thanos/pull/2732) Swift: Switched to a new library [ncw/swift](https://github.com/ncw/swift) providing large objects support.
By default, segments will be uploaded to the same container directory `segments/` if the file is bigger than `1GB`.
To change the defaults see [the docs](./docs/storage.md#openstack-swift).
- [#3626](https://github.com/thanos-io/thanos/pull/3626) Failed upload of `meta.json` file doesn't cause block cleanup anymore. This has a potential to generate corrupted blocks under specific conditions. Partial block is left in bucket for later cleanup.

- [#3626](https://github.com/thanos-io/thanos/pull/3626) Shipper: Failed upload of `meta.json` file doesn't cause block cleanup anymore. This has a potential to generate corrupted blocks under specific conditions. Partial block is left in bucket for later cleanup.

## [v0.17.2](https://github.com/thanos-io/thanos/releases/tag/v0.17.2) - 2020.12.07

Expand All @@ -66,7 +72,8 @@ We use _breaking :warning:_ to mark changes that are not backward compatible (re

### Fixed

- [#3480](https://github.com/thanos-io/thanos/pull/3480) Query-frontend: Fixed regression.
- [#3480](https://github.com/thanos-io/thanos/pull/3480) Query Frontend: Fixed regression.
- [#3734](https://github.com/thanos-io/thanos/pull/3734) pkg/rules/proxy: fix hotlooping when receiving client errors

### Changed

Expand Down Expand Up @@ -278,7 +285,6 @@ sse_config:
- [#2579](https://github.com/thanos-io/thanos/pull/2579) Store: Experimental caching bucket can now cache metadata as well. Config has changed from #2532.
- [#2526](https://github.com/thanos-io/thanos/pull/2526) Compact: In case there are no labels left after deduplication via `--deduplication.replica-label`, assign first `replica-label` with value `deduped`.
- [#2621](https://github.com/thanos-io/thanos/pull/2621) Receive: Added flag to configure forward request timeout. Receive write will complete request as soon as quorum of writes succeeds.
- [#3380](https://github.com/thanos-io/thanos/pull/3380) mixin: Add block deletion panels for compactor dashboards.

### Changed

Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.18.0-dev
0.18.0
7 changes: 6 additions & 1 deletion pkg/rules/proxy.go
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,9 @@ func (stream *rulesStream) receive(ctx context.Context) error {
}

if err != nil {
// An error happened in Recv(), hence the underlying stream is aborted
// as per https://github.com/grpc/grpc-go/blob/7f2581f910fc21497091c4109b56d310276fc943/stream.go#L117-L125.
// We must not continue receiving additional data from it and must return.
err = errors.Wrapf(err, "receiving rules from rules client %v", stream.client)

if stream.request.PartialResponseStrategy == storepb.PartialResponseStrategy_ABORT {
Expand All @@ -118,13 +121,15 @@ func (stream *rulesStream) receive(ctx context.Context) error {
return errors.Wrapf(err, "sending rules error to server %v", stream.server)
}

continue
// Return no error if response strategy is warning.
return nil
}

if w := rule.GetWarning(); w != "" {
if err := stream.server.Send(rulespb.NewWarningRulesResponse(errors.New(w))); err != nil {
return errors.Wrapf(err, "sending rules warning to server %v", stream.server)
}
// Client stream is not aborted, it is ok to receive additional data.
continue
}

Expand Down
253 changes: 253 additions & 0 deletions pkg/rules/proxy_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,253 @@
// Copyright (c) The Thanos Authors.
// Licensed under the Apache License 2.0.

package rules

import (
"context"
"io"
"os"
"reflect"
"testing"

"github.com/go-kit/kit/log"
"github.com/pkg/errors"
"github.com/thanos-io/thanos/pkg/rules/rulespb"
"github.com/thanos-io/thanos/pkg/store/storepb"
"google.golang.org/grpc"
)

type testRulesClient struct {
grpc.ClientStream
rulesErr, recvErr error
response *rulespb.RulesResponse
sentResponse bool
}

func (t *testRulesClient) String() string {
return "test"
}

func (t *testRulesClient) Recv() (*rulespb.RulesResponse, error) {
// A simulation of underlying grpc Recv behavior as per https://github.com/grpc/grpc-go/blob/7f2581f910fc21497091c4109b56d310276fc943/stream.go#L117-L125.
if t.recvErr != nil {
return nil, t.recvErr
}

if t.sentResponse {
return nil, io.EOF
}
t.sentResponse = true

return t.response, nil
}

func (t *testRulesClient) Rules(ctx context.Context, in *rulespb.RulesRequest, opts ...grpc.CallOption) (rulespb.Rules_RulesClient, error) {
return t, t.rulesErr
}

var _ rulespb.RulesClient = &testRulesClient{}

type testRulesServer struct {
grpc.ServerStream
sendErr error
response *rulespb.RulesResponse
}

func (t *testRulesServer) String() string {
return "test"
}

func (t *testRulesServer) Send(response *rulespb.RulesResponse) error {
if t.sendErr != nil {
return t.sendErr
}
t.response = response
return nil
}

func (t *testRulesServer) Context() context.Context {
return context.Background()
}

func TestProxy(t *testing.T) {
logger := log.NewLogfmtLogger(os.Stderr)

for _, tc := range []struct {
name string
request *rulespb.RulesRequest
client rulespb.RulesClient
server *testRulesServer
wantResponse *rulespb.RulesResponse
wantError error
}{
{
name: "rule group proxy success",
request: &rulespb.RulesRequest{
Type: rulespb.RulesRequest_ALL,
PartialResponseStrategy: storepb.PartialResponseStrategy_WARN,
},
client: &testRulesClient{
response: rulespb.NewRuleGroupRulesResponse(&rulespb.RuleGroup{
Name: "foo",
}),
recvErr: nil,
},
server: &testRulesServer{},
wantResponse: rulespb.NewRuleGroupRulesResponse(&rulespb.RuleGroup{
Name: "foo",
}),
},
{
name: "warning proxy success",
request: &rulespb.RulesRequest{
Type: rulespb.RulesRequest_ALL,
PartialResponseStrategy: storepb.PartialResponseStrategy_WARN,
},
client: &testRulesClient{
response: rulespb.NewWarningRulesResponse(errors.New("warning from client")),
recvErr: nil,
},
server: &testRulesServer{},
wantResponse: rulespb.NewWarningRulesResponse(errors.New("warning from client")),
},
{
name: "warn: retreiving rules client failed",
request: &rulespb.RulesRequest{
Type: rulespb.RulesRequest_ALL,
PartialResponseStrategy: storepb.PartialResponseStrategy_WARN,
},
client: &testRulesClient{
response: nil,
rulesErr: errors.New("retreiving rules failed"),
},
server: &testRulesServer{},
wantResponse: rulespb.NewWarningRulesResponse(errors.New("fetching rules from rules client test: retreiving rules failed")),
},
{
name: "warn: retreiving rules client failed, forward warning failed",
request: &rulespb.RulesRequest{
Type: rulespb.RulesRequest_ALL,
PartialResponseStrategy: storepb.PartialResponseStrategy_WARN,
},
client: &testRulesClient{
response: nil,
rulesErr: errors.New("retreiving rules failed"),
},
server: &testRulesServer{
sendErr: errors.New("forwarding warning response failed"),
},
wantError: errors.New("forwarding warning response failed"),
},
{
name: "abort: retreiving rules client failed",
request: &rulespb.RulesRequest{
Type: rulespb.RulesRequest_ALL,
PartialResponseStrategy: storepb.PartialResponseStrategy_ABORT,
},
client: &testRulesClient{
response: nil,
rulesErr: errors.New("retreiving rules failed"),
},
server: &testRulesServer{},
wantError: errors.New("fetching rules from rules client test: retreiving rules failed"),
},
{
name: "warn: receive failed",
request: &rulespb.RulesRequest{
Type: rulespb.RulesRequest_ALL,
PartialResponseStrategy: storepb.PartialResponseStrategy_WARN,
},
client: &testRulesClient{
response: nil,
recvErr: errors.New("503 from Prometheus"),
},
server: &testRulesServer{},
wantResponse: rulespb.NewWarningRulesResponse(errors.New("receiving rules from rules client test: 503 from Prometheus")),
},
{
name: "warn: receive failed, forward warning failed",
request: &rulespb.RulesRequest{
Type: rulespb.RulesRequest_ALL,
PartialResponseStrategy: storepb.PartialResponseStrategy_WARN,
},
client: &testRulesClient{
response: nil,
recvErr: errors.New("503 from Prometheus"),
},
server: &testRulesServer{
sendErr: errors.New("forwarding warning response failed"),
},
wantError: errors.New("sending rules error to server test: forwarding warning response failed"),
},
{
name: "abort: receive failed",
request: &rulespb.RulesRequest{
Type: rulespb.RulesRequest_ALL,
PartialResponseStrategy: storepb.PartialResponseStrategy_ABORT,
},
client: &testRulesClient{
response: nil,
recvErr: errors.New("503 from Prometheus"),
},
server: &testRulesServer{},
wantError: errors.New("receiving rules from rules client test: 503 from Prometheus"),
},
{
name: "send failed",
request: &rulespb.RulesRequest{
Type: rulespb.RulesRequest_ALL,
PartialResponseStrategy: storepb.PartialResponseStrategy_WARN,
},
client: &testRulesClient{
response: rulespb.NewRuleGroupRulesResponse(&rulespb.RuleGroup{
Name: "foo",
}),
recvErr: nil,
},
server: &testRulesServer{
sendErr: errors.New("sending message failed"),
},
wantError: errors.New("rpc error: code = Unknown desc = send rules response: sending message failed"),
},
{
name: "sending warning response failed",
request: &rulespb.RulesRequest{
Type: rulespb.RulesRequest_ALL,
PartialResponseStrategy: storepb.PartialResponseStrategy_WARN,
},
client: &testRulesClient{
response: rulespb.NewWarningRulesResponse(errors.New("warning from client")),
recvErr: nil,
},
server: &testRulesServer{
sendErr: errors.New("sending message failed"),
},
wantError: errors.New("sending rules warning to server test: sending message failed"),
},
} {
t.Run(tc.name, func(t *testing.T) {
p := NewProxy(logger, func() []rulespb.RulesClient {
return []rulespb.RulesClient{tc.client}
})

err := p.Rules(tc.request, tc.server)
gotErr := "<nil>"
if err != nil {
gotErr = err.Error()
}
wantErr := "<nil>"
if tc.wantError != nil {
wantErr = tc.wantError.Error()
}

if gotErr != wantErr {
t.Errorf("want error %q, got %q", wantErr, gotErr)
}

if !reflect.DeepEqual(tc.wantResponse, tc.server.response) {
t.Errorf("want response %v, got %v", tc.wantResponse, tc.server.response)
}
})
}
}
2 changes: 1 addition & 1 deletion tutorials/katacoda/thanos/1-globalview/courseBase.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/usr/bin/env bash

docker pull quay.io/prometheus/prometheus:v2.16.0
docker pull quay.io/thanos/thanos:v0.17.2
docker pull quay.io/thanos/thanos:v0.18.0
8 changes: 4 additions & 4 deletions tutorials/katacoda/thanos/1-globalview/step2.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ component and can be invoked in a single command.
Let's take a look at all the Thanos commands:

```
docker run --rm quay.io/thanos/thanos:v0.17.2 --help
docker run --rm quay.io/thanos/thanos:v0.18.0 --help
```{{execute}}
You should see multiple commands that solves different purposes.
Expand Down Expand Up @@ -53,7 +53,7 @@ docker run -d --net=host --rm \
-v $(pwd)/prometheus0_eu1.yml:/etc/prometheus/prometheus.yml \
--name prometheus-0-sidecar-eu1 \
-u root \
quay.io/thanos/thanos:v0.17.2 \
quay.io/thanos/thanos:v0.18.0 \
sidecar \
--http-address 0.0.0.0:19090 \
--grpc-address 0.0.0.0:19190 \
Expand All @@ -68,7 +68,7 @@ docker run -d --net=host --rm \
-v $(pwd)/prometheus0_us1.yml:/etc/prometheus/prometheus.yml \
--name prometheus-0-sidecar-us1 \
-u root \
quay.io/thanos/thanos:v0.17.2 \
quay.io/thanos/thanos:v0.18.0 \
sidecar \
--http-address 0.0.0.0:19091 \
--grpc-address 0.0.0.0:19191 \
Expand All @@ -81,7 +81,7 @@ docker run -d --net=host --rm \
-v $(pwd)/prometheus1_us1.yml:/etc/prometheus/prometheus.yml \
--name prometheus-1-sidecar-us1 \
-u root \
quay.io/thanos/thanos:v0.17.2 \
quay.io/thanos/thanos:v0.18.0 \
sidecar \
--http-address 0.0.0.0:19092 \
--grpc-address 0.0.0.0:19192 \
Expand Down
Loading

0 comments on commit e4aa01a

Please sign in to comment.