Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Web metric provider #318

Merged
merged 29 commits into from
Jan 16, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
296169f
Initial implementation for the WebMetric type of analysis.
tomsanbear Dec 3, 2019
eb42130
Wrote basic test, and fixed some functionality for an initial POC
tomsanbear Dec 3, 2019
56074a1
Clean up debug logging
tomsanbear Dec 3, 2019
94cd16b
Ran codegen
tomsanbear Dec 3, 2019
ff388e4
Made minor fixes suggested in by Rollouts team: WebMetric naming upda…
tomsanbear Dec 9, 2019
9eac5a3
Smarter logic for detecting type in the JSON response. Also parameter…
tomsanbear Dec 9, 2019
3fa6f71
Fix Infinite loop with PreviewReplicaCount set (#308)
dthomson25 Dec 3, 2019
084b1d7
Fix incorrect patch on getting started guide (#315)
dthomson25 Dec 3, 2019
1046115
Create one background analysis per revision (#309)
dthomson25 Dec 3, 2019
a24a733
Bluegreen: allow preview service/replica sets to be replaced and fix …
mathetake Dec 4, 2019
5feb382
Set StableRS hash to current if replicaset does not actually exist (#…
dthomson25 Dec 5, 2019
53ee15b
Update version to v0.6.1
dthomson25 Dec 6, 2019
81f59fe
Add Community Blogs and Presentations section (#322)
saradhis Dec 10, 2019
bdc1ec8
Fix a typo (#321)
bpaquet Dec 10, 2019
867a0f6
Fix link to deployment concepts in docs (#325)
petvaa01 Dec 12, 2019
c61a43b
fix: omitted revisionHistoryLimit was not defaulting to 10 (#330)
jessesuen Dec 16, 2019
f870efe
Fix panics with incorrectly configured rollouts (#328)
dthomson25 Dec 16, 2019
88e59d6
Add Nginx docs for traffic management (#326)
dthomson25 Dec 16, 2019
53b806e
Update version to v0.6.2
dthomson25 Dec 16, 2019
4496241
Fix error handling with HTTP client, change timeout field to timeoutS…
tomsanbear Dec 17, 2019
5dd2fce
Add simple non 2xx test.
tomsanbear Dec 17, 2019
e3321fe
Add test for headers, fix bug where header is unitialized, causing NPE.
tomsanbear Dec 18, 2019
0feb20b
Add a few more corner cases for HTTP responses.
tomsanbear Dec 18, 2019
81b91e1
golang-ci fix
tomsanbear Dec 18, 2019
eb30b36
First pass at a using the evaluation engine to enforce typing.
tomsanbear Jan 7, 2020
6bd5577
Increment numProviders when Web is present.
tomsanbear Jan 12, 2020
99c8430
Deal with merge issues.
tomsanbear Jan 12, 2020
79d8718
Fix leftover merge conflict.
tomsanbear Jan 15, 2020
e476ea7
Add tests to cover panic cases for the type conversions, and also add…
tomsanbear Jan 15, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/features/traffic-management/istio.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,4 +85,4 @@ Both of these issues adds more complexity to the users and Argo Rollouts develop

### Implement Istio support through the SMI

[SMI](https://smi-spec.io/) is the Service Mesh Interface, which serves as a standard interface for all common features of a service mesh. This feature is GitOps friendly, but native Istio has extra functionality that SMI does not currently provide. Granted, Argo Rollouts should integrate with the SMI independent of the native Istio integration.
[SMI](https://smi-spec.io/) is the Service Mesh Interface, which serves as a standard interface for all common features of a service mesh. This feature is GitOps friendly, but native Istio has extra functionality that SMI does not currently provide. Granted, Argo Rollouts should integrate with the SMI independent of the native Istio integration.
128 changes: 98 additions & 30 deletions manifests/crds/analysis-run-crd.yaml

Large diffs are not rendered by default.

128 changes: 98 additions & 30 deletions manifests/crds/analysis-template-crd.yaml

Large diffs are not rendered by default.

104 changes: 74 additions & 30 deletions manifests/crds/experiment-crd.yaml

Large diffs are not rendered by default.

110 changes: 78 additions & 32 deletions manifests/crds/rollout-crd.yaml

Large diffs are not rendered by default.

429 changes: 307 additions & 122 deletions manifests/install.yaml

Large diffs are not rendered by default.

429 changes: 307 additions & 122 deletions manifests/namespace-install.yaml

Large diffs are not rendered by default.

8 changes: 8 additions & 0 deletions metricproviders/metricproviders.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
"github.com/argoproj/argo-rollouts/metricproviders/wavefront"

"github.com/argoproj/argo-rollouts/metricproviders/kayenta"
"github.com/argoproj/argo-rollouts/metricproviders/webmetric"

log "github.com/sirupsen/logrus"
"k8s.io/client-go/kubernetes"
Expand Down Expand Up @@ -52,6 +53,13 @@ func (f *ProviderFactory) NewProvider(logCtx log.Entry, metric v1alpha1.Metric)
} else if metric.Provider.Kayenta != nil {
c := kayenta.NewHttpClient()
return kayenta.NewKayentaProvider(logCtx, c), nil
} else if metric.Provider.Web != nil {
c := webmetric.NewWebMetricHttpClient(metric)
p, err := webmetric.NewWebMetricJsonParser(metric)
if err != nil {
return nil, err
}
return webmetric.NewWebMetricProvider(logCtx, c, p), nil
} else if metric.Provider.Wavefront != nil {
client, err := wavefront.NewWavefrontAPI(metric, f.KubeClient)
if err != nil {
Expand Down
197 changes: 197 additions & 0 deletions metricproviders/webmetric/webmetric.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
package webmetric

import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"net/url"
"time"

"github.com/argoproj/argo-rollouts/pkg/apis/rollouts/v1alpha1"
"github.com/argoproj/argo-rollouts/utils/evaluate"
metricutil "github.com/argoproj/argo-rollouts/utils/metric"
log "github.com/sirupsen/logrus"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/util/jsonpath"
)

const (
//ProviderType indicates the provider is prometheus
ProviderType = "WebMetric"
)

// Provider contains all the required components to run a WebMetric query
// Implements the Provider Interface
type Provider struct {
logCtx log.Entry
client *http.Client
jsonParser *jsonpath.JSONPath
}

// Type incidates provider is a WebMetric provider
func (p *Provider) Type() string {
return ProviderType
}

func (p *Provider) Run(run *v1alpha1.AnalysisRun, metric v1alpha1.Metric) v1alpha1.Measurement {
startTime := metav1.Now()

// Measurement to pass back
measurement := v1alpha1.Measurement{
StartedAt: &startTime,
}

// Create request
request := &http.Request{
Method: "GET", // TODO maybe make this configurable....also implies we will need body templates
}

url, err := url.Parse(metric.Provider.Web.URL)
if err != nil {
return metricutil.MarkMeasurementError(measurement, err)
}
request.URL = url

request.Header = make(http.Header)
for _, header := range metric.Provider.Web.Headers {
request.Header.Set(header.Key, header.Value)
}

// Send Request
response, err := p.client.Do(request)
if err != nil {
return metricutil.MarkMeasurementError(measurement, err)
dthomson25 marked this conversation as resolved.
Show resolved Hide resolved
} else if response.StatusCode < 200 || response.StatusCode >= 300 {
return metricutil.MarkMeasurementError(measurement, fmt.Errorf("received non 2xx response code: %v", response.StatusCode))
}

value, status, err := p.parseResponse(metric, response)
if err != nil {
return metricutil.MarkMeasurementError(measurement, err)
}

measurement.Value = value
measurement.Phase = status
finishedTime := metav1.Now()
measurement.FinishedAt = &finishedTime

return measurement
}

func (p *Provider) parseResponse(metric v1alpha1.Metric, response *http.Response) (string, v1alpha1.AnalysisPhase, error) {
var data interface{}

bodyBytes, err := ioutil.ReadAll(response.Body)
if err != nil {
return "", v1alpha1.AnalysisPhaseError, fmt.Errorf("Received no bytes in response: %v", err)
}

err = json.Unmarshal(bodyBytes, &data)
if err != nil {
return "", v1alpha1.AnalysisPhaseError, fmt.Errorf("Could not parse JSON body: %v", err)
}

buf := new(bytes.Buffer)
err = p.jsonParser.Execute(buf, data)
if err != nil {
return "", v1alpha1.AnalysisPhaseError, fmt.Errorf("Could not find JSONPath in body: %s", err)
}
out := buf.String()

status := p.evaluateResponse(metric, out)
return out, status, nil
}

func (p *Provider) evaluateResponse(metric v1alpha1.Metric, result interface{}) v1alpha1.AnalysisPhase {
successCondition := false
failCondition := false
var err error

if metric.SuccessCondition != "" {
successCondition, err = evaluate.EvalCondition(result, metric.SuccessCondition)
if err != nil {
p.logCtx.Warning(err.Error())
return v1alpha1.AnalysisPhaseError
}
}
if metric.FailureCondition != "" {
failCondition, err = evaluate.EvalCondition(result, metric.FailureCondition)
if err != nil {
return v1alpha1.AnalysisPhaseError
}
}

switch {
case metric.SuccessCondition == "" && metric.FailureCondition == "":
//Always return success unless there is an error
return v1alpha1.AnalysisPhaseSuccessful
case metric.SuccessCondition != "" && metric.FailureCondition == "":
// Without a failure condition, a measurement is considered a failure if the measurement's success condition is not true
failCondition = !successCondition
case metric.SuccessCondition == "" && metric.FailureCondition != "":
// Without a success condition, a measurement is considered a successful if the measurement's failure condition is not true
successCondition = !failCondition
}

if failCondition {
return v1alpha1.AnalysisPhaseFailed
}

if !failCondition && !successCondition {
return v1alpha1.AnalysisPhaseInconclusive
}

// If we reach this code path, failCondition is false and successCondition is true
return v1alpha1.AnalysisPhaseSuccessful
}

// Resume should not be used the WebMetric provider since all the work should occur in the Run method
func (p *Provider) Resume(run *v1alpha1.AnalysisRun, metric v1alpha1.Metric, measurement v1alpha1.Measurement) v1alpha1.Measurement {
p.logCtx.Warn("WebMetric provider should not execute the Resume method")
return measurement
}

// Terminate should not be used the WebMetric provider since all the work should occur in the Run method
func (p *Provider) Terminate(run *v1alpha1.AnalysisRun, metric v1alpha1.Metric, measurement v1alpha1.Measurement) v1alpha1.Measurement {
p.logCtx.Warn("WebMetric provider should not execute the Terminate method")
return measurement
}

// GarbageCollect is a no-op for the WebMetric provider
func (p *Provider) GarbageCollect(run *v1alpha1.AnalysisRun, metric v1alpha1.Metric, limit int) error {
return nil
}

func NewWebMetricHttpClient(metric v1alpha1.Metric) *http.Client {
var timeout time.Duration

// Using a default timeout of 10 seconds
if metric.Provider.Web.TimeoutSeconds <= 0 {
timeout = time.Duration(10) * time.Second
} else {
timeout = time.Duration(metric.Provider.Web.TimeoutSeconds) * time.Second
}

c := &http.Client{
Timeout: timeout,
}
return c
}

func NewWebMetricJsonParser(metric v1alpha1.Metric) (*jsonpath.JSONPath, error) {
jsonParser := jsonpath.New("metrics")

err := jsonParser.Parse(metric.Provider.Web.JSONPath)

return jsonParser, err
}

func NewWebMetricProvider(logCtx log.Entry, client *http.Client, jsonParser *jsonpath.JSONPath) *Provider {
return &Provider{
logCtx: logCtx,
client: client,
jsonParser: jsonParser,
}
}
Loading