Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support CloudWatch as a metric provider #1338

Merged
merged 9 commits into from
Sep 15, 2021

Conversation

ryota-sakamoto
Copy link
Member

@ryota-sakamoto ryota-sakamoto commented Jul 12, 2021

Signed-off-by: Ryota Sakamoto [email protected]

close #1014

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this is a chore.
  • The title of the PR is (a) conventional, (b) states what changed, and (c) suffixes the related issues number. E.g. "fix(controller): Updates such and such. Fixes #1234".
  • I've signed my commits with DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My builds are green. Try syncing with master if they are not.
  • My organization is added to USERS.md.

@codecov
Copy link

codecov bot commented Jul 12, 2021

Codecov Report

Merging #1338 (2906197) into master (ba9af78) will increase coverage by 0.03%.
The diff coverage is 73.21%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1338      +/-   ##
==========================================
+ Coverage   81.35%   81.38%   +0.03%     
==========================================
  Files         108      109       +1     
  Lines       14505    14685     +180     
==========================================
+ Hits        11801    11952     +151     
- Misses       2089     2111      +22     
- Partials      615      622       +7     
Impacted Files Coverage Δ
metricproviders/cloudwatch/cloudwatch.go 72.47% <72.47%> (ø)
utils/analysis/factory.go 94.44% <100.00%> (+0.10%) ⬆️
rollout/controller.go 75.19% <0.00%> (-0.35%) ⬇️
rollout/trafficrouting/istio/istio.go 80.96% <0.00%> (+0.24%) ⬆️
utils/istio/istio.go 95.00% <0.00%> (+1.52%) ⬆️
.../apis/rollouts/validation/validation_references.go 84.58% <0.00%> (+2.10%) ⬆️
rollout/trafficrouting/istio/controller.go 52.43% <0.00%> (+5.00%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba9af78...2906197. Read the comment docs.

@ryota-sakamoto ryota-sakamoto force-pushed the support-cloud-watch-metrics branch 4 times, most recently from b9cd1ec to 24dcab9 Compare July 17, 2021 16:42
@ryota-sakamoto ryota-sakamoto force-pushed the support-cloud-watch-metrics branch 4 times, most recently from ecc5061 to 77d06b7 Compare July 20, 2021 12:31
Signed-off-by: Ryota Sakamoto <[email protected]>
Signed-off-by: Ryota Sakamoto <[email protected]>
@ryota-sakamoto ryota-sakamoto force-pushed the support-cloud-watch-metrics branch from 77d06b7 to a57f475 Compare July 20, 2021 12:37
@ryota-sakamoto ryota-sakamoto changed the base branch from master to release-1.0 July 20, 2021 12:59
@ryota-sakamoto ryota-sakamoto changed the base branch from release-1.0 to master July 20, 2021 12:59
@ryota-sakamoto
Copy link
Member Author

I have no idea why the codecov report is weird.

@ryota-sakamoto ryota-sakamoto marked this pull request as ready for review July 20, 2021 13:13
pkg/apis/rollouts/v1alpha1/analysis_types.go Outdated Show resolved Hide resolved
metricproviders/cloudwatch/cloudwatch.go Show resolved Hide resolved
metricproviders/cloudwatch/cloudwatch.go Show resolved Hide resolved
Signed-off-by: Ryota Sakamoto <[email protected]>
@jessesuen
Copy link
Member

@ryota-sakamoto we're seeing if this feature could make the v1.1 release in next few weeks. Did you have thoughts about my proposed changes to the spec?

@ryota-sakamoto
Copy link
Member Author

@jessesuen
Sorry for being late.
Your idea is great and suitable, and then I'm working and testing.
There is complete when this week.

Comment on lines 205 to 231
type CloudWatchMetricDataQuery struct {
Id *string `json:"Id,omitempty" protobuf:"bytes,1,opt,name=Id"`
Expression *string `json:"Expression,omitempty" protobuf:"bytes,2,opt,name=Expression"`
Label *string `json:"Label,omitempty" protobuf:"bytes,3,opt,name=Label"`
MetricStat *CloudWatchMetricStat `json:"MetricStat,omitempty" protobuf:"bytes,4,opt,name=MetricStat"`
Period *int32 `json:"Period,omitempty" protobuf:"varint,5,opt,name=Period"`
ReturnData *bool `json:"ReturnData,omitempty" protobuf:"bytes,6,opt,name=ReturnData"`
}

type CloudWatchMetricStat struct {
Metric *CloudWatchMetricStatMetric `json:"Metric,omitempty" protobuf:"bytes,1,opt,name=Metric"`
Period *int32 `json:"Period,omitempty" protobuf:"varint,2,opt,name=Period"`
Stat *string `json:"Stat,omitempty" protobuf:"bytes,3,opt,name=Stat"`
Unit string `json:"Unit,omitempty" protobuf:"bytes,4,opt,name=Unit"`
}

type CloudWatchMetricStatMetric struct {
Dimensions []CloudWatchMetricStatMetricDimension `json:"Dimensions,omitempty" protobuf:"bytes,1,opt,name=Dimensions"`
MetricName *string `json:"MetricName,omitempty" protobuf:"bytes,2,opt,name=MetricName"`
Namespace *string `json:"Namespace,omitempty" protobuf:"bytes,3,opt,name=Namespace"`
}

type CloudWatchMetricStatMetricDimension struct {
Name *string `json:"Name,omitempty" protobuf:"bytes,1,opt,name=Name"`
Value *string `json:"Value,omitempty" protobuf:"bytes,2,opt,name=Value"`
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this new approach you use to use structured objects, instead of my suggestion of making CloudWatchMetricDataQuery being a string field. However, I think we'll need to modify this some more. I believe issues 1 and 2 may be because you copied exact datastructures from AWS go client and used as our own types:

  1. There are now a lot of new "API rule violations" because of the casing of the json field name. Kubernetes APIs require names to be camel-cased always starting with lowercase letter, but here we use uppercase in the field names. e.g.:
Name  *string `json:"Name,omitempty" protobuf:"bytes,1,opt,name=Name"`
  1. Some fields were made as pointers, when they should always have a value required. For example, I don't think these should be allowed to be pointers. Can we make everything non-pointers unless we need to explicitly distinguish between not supplied vs. empty?
type CloudWatchMetricStatMetricDimension struct {
	Name  *string `json:"Name,omitempty" protobuf:"bytes,1,opt,name=Name"`
	Value *string `json:"Value,omitempty" protobuf:"bytes,2,opt,name=Value"`
}
  1. With this approach for using structured data (instead of a giant string field), we will not allow some things to be parameterized with Analysis arguments. For example, the Period field cannot be parameterized because it is an int32.
	Period     *int32                `json:"Period,omitempty" protobuf:"varint,5,opt,name=Period"`

My question is: is Period something we should allow to be parameterized? If so, i think this should be made into an intstrutil.IntOrString instead of an int32

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I've fixed
  2. I've fixed the pointer which isn't necessary I refer https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDataQuery.html
  3. I've fixed for that I use intstrutil.IntOrString instead of int32

@ryota-sakamoto ryota-sakamoto force-pushed the support-cloud-watch-metrics branch 3 times, most recently from 26fb8a2 to 5b2994e Compare September 5, 2021 08:17
@ryota-sakamoto ryota-sakamoto force-pushed the support-cloud-watch-metrics branch from 5b2994e to ccbbe43 Compare September 5, 2021 11:53
@ryota-sakamoto ryota-sakamoto force-pushed the support-cloud-watch-metrics branch 8 times, most recently from fa224b9 to 79fa536 Compare September 8, 2021 18:40
Signed-off-by: Ryota Sakamoto <[email protected]>
@ryota-sakamoto ryota-sakamoto force-pushed the support-cloud-watch-metrics branch from 79fa536 to 2906197 Compare September 8, 2021 18:44
@sonarcloud
Copy link

sonarcloud bot commented Sep 8, 2021

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 3 Code Smells

No Coverage information No Coverage information
2.5% 2.5% Duplication

@harikrongali
Copy link
Contributor

@jessesuen can we merge this PR? seems @ryota-sakamoto addressed review comments

@harikrongali harikrongali added this to the v1.1 milestone Sep 13, 2021
Copy link
Member

@jessesuen jessesuen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!

@jessesuen jessesuen merged commit 180b6fb into argoproj:master Sep 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support CloudWatch as a metric provider
3 participants