Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make AWS credentials configurable easily #10868

Merged
merged 1 commit into from
Jan 22, 2019
+40 −36
Merged

Make AWS credentials configurable easily #10868

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion config/jobs/kubernetes-security/generated-security-jobs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ presubmits:
cluster: security
context: pull-security-kubernetes-e2e-aws-eks-1-11-correctness
labels:
preset-aws-credential: aws-oss-testing
preset-kubernetes-e2e-aws-eks-1-11: "true"
preset-kubernetes-e2e-aws-eks-common: "true"
preset-service-account: "true"
name: pull-security-kubernetes-e2e-aws-eks-1-11-correctness
optional: true
Expand Down
10 changes: 5 additions & 5 deletions config/jobs/kubernetes/sig-aws/eks/k8s-aws-eks-periodics.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ periodics:
name: ci-kubernetes-e2e-aws-eks-1-11-correctness
labels:
preset-service-account: "true"
preset-kubernetes-e2e-aws-eks-common: "true"
preset-aws-credential: "aws-oss-testing"
preset-kubernetes-e2e-aws-eks-1-11: "true"
spec:
containers:
Expand All @@ -29,7 +29,7 @@ periodics:
name: ci-kubernetes-e2e-aws-eks-1-11-conformance
labels:
preset-service-account: "true"
preset-kubernetes-e2e-aws-eks-common: "true"
preset-aws-credential: "aws-oss-testing"
preset-kubernetes-e2e-aws-eks-1-11: "true"
spec:
containers:
Expand All @@ -54,7 +54,7 @@ periodics:
name: ci-kubernetes-e2e-aws-eks-1-11-scalability
labels:
preset-service-account: "true"
preset-kubernetes-e2e-aws-eks-common: "true"
preset-aws-credential: "aws-oss-testing"
preset-kubernetes-e2e-aws-eks-1-11: "true"
spec:
containers:
Expand All @@ -78,7 +78,7 @@ periodics:
name: ci-kubernetes-e2e-aws-eks-1-10-correctness
labels:
preset-service-account: "true"
preset-kubernetes-e2e-aws-eks-common: "true"
preset-aws-credential: "aws-oss-testing"
preset-kubernetes-e2e-aws-eks-1-10: "true"
spec:
containers:
Expand All @@ -103,7 +103,7 @@ periodics:
name: ci-kubernetes-e2e-aws-eks-1-10-conformance
labels:
preset-service-account: "true"
preset-kubernetes-e2e-aws-eks-common: "true"
preset-aws-credential: "aws-oss-testing"
preset-kubernetes-e2e-aws-eks-1-10: "true"
spec:
containers:
Expand Down
31 changes: 18 additions & 13 deletions config/jobs/kubernetes/sig-aws/eks/k8s-aws-eks-presets.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,16 @@
presets:
- env:
# URL to download 'kubectl', required for 'kubectl' calls to EKS (https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html)
# TODO: use upstream 'kubectl'
- name: AWS_K8S_TESTER_EKS_KUBECTL_DOWNLOAD_URL
value: https://amazon-eks.s3-us-west-2.amazonaws.com/1.11.5/2018-12-06/bin/linux/amd64/kubectl
# URL to download 'aws-iam-authenticator', required for 'kubectl' calls to EKS (https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html)
- name: AWS_K8S_TESTER_EKS_AWS_IAM_AUTHENTICATOR_DOWNLOAD_URL
value: https://amazon-eks.s3-us-west-2.amazonaws.com/1.11.5/2018-12-06/bin/linux/amd64/aws-iam-authenticator
# AWS test account credential mounted path, required for AWS API call
# Credentials for using AWS test account 607362164682.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will leave for a follow-up, but perhaps the intention of who should be using this account for what kind of testing? :^)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm soon planning to organize all aws testing account creds at a central place under jobs/sig-aws/. Working with @krzyzacy to get the prow secrets right. I'll clarify it in follow-up PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #10886

- name: AWS_SHARED_CREDENTIALS_FILE
value: /etc/eks-aws-credentials/eks-aws-credentials
value: /etc/aws-cred/credentials
labels:
preset-kubernetes-e2e-aws-eks-common: "true"
preset-aws-credential: "aws-oss-testing"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we put a comment describing what the usage should be?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Added comment above preset definition.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, this should be a nice step towards being easier to follow the config :-)

volumeMounts:
- mountPath: /etc/eks-aws-credentials
name: eks-aws-credentials
- mountPath: /etc/aws-cred
name: aws-cred
readOnly: true
volumes:
- name: eks-aws-credentials
- name: aws-cred
secret:
secretName: eks-aws-credentials

Expand All @@ -28,6 +21,12 @@ presets:
# Amazon EKS-optimized AMI (non-GPU, https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html)
- name: AWS_K8S_TESTER_EKS_WORKER_NODE_AMI
value: ami-0a2abab4107669c1b
# URL to download 'kubectl', required for 'kubectl' calls to EKS (https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html)
- name: AWS_K8S_TESTER_EKS_KUBECTL_DOWNLOAD_URL
value: https://amazon-eks.s3-us-west-2.amazonaws.com/1.11.5/2018-12-06/bin/linux/amd64/kubectl
# URL to download 'aws-iam-authenticator', required for 'kubectl' calls to EKS (https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html)
- name: AWS_K8S_TESTER_EKS_AWS_IAM_AUTHENTICATOR_DOWNLOAD_URL
value: https://amazon-eks.s3-us-west-2.amazonaws.com/1.11.5/2018-12-06/bin/linux/amd64/aws-iam-authenticator
labels:
preset-kubernetes-e2e-aws-eks-1-11: "true"

Expand All @@ -38,5 +37,11 @@ presets:
# Amazon EKS-optimized AMI (non-GPU, https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html)
- name: AWS_K8S_TESTER_EKS_WORKER_NODE_AMI
value: ami-09e1df3bad220af0b
# URL to download 'kubectl', required for 'kubectl' calls to EKS (https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html)
- name: AWS_K8S_TESTER_EKS_KUBECTL_DOWNLOAD_URL
value: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.11/2018-12-06/bin/linux/amd64/kubectl
# URL to download 'aws-iam-authenticator', required for 'kubectl' calls to EKS (https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html)
- name: AWS_K8S_TESTER_EKS_AWS_IAM_AUTHENTICATOR_DOWNLOAD_URL
value: https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.11/2018-12-06/bin/linux/amd64/aws-iam-authenticator
labels:
preset-kubernetes-e2e-aws-eks-1-10: "true"
2 changes: 1 addition & 1 deletion config/jobs/kubernetes/sig-aws/eks/k8s-aws-eks-presubmits.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ presubmits:
optional: true
labels:
preset-service-account: "true"
preset-kubernetes-e2e-aws-eks-common: "true"
preset-aws-credential: "aws-oss-testing"
preset-kubernetes-e2e-aws-eks-1-11: "true"
spec:
containers:
Expand Down
20 changes: 9 additions & 11 deletions config/tests/jobs/jobs_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -589,7 +589,7 @@ func TestLatestUsesImagePullPolicy(t *testing.T) {

// checkKubekinsPresets returns an error if a spec references to kubekins-e2e|bootstrap image,
// but doesn't use service preset or ssh preset
func checkKubekinsPresets(jobName string, spec *v1.PodSpec, labels, validLabels map[string]string) error {
func checkKubekinsPresets(jobName string, spec *v1.PodSpec, labels map[string]string, validLabels map[string]bool) error {
service := true
ssh := true

Expand Down Expand Up @@ -629,10 +629,9 @@ func checkKubekinsPresets(jobName string, spec *v1.PodSpec, labels, validLabels
}

for key, val := range labels {
if validVal, ok := validLabels[key]; !ok {
return fmt.Errorf("label %s is not a valid preset label", key)
} else if validVal != val {
return fmt.Errorf("label %s does not have valid value, have %s, expect %s", key, val, validVal)
pair := key + ":" + val
if validVal, ok := validLabels[pair]; !ok || !validVal {
return fmt.Errorf("key-value pair %s is not found in list of valid presets list", pair)
}
}

Expand All @@ -642,18 +641,17 @@ func checkKubekinsPresets(jobName string, spec *v1.PodSpec, labels, validLabels
// TestValidPresets makes sure all presets name starts with 'preset-', all job presets are valid,
// and jobs that uses kubekins-e2e image has the right service account preset
func TestValidPresets(t *testing.T) {
validLabels := map[string]string{}
validLabels := map[string]bool{}
for _, preset := range c.Presets {
for label, val := range preset.Labels {
if !strings.HasPrefix(label, "preset-") {
t.Errorf("Preset label %s - label name should start with 'preset-'", label)
} else if val != "true" {
t.Errorf("Preset label %s - label value should be true", label)
}
if _, ok := validLabels[label]; ok {
t.Errorf("Duplicated preset label : %s", label)
pair := label + ":" + val
if _, ok := validLabels[pair]; ok {
t.Errorf("Duplicated preset 'label:value' pair : %s", pair)
} else {
validLabels[label] = val
validLabels[pair] = true
}
}
}
Expand Down
11 changes: 6 additions & 5 deletions prow/config/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -469,14 +469,15 @@ func (c *Config) mergeJobConfig(jc JobConfig) error {
// *** Presets ***
c.Presets = append(c.Presets, jc.Presets...)

// validate no duplicated presets
validLabels := map[string]string{}
// validate no duplicated preset key-value pairs
validLabels := map[string]bool{}
for _, preset := range c.Presets {
for label, val := range preset.Labels {
if _, ok := validLabels[label]; ok {
return fmt.Errorf("duplicated preset label : %s", label)
pair := label + ":" + val
if _, ok := validLabels[pair]; ok {
return fmt.Errorf("duplicated preset 'label:value' pair : %s", pair)
}
validLabels[label] = val
validLabels[pair] = true
}
}

Expand Down