-
Notifications
You must be signed in to change notification settings - Fork 542
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Scalability and Performance Test Cases for Ephemeral Volumes #601
Merged
k8s-ci-robot
merged 5 commits into
kubernetes:master
from
mucahitkurt:storage/scalability-performance-tests-pod-startup
Jul 16, 2019
Merged
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
e914fd1
pod with ephemeral volumes test cases
mucahitkurt 185997a
simplify test cases only 1-node test cases and separate pod configs f…
mucahitkurt bf31435
move podstartupmeasurements to up
mucahitkurt 30effd8
move podstartupmeasurements gather to bottom, rename volume config files
mucahitkurt 377ab09
override file is added for max volumes per node test case, update def…
mucahitkurt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
8 changes: 8 additions & 0 deletions
8
...er2/testing/experimental/storage/pod-startup/ephemeral-volumes/1_node/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
Below test cases can be run with the parameters like; | ||
|
||
- 1 pod with X volumes and 1 node | ||
-- `.Nodes := 1`, `$TOTAL_PODS := 1`, `$VOLUMES_PER_POD := <X>` | ||
- X pods with 1 volume each on 1 node in parallel | ||
-- `.Nodes := 1`, `$TOTAL_PODS := <X>`, `$VOLUMES_PER_POD := 1` | ||
|
||
To test with different volume types please use the override file for the specific volume type, default volume type is `EmptyDir` |
93 changes: 93 additions & 0 deletions
93
clusterloader2/testing/experimental/storage/pod-startup/ephemeral-volumes/1_node/config.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,93 @@ | ||
# ASSUMPTIONS: | ||
# - this test cases should be run on only 1 node | ||
|
||
#Constants | ||
{{$TOTAL_PODS := DefaultParam .TOTAL_PODS 1}} | ||
{{$POD_TEMPLATE_PATH := DefaultParam .POD_TEMPLATE_PATH "pod_with_emptydir.yaml"}} | ||
{{$LOAD_TEST_THROUGHPUT := DefaultParam .LOAD_TEST_THROUGHPUT 10}} | ||
|
||
{{$VOLUMES_PER_POD := DefaultParam .VOLUMES_PER_POD 4}} | ||
{{$VOLUME_TEMPLATE_PATH := .VOLUME_TEMPLATE_PATH}} | ||
{{$PROVISION_VOLUME := DefaultParam .PROVISION_VOLUME false}} | ||
|
||
{{$APP_NAME := "pod-load"}} | ||
{{$GROUP := "pod-with-ephemeral-volume-startup-latency"}} | ||
|
||
#Variables | ||
# used stateless pod SLO for initial run | ||
{{$podStartupTimeout := 5}} | ||
{{$totalVolumes := MultiplyInt $TOTAL_PODS $VOLUMES_PER_POD}} | ||
|
||
name: pod-with-ephemeral-volume-startup-latency | ||
automanagedNamespaces: 1 | ||
tuningSets: | ||
- name: UniformQPS | ||
qpsLoad: | ||
qps: {{$LOAD_TEST_THROUGHPUT}} | ||
steps: | ||
# Start measurements | ||
- measurements: | ||
- Identifier: APIResponsiveness | ||
Method: APIResponsiveness | ||
Params: | ||
action: reset | ||
- Identifier: PodWithMultiVolumeStartupLatency | ||
Method: PodStartupLatency | ||
Params: | ||
action: start | ||
labelSelector: group = {{$GROUP}} | ||
threshold: {{$podStartupTimeout}}s | ||
{{ if $PROVISION_VOLUME }} | ||
# Provision volumes | ||
- phases: | ||
- namespaceRange: | ||
min: 1 | ||
max: 1 | ||
replicasPerNamespace: {{$totalVolumes}} | ||
tuningSet: UniformQPS | ||
objectBundle: | ||
- basename: vol-{{$APP_NAME}} | ||
objectTemplatePath: {{$VOLUME_TEMPLATE_PATH}} | ||
{{ end }} | ||
# Create pods | ||
- phases: | ||
- namespaceRange: | ||
min: 1 | ||
max: 1 | ||
replicasPerNamespace: {{$TOTAL_PODS}} | ||
tuningSet: UniformQPS | ||
objectBundle: | ||
- basename: {{$APP_NAME}} | ||
objectTemplatePath: {{$POD_TEMPLATE_PATH}} | ||
templateFillMap: | ||
Group: {{$GROUP}} | ||
VolumesPerPod: {{$VOLUMES_PER_POD}} | ||
AppName: {{$APP_NAME}} | ||
- measurements: | ||
- Identifier: WaitForRunningPodsWithStorage | ||
Method: WaitForRunningPods | ||
Params: | ||
desiredPodCount: {{$TOTAL_PODS}} | ||
labelSelector: group = {{$GROUP}} | ||
# TODO decide this after test roll-out phase | ||
timeout: 15m | ||
# Delete pods | ||
- phases: | ||
- namespaceRange: | ||
min: 1 | ||
max: 1 | ||
replicasPerNamespace: 0 | ||
tuningSet: UniformQPS | ||
objectBundle: | ||
- basename: {{$APP_NAME}} | ||
objectTemplatePath: {{$POD_TEMPLATE_PATH}} | ||
# Collect measurements | ||
- measurements: | ||
- Identifier: PodWithMultiVolumeStartupLatency | ||
Method: PodStartupLatency | ||
Params: | ||
action: gather | ||
- Identifier: APIResponsiveness | ||
Method: APIResponsiveness | ||
Params: | ||
action: gather |
6 changes: 6 additions & 0 deletions
6
...esting/experimental/storage/pod-startup/ephemeral-volumes/1_node/configmap/configmap.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
apiVersion: v1 | ||
kind: ConfigMap | ||
metadata: | ||
name: {{.Name}} | ||
data: | ||
TEST_KEY1: testData1 |
3 changes: 3 additions & 0 deletions
3
...testing/experimental/storage/pod-startup/ephemeral-volumes/1_node/configmap/override.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
PROVISION_VOLUME: true | ||
POD_TEMPLATE_PATH: "configmap/pod_with_configmap.yaml" | ||
VOLUME_TEMPLATE_PATH: "configmap/configmap.yaml" |
26 changes: 26 additions & 0 deletions
26
...perimental/storage/pod-startup/ephemeral-volumes/1_node/configmap/pod_with_configmap.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
{{$index := .Index}} | ||
{{$appName := .AppName}} | ||
{{$volumesPerPod := .VolumesPerPod}} | ||
apiVersion: v1 | ||
kind: Pod | ||
metadata: | ||
name: {{.Name}} | ||
labels: | ||
app: {{.Name}} | ||
group: {{.Group}} | ||
spec: | ||
containers: | ||
- name: {{.Name}} | ||
image: k8s.gcr.io/pause:3.1 | ||
imagePullPolicy: IfNotPresent | ||
volumeMounts: | ||
{{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
- name: vol-{{$volumeIndex}} | ||
mountPath: /usr/share/{{$volumeIndex}} | ||
{{ end }} | ||
volumes: | ||
{{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
- name: vol-{{$volumeIndex}} | ||
configMap: | ||
name: vol-{{$appName}}-{{AddInt $volumeIndex (MultiplyInt $index $volumesPerPod)}} | ||
{{ end }} |
1 change: 1 addition & 0 deletions
1
...sting/experimental/storage/pod-startup/ephemeral-volumes/1_node/downwardapi/override.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
POD_TEMPLATE_PATH: "downwardapi/pod_with_downwardapi.yaml" |
32 changes: 32 additions & 0 deletions
32
...mental/storage/pod-startup/ephemeral-volumes/1_node/downwardapi/pod_with_downwardapi.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
{{$index := .Index}} | ||
{{$appName := .AppName}} | ||
{{$volumesPerPod := .VolumesPerPod}} | ||
apiVersion: v1 | ||
kind: Pod | ||
metadata: | ||
name: {{.Name}} | ||
labels: | ||
app: {{.Name}} | ||
group: {{.Group}} | ||
spec: | ||
containers: | ||
- name: {{.Name}} | ||
image: k8s.gcr.io/pause:3.1 | ||
imagePullPolicy: IfNotPresent | ||
volumeMounts: | ||
{{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
- name: vol-{{$volumeIndex}} | ||
mountPath: /usr/share/{{$volumeIndex}} | ||
{{ end }} | ||
volumes: | ||
{{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
- name: vol-{{$volumeIndex}} | ||
downwardAPI: | ||
items: | ||
- path: "pod_name" | ||
fieldRef: | ||
fieldPath: metadata.name | ||
- path: "pod_namespace" | ||
fieldRef: | ||
fieldPath: metadata.namespace | ||
{{ end }} |
25 changes: 25 additions & 0 deletions
25
.../testing/experimental/storage/pod-startup/ephemeral-volumes/1_node/pod_with_emptydir.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
{{$index := .Index}} | ||
{{$appName := .AppName}} | ||
{{$volumesPerPod := .VolumesPerPod}} | ||
apiVersion: v1 | ||
kind: Pod | ||
metadata: | ||
name: {{.Name}} | ||
labels: | ||
app: {{.Name}} | ||
group: {{.Group}} | ||
spec: | ||
containers: | ||
- name: {{.Name}} | ||
image: k8s.gcr.io/pause:3.1 | ||
imagePullPolicy: IfNotPresent | ||
volumeMounts: | ||
{{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
- name: vol-{{$volumeIndex}} | ||
mountPath: /usr/share/{{$volumeIndex}} | ||
{{ end }} | ||
volumes: | ||
{{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
- name: vol-{{$volumeIndex}} | ||
emptyDir: {} | ||
{{ end }} |
3 changes: 3 additions & 0 deletions
3
...r2/testing/experimental/storage/pod-startup/ephemeral-volumes/1_node/secret/override.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
PROVISION_VOLUME: true | ||
POD_TEMPLATE_PATH: "secret/pod_with_secret.yaml" | ||
VOLUME_TEMPLATE_PATH: "secret/secret.yaml" |
26 changes: 26 additions & 0 deletions
26
...ing/experimental/storage/pod-startup/ephemeral-volumes/1_node/secret/pod_with_secret.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
{{$index := .Index}} | ||
{{$appName := .AppName}} | ||
{{$volumesPerPod := .VolumesPerPod}} | ||
apiVersion: v1 | ||
kind: Pod | ||
metadata: | ||
name: {{.Name}} | ||
labels: | ||
app: {{.Name}} | ||
group: {{.Group}} | ||
spec: | ||
containers: | ||
- name: {{.Name}} | ||
image: k8s.gcr.io/pause:3.1 | ||
imagePullPolicy: IfNotPresent | ||
volumeMounts: | ||
{{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
- name: vol-{{$volumeIndex}} | ||
mountPath: /usr/share/{{$volumeIndex}} | ||
{{ end }} | ||
volumes: | ||
{{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
- name: vol-{{$volumeIndex}} | ||
secret: | ||
secretName: vol-{{$appName}}-{{AddInt $volumeIndex (MultiplyInt $index $volumesPerPod)}} | ||
{{ end }} |
8 changes: 8 additions & 0 deletions
8
...der2/testing/experimental/storage/pod-startup/ephemeral-volumes/1_node/secret/secret.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
apiVersion: v1 | ||
kind: Secret | ||
metadata: | ||
name: {{.Name}} | ||
type: Opaque | ||
data: | ||
username: dXNlcm5hbWVfCg== | ||
password: cGFzc3dvcmRfCg== |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@krzysied - I was thinking about it in the past, but I don't know if we ever did that (and if I ever shared with you that I was thinking about it).
In majority of cases, we're setting "namespaceRange" to basically all automanaged-namespaces.
What I would like to do is to remove all of those and say:
"if you don't specify namespaceRange, we default to all automanaged namespaces"
Is that done? If not, would you be able to add that (should be quite simple to do).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
perf-tests/clusterloader2/pkg/test/simple_test_executor.go
Lines 312 to 315 in 459486e
If namespaceRange is unset, the cluster level object will be created. IIRC currently we are not using this feature. The question is if we want to create non-namespace objects in the future...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ahh ok - we definitely would like to be able to create non-namespaced objects (e.g. CRD is non-namespaced).
We should probably come up with some "allNamespaces" thing to reduce amount of boilerplate...