Scalability and Performance Test Cases for Ephemeral Volumes #601
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| Below test cases can be run with the parameters like; | ||
|
|
||
| - 1 pod with X volumes and 1 node | ||
| -- `.Nodes := 1`, `$TOTAL_PODS := 1`, `$VOLUMES_PER_POD := <X>` | ||
| - X pods with 1 volume each on 1 node in parallel | ||
| -- `.Nodes := 1`, `$TOTAL_PODS := <X>`, `$VOLUMES_PER_POD := 1` | ||
|
|
||
| To test with different volume types please use the override file for the specific volume type, default volume type is `EmptyDir` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| # ASSUMPTIONS: | ||
| # - this test cases should be run on only 1 node | ||
|
|
||
| #Constants | ||
| {{$TOTAL_PODS := DefaultParam .TOTAL_PODS 1}} | ||
| {{$POD_TEMPLATE_PATH := DefaultParam .POD_TEMPLATE_PATH "pod_with_EmptyDirs.yaml"}} | ||
| {{$LOAD_TEST_THROUGHPUT := DefaultParam .LOAD_TEST_THROUGHPUT 10}} | ||
|
|
||
| {{$VOLUMES_PER_POD := DefaultParam .VOLUMES_PER_POD 4}} | ||
| {{$VOLUME_TEMPLATE_PATH := .VOLUME_TEMPLATE_PATH}} | ||
| {{$PROVISION_VOLUME := DefaultParam .PROVISION_VOLUME false}} | ||
|
|
||
| {{$APP_NAME := "pod-load"}} | ||
| {{$GROUP := "pod-with-ephemeral-volume-startup-latency"}} | ||
|
|
||
| #Variables | ||
| # used stateless pod SLO for initial run | ||
| {{$podStartupTimeout := 5}} | ||
| {{$totalVolumes := MultiplyInt $TOTAL_PODS $VOLUMES_PER_POD}} | ||
|
|
||
| name: pod-with-ephemeral-volume-startup-latency | ||
| automanagedNamespaces: 1 | ||
| tuningSets: | ||
| - name: UniformQPS | ||
| qpsLoad: | ||
| qps: {{$LOAD_TEST_THROUGHPUT}} | ||
| steps: | ||
| - measurements: | ||
| - Identifier: APIResponsiveness | ||
| Method: APIResponsiveness | ||
| Params: | ||
| action: reset | ||
| {{ if $PROVISION_VOLUME }} | ||
| # Provision volumes | ||
| - phases: | ||
| - namespaceRange: | ||
| min: 1 | ||
| max: 1 | ||
| replicasPerNamespace: {{$totalVolumes}} | ||
| tuningSet: UniformQPS | ||
| objectBundle: | ||
| - basename: vol-{{$APP_NAME}} | ||
| objectTemplatePath: {{$VOLUME_TEMPLATE_PATH}} | ||
| {{ end }} | ||
| # Start measurements | ||
| - measurements: | ||
| - Identifier: PodWithMultiVolumeStartupLatency | ||
| Method: PodStartupLatency | ||
| Params: | ||
| action: start | ||
| labelSelector: group = {{$GROUP}} | ||
| threshold: {{$podStartupTimeout}}s | ||
|
|
||
| # Create pods | ||
| - phases: | ||
| - namespaceRange: | ||
| min: 1 | ||
| max: 1 | ||
| replicasPerNamespace: {{$TOTAL_PODS}} | ||
| tuningSet: UniformQPS | ||
| objectBundle: | ||
| - basename: {{$APP_NAME}} | ||
| objectTemplatePath: {{$POD_TEMPLATE_PATH}} | ||
| templateFillMap: | ||
| Group: {{$GROUP}} | ||
| VolumesPerPod: {{$VOLUMES_PER_POD}} | ||
| AppName: {{$APP_NAME}} | ||
| - measurements: | ||
| - Identifier: WaitForRunningPodsWithStorage | ||
| Method: WaitForRunningPods | ||
| Params: | ||
| desiredPodCount: {{$TOTAL_PODS}} | ||
| labelSelector: group = {{$GROUP}} | ||
| # TODO decide this after test roll-out phase | ||
| timeout: 15m | ||
| - measurements: | ||
|
There was a problem hiding this comment. Similarly as for starting measurement, can you also move it to the bottom? [As I mentioned somewhere else, medium term, we will simply bake all the necessary thresholds into the measurement itself and we will bundle all SLO-related ones together to limit boilerplate. It will just be simpler to do then :)] There was a problem hiding this comment. ok., I think deleting pods before the gather action of PodStartupLatency measurement has no effect? There was a problem hiding this comment. PodStartupLatency measurement can only change if when pods are being created - so yes, that will have no effect. |
||
| - Identifier: PodWithMultiVolumeStartupLatency | ||
| Method: PodStartupLatency | ||
| Params: | ||
| action: gather | ||
| # Delete pods | ||
| - phases: | ||
| - namespaceRange: | ||
| min: 1 | ||
| max: 1 | ||
| replicasPerNamespace: 0 | ||
| tuningSet: UniformQPS | ||
| objectBundle: | ||
| - basename: {{$APP_NAME}} | ||
| objectTemplatePath: {{$POD_TEMPLATE_PATH}} | ||
| # Collect measurements | ||
| - measurements: | ||
| - Identifier: APIResponsiveness | ||
| Method: APIResponsiveness | ||
| Params: | ||
| action: gather | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: {{.Name}} | ||
| data: | ||
| TEST_KEY1: testData1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| PROVISION_VOLUME: true | ||
| POD_TEMPLATE_PATH: "configmap/pod_with_ConfigMaps.yaml" | ||
| VOLUME_TEMPLATE_PATH: "configmap/ConfigMap.yaml" | ||
|
There was a problem hiding this comment. Do you somewhere use the assumption that the file names are capitalized ConfigMap (I don't see that, but maybe I'm missing something). [Same for secrets and others] There was a problem hiding this comment. There was but no longer, I'll rename them, but volume names are not used as separate words generally, usages are like There was a problem hiding this comment. Sure that sounds fine, thanks! |
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| {{$index := .Index}} | ||
| {{$appName := .AppName}} | ||
| {{$volumesPerPod := .VolumesPerPod}} | ||
| apiVersion: v1 | ||
| kind: Pod | ||
| metadata: | ||
| name: {{.Name}} | ||
| labels: | ||
| app: {{.Name}} | ||
| group: {{.Group}} | ||
| spec: | ||
| containers: | ||
| - name: {{.Name}} | ||
| image: k8s.gcr.io/pause:3.1 | ||
| imagePullPolicy: IfNotPresent | ||
| volumeMounts: | ||
| {{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
| - name: vol-{{$volumeIndex}} | ||
| mountPath: /usr/share/{{$volumeIndex}} | ||
| {{ end }} | ||
| volumes: | ||
| {{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
| - name: vol-{{$volumeIndex}} | ||
| configMap: | ||
| name: vol-{{$appName}}-{{AddInt $volumeIndex (MultiplyInt $index $volumesPerPod)}} | ||
| {{ end }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| POD_TEMPLATE_PATH: "downwardapi/pod_with_DownwardAPIs.yaml" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| {{$index := .Index}} | ||
| {{$appName := .AppName}} | ||
| {{$volumesPerPod := .VolumesPerPod}} | ||
| apiVersion: v1 | ||
| kind: Pod | ||
| metadata: | ||
| name: {{.Name}} | ||
| labels: | ||
| app: {{.Name}} | ||
| group: {{.Group}} | ||
| spec: | ||
| containers: | ||
| - name: {{.Name}} | ||
| image: k8s.gcr.io/pause:3.1 | ||
| imagePullPolicy: IfNotPresent | ||
| volumeMounts: | ||
| {{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
| - name: vol-{{$volumeIndex}} | ||
| mountPath: /usr/share/{{$volumeIndex}} | ||
| {{ end }} | ||
| volumes: | ||
| {{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
| - name: vol-{{$volumeIndex}} | ||
| downwardAPI: | ||
| items: | ||
| - path: "pod_name" | ||
| fieldRef: | ||
| fieldPath: metadata.name | ||
| - path: "pod_namespace" | ||
| fieldRef: | ||
| fieldPath: metadata.namespace | ||
| {{ end }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| {{$index := .Index}} | ||
| {{$appName := .AppName}} | ||
| {{$volumesPerPod := .VolumesPerPod}} | ||
| apiVersion: v1 | ||
| kind: Pod | ||
| metadata: | ||
| name: {{.Name}} | ||
| labels: | ||
| app: {{.Name}} | ||
| group: {{.Group}} | ||
| spec: | ||
| containers: | ||
| - name: {{.Name}} | ||
| image: k8s.gcr.io/pause:3.1 | ||
| imagePullPolicy: IfNotPresent | ||
| volumeMounts: | ||
| {{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
| - name: vol-{{$volumeIndex}} | ||
| mountPath: /usr/share/{{$volumeIndex}} | ||
| {{ end }} | ||
| volumes: | ||
| {{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
| - name: vol-{{$volumeIndex}} | ||
| emptyDir: {} | ||
| {{ end }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| apiVersion: v1 | ||
| kind: Secret | ||
| metadata: | ||
| name: {{.Name}} | ||
| type: Opaque | ||
| data: | ||
| username: dXNlcm5hbWVfCg== | ||
| password: cGFzc3dvcmRfCg== |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| PROVISION_VOLUME: true | ||
| POD_TEMPLATE_PATH: "secret/pod_with_Secrets.yaml" | ||
| VOLUME_TEMPLATE_PATH: "secret/Secret.yaml" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| {{$index := .Index}} | ||
| {{$appName := .AppName}} | ||
| {{$volumesPerPod := .VolumesPerPod}} | ||
| apiVersion: v1 | ||
| kind: Pod | ||
| metadata: | ||
| name: {{.Name}} | ||
| labels: | ||
| app: {{.Name}} | ||
| group: {{.Group}} | ||
| spec: | ||
| containers: | ||
| - name: {{.Name}} | ||
| image: k8s.gcr.io/pause:3.1 | ||
| imagePullPolicy: IfNotPresent | ||
| volumeMounts: | ||
| {{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
| - name: vol-{{$volumeIndex}} | ||
| mountPath: /usr/share/{{$volumeIndex}} | ||
| {{ end }} | ||
| volumes: | ||
| {{ range $volumeIndex, $vol := Seq .VolumesPerPod }} | ||
| - name: vol-{{$volumeIndex}} | ||
| secret: | ||
| secretName: vol-{{$appName}}-{{AddInt $volumeIndex (MultiplyInt $index $volumesPerPod)}} | ||
| {{ end }} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@krzysied - I was thinking about it in the past, but I don't know if we ever did that (and if I ever shared with you that I was thinking about it).
In majority of cases, we're setting "namespaceRange" to basically all automanaged-namespaces.
What I would like to do is to remove all of those and say:
"if you don't specify namespaceRange, we default to all automanaged namespaces"
Is that done? If not, would you be able to add that (should be quite simple to do).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
perf-tests/clusterloader2/pkg/test/simple_test_executor.go
Lines 312 to 315 in 459486e
If namespaceRange is unset, the cluster level object will be created. IIRC currently we are not using this feature. The question is if we want to create non-namespace objects in the future...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ahh ok - we definitely would like to be able to create non-namespaced objects (e.g. CRD is non-namespaced).
We should probably come up with some "allNamespaces" thing to reduce amount of boilerplate...