Automate scale test #1804
Closed
Automate scale test #1804
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
dependencies
sjberman
reviewed
Apr 4, 2024
Comment on lines
+12
to
+13
| monitoring "cloud.google.com/go/monitoring/apiv3/v2" | ||
| "cloud.google.com/go/monitoring/apiv3/v2/monitoringpb" |
There was a problem hiding this comment.
Per a conversation offline, I bet it would be easier to get metrics if we just deploy our own Prometheus, and not use the GCP monitoring. Then we also don't need this extra dependency.
| // NewCSVEncoder returns a vegeta CSV encoder. | ||
| func NewCSVEncoder(w io.Writer) vegeta.Encoder { | ||
| // NewVegetaCSVEncoder returns a vegeta CSV encoder. | ||
| func NewVegetaCSVEncoder(w io.Writer) vegeta.Encoder { |
There was a problem hiding this comment.
I think the reason we kept this agnostic is if we ever change the test framework, we don't have to update the function calls.
| resultsDir, err = framework.CreateResultsDir("scale", version) | ||
| Expect(err).ToNot(HaveOccurred()) | ||
|
|
||
| filename := filepath.Join(resultsDir, fmt.Sprintf("%s.md", version)) |
There was a problem hiding this comment.
This filename formatting is slightly changed now in other tests to account for N+
|
|
||
| BeforeAll(func() { | ||
| // Need longer create timeout to allow for all the apps | ||
| resourceManager.TimeoutConfig.CreateTimeout = 180 * time.Second |
There was a problem hiding this comment.
This will change it for all tests since this is a global object
|
This PR is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 14 days. |
pleshakov
added a commit
to pleshakov/nginx-gateway-fabric
that referenced
this pull request
May 2, 2024
Problem: Non-functional scale test needs to be run manually. Solution: - Automate scale test. - Use in-cluster Prometheus to collect CPU, memory and NGF metrics. - Use Kubernetes API server to get NGF logs. For development and troubleshooting, it is possible to run scale test locally in Kind cluster. However, it is necessary to bring down the number of HTTPRoutes to 50 or less (roughly). Testing: - Ran this test locally with 64 listeners, 50 routes and 50 upstreams. - Ran this test on GKE with the default configuration. Out of scope: ensuring this test runs successfully via GitHub pipeline. Closes nginxinc#1368 Largely based on work by Ciara in nginxinc#1804 Co-authored-by: Ciara Stacke <[email protected]>
pleshakov
added a commit
to pleshakov/nginx-gateway-fabric
that referenced
this pull request
May 2, 2024
Problem: Non-functional scale test needs to be run manually. Solution: - Automate scale test. - Use in-cluster Prometheus to collect CPU, memory and NGF metrics. - Use Kubernetes API server to get NGF logs. For development and troubleshooting, it is possible to run scale test locally in Kind cluster. However, it is necessary to bring down the number of HTTPRoutes to 50 or less (roughly). Testing: - Ran this test locally with 64 listeners, 50 routes and 50 upstreams. - Ran this test on GKE with the default configuration. Out of scope: ensuring this test runs successfully via GitHub pipeline. Closes nginxinc#1368 Largely based on work by Ciara in nginxinc#1804 Co-authored-by: Ciara Stacke <[email protected]>
pleshakov
added a commit
to pleshakov/nginx-gateway-fabric
that referenced
this pull request
May 2, 2024
Problem: Non-functional scale test needs to be run manually. Solution: - Automate scale test. - Use in-cluster Prometheus to collect CPU, memory and NGF metrics. - Use Kubernetes API server to get NGF logs. For development and troubleshooting, it is possible to run scale test locally in Kind cluster. However, it is necessary to bring down the number of HTTPRoutes to 50 or less (roughly). Testing: - Ran this test locally with 64 listeners, 50 routes and 50 upstreams with NGINX OSS. - Ran this test on GKE with the default configuration with NGINX OSS. Out of scope: ensuring this test runs successfully via GitHub pipeline. Closes nginxinc#1368 Largely based on work by Ciara in nginxinc#1804 Co-authored-by: Ciara Stacke <[email protected]>
|
Shall we close in favor of #1926? |
pleshakov
added a commit
to pleshakov/nginx-gateway-fabric
that referenced
this pull request
May 16, 2024
Problem: Non-functional scale test needs to be run manually. Solution: - Automate scale test. - Use in-cluster Prometheus to collect CPU, memory and NGF metrics. - Use Kubernetes API server to get NGF logs. For development and troubleshooting, it is possible to run scale test locally in Kind cluster. However, it is necessary to bring down the number of HTTPRoutes to 50 or less (roughly). Testing: - Ran this test locally with 64 listeners, 50 routes and 50 upstreams with NGINX OSS. - Ran this test on GKE with the default configuration with NGINX OSS. Out of scope: ensuring this test runs successfully via GitHub pipeline. Closes nginxinc#1368 Largely based on work by Ciara in nginxinc#1804 Co-authored-by: Ciara Stacke <[email protected]>
pleshakov
added a commit
to pleshakov/nginx-gateway-fabric
that referenced
this pull request
May 17, 2024
Problem: Non-functional scale test needs to be run manually. Solution: - Automate scale test. - Use in-cluster Prometheus to collect CPU, memory and NGF metrics. - Use Kubernetes API server to get NGF logs. For development and troubleshooting, it is possible to run scale test locally in Kind cluster. However, it is necessary to bring down the number of HTTPRoutes to 50 or less (roughly). Testing: - Ran this test locally with 64 listeners, 50 routes and 50 upstreams with NGINX OSS. - Ran this test on GKE with the default configuration with NGINX OSS. Out of scope: ensuring this test runs successfully via GitHub pipeline. Closes nginxinc#1368 Largely based on work by Ciara in nginxinc#1804 Co-authored-by: Ciara Stacke <[email protected]>
pleshakov
added a commit
that referenced
this pull request
May 17, 2024
Problem: Non-functional scale test needs to be run manually. Solution: - Automate scale test. - Use in-cluster Prometheus to collect CPU, memory and NGF metrics. - Use Kubernetes API server to get NGF logs. For development and troubleshooting, it is possible to run scale test locally in Kind cluster. However, it is necessary to bring down the number of HTTPRoutes to 50 or less (roughly). Testing: - Ran this test locally with 64 listeners, 50 routes and 50 upstreams with NGINX OSS. - Ran this test locally with 64 listeners, 10 routes and 10 upstreams with NGINX Plus. - Ran this test on GKE with the default configuration with NGINX OSS. Out of scope: ensuring this test runs successfully via GitHub pipeline. Closes #1368 Largely based on work by Ciara in #1804 Co-authored-by: Ciara Stacke <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
Problem: Give a brief overview of the problem or feature being addressed.
Solution: Explain the approach you took to implement the solution, highlighting any significant design decisions or
considerations.
Testing: Describe any testing that you did.
Please focus on (optional): If you any specific areas where you would like reviewers to focus their attention or provide
specific feedback, add them here.
Closes #ISSUE
Checklist
Before creating a PR, run through this checklist and mark each as complete.
Release notes
If this PR introduces a change that affects users and needs to be mentioned in the release notes,
please add a brief note that summarizes the change.