Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automate Scale Test #1368

Closed
Tracked by #994
mpstefan opened this issue Dec 11, 2023 · 1 comment · Fixed by #1926
Closed
Tracked by #994

Automate Scale Test #1368

mpstefan opened this issue Dec 11, 2023 · 1 comment · Fixed by #1926
Assignees
@pleshakov
Labels
refined Requirements are refined and the issue is ready to be implemented. size/medium Estimated to be completed within a week tests Pull requests that update tests
Milestone
v1.3.0

Comments

@mpstefan
Copy link
Collaborator

mpstefan commented Dec 11, 2023 •
edited
Loading

As a maintainer of NGF
I want to automate the scale test
So that I do not need to run it manually in the future.

Acceptance

  • The scale test is automated to the point of being able to run the entire test from running a script.
  • If the results of the test can be retrieved in an automated fashion, the results are collected and presented in the output.
@mpstefan mpstefan mentioned this issue Dec 11, 2023
Closed
@mpstefan mpstefan added the tests Pull requests that update tests label Dec 11, 2023
@mpstefan mpstefan added this to the v1.2.0 milestone Dec 11, 2023
@mpstefan mpstefan added refined Requirements are refined and the issue is ready to be implemented. size/medium Estimated to be completed within a week labels Jan 22, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 6, 2024

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 14 days.

@github-actions github-actions bot added the stale Pull requests/issues with no activity label Feb 6, 2024
@sjberman sjberman removed the stale Pull requests/issues with no activity label Feb 6, 2024
@mpstefan mpstefan modified the milestones: v1.2.0, v1.3.0 Mar 6, 2024
@ciarams87 ciarams87 self-assigned this Mar 7, 2024
@ciarams87 ciarams87 moved this from 🆕 New to 🏗 In Progress in NGINX Gateway Fabric Mar 7, 2024
@mpstefan mpstefan assigned pleshakov and unassigned ciarams87 Apr 17, 2024
pleshakov added a commit to pleshakov/nginx-gateway-fabric that referenced this issue May 2, 2024
@pleshakov @ciarams87
Automate scale test
2ba7288 
Problem:
Non-functional scale test needs to be run manually.

Solution:
- Automate scale test.
- Use in-cluster Prometheus to collect CPU, memory and NGF metrics.
- Use Kubernetes API server to get NGF logs.

For development and troubleshooting, it is possible to run scale test
locally in Kind cluster. However, it is necessary to bring down
the number of HTTPRoutes to 50 or less (roughly).

Testing:
- Ran this test locally with 64 listeners, 50 routes and 50 upstreams.
- Ran this test on GKE with the default configuration.

Out of scope: ensuring this test runs successfully via GitHub pipeline.

Closes nginxinc#1368

Largely based on work by Ciara in
nginxinc#1804

Co-authored-by: Ciara Stacke <[email protected]>
pleshakov added a commit to pleshakov/nginx-gateway-fabric that referenced this issue May 2, 2024
@pleshakov @ciarams87
Automate scale test
f97122c 
Problem:
Non-functional scale test needs to be run manually.

Solution:
- Automate scale test.
- Use in-cluster Prometheus to collect CPU, memory and NGF metrics.
- Use Kubernetes API server to get NGF logs.

For development and troubleshooting, it is possible to run scale test
locally in Kind cluster. However, it is necessary to bring down
the number of HTTPRoutes to 50 or less (roughly).

Testing:
- Ran this test locally with 64 listeners, 50 routes and 50 upstreams.
- Ran this test on GKE with the default configuration.

Out of scope: ensuring this test runs successfully via GitHub pipeline.

Closes nginxinc#1368

Largely based on work by Ciara in
nginxinc#1804

Co-authored-by: Ciara Stacke <[email protected]>
pleshakov added a commit to pleshakov/nginx-gateway-fabric that referenced this issue May 2, 2024
@pleshakov @ciarams87
Automate scale test
3d61535 
Problem:
Non-functional scale test needs to be run manually.

Solution:
- Automate scale test.
- Use in-cluster Prometheus to collect CPU, memory and NGF metrics.
- Use Kubernetes API server to get NGF logs.

For development and troubleshooting, it is possible to run scale test
locally in Kind cluster. However, it is necessary to bring down
the number of HTTPRoutes to 50 or less (roughly).

Testing:
- Ran this test locally with 64 listeners, 50 routes and 50 upstreams
  with NGINX OSS.
- Ran this test on GKE with the default configuration with NGINX OSS.

Out of scope: ensuring this test runs successfully via GitHub pipeline.

Closes nginxinc#1368

Largely based on work by Ciara in
nginxinc#1804

Co-authored-by: Ciara Stacke <[email protected]>
@pleshakov pleshakov mentioned this issue May 2, 2024
Merged
6 tasks
@pleshakov pleshakov moved this from 🏗 In Progress to 👀 In Review in NGINX Gateway Fabric May 3, 2024
pleshakov added a commit to pleshakov/nginx-gateway-fabric that referenced this issue May 16, 2024
@pleshakov @ciarams87
Automate scale test
e216554 
Problem:
Non-functional scale test needs to be run manually.

Solution:
- Automate scale test.
- Use in-cluster Prometheus to collect CPU, memory and NGF metrics.
- Use Kubernetes API server to get NGF logs.

For development and troubleshooting, it is possible to run scale test
locally in Kind cluster. However, it is necessary to bring down
the number of HTTPRoutes to 50 or less (roughly).

Testing:
- Ran this test locally with 64 listeners, 50 routes and 50 upstreams
  with NGINX OSS.
- Ran this test on GKE with the default configuration with NGINX OSS.

Out of scope: ensuring this test runs successfully via GitHub pipeline.

Closes nginxinc#1368

Largely based on work by Ciara in
nginxinc#1804

Co-authored-by: Ciara Stacke <[email protected]>
pleshakov added a commit to pleshakov/nginx-gateway-fabric that referenced this issue May 17, 2024
@pleshakov @ciarams87
Automate scale test
f8a9d47 
Problem:
Non-functional scale test needs to be run manually.

Solution:
- Automate scale test.
- Use in-cluster Prometheus to collect CPU, memory and NGF metrics.
- Use Kubernetes API server to get NGF logs.

For development and troubleshooting, it is possible to run scale test
locally in Kind cluster. However, it is necessary to bring down
the number of HTTPRoutes to 50 or less (roughly).

Testing:
- Ran this test locally with 64 listeners, 50 routes and 50 upstreams
  with NGINX OSS.
- Ran this test on GKE with the default configuration with NGINX OSS.

Out of scope: ensuring this test runs successfully via GitHub pipeline.

Closes nginxinc#1368

Largely based on work by Ciara in
nginxinc#1804

Co-authored-by: Ciara Stacke <[email protected]>
pleshakov added a commit that referenced this issue May 17, 2024
@pleshakov @ciarams87
Automate scale test (#1926)
ee4319c 
Problem:
Non-functional scale test needs to be run manually.

Solution:
- Automate scale test.
- Use in-cluster Prometheus to collect CPU, memory and NGF metrics.
- Use Kubernetes API server to get NGF logs.

For development and troubleshooting, it is possible to run scale test
locally in Kind cluster. However, it is necessary to bring down
the number of HTTPRoutes to 50 or less (roughly).

Testing:
- Ran this test locally with 64 listeners, 50 routes and 50 upstreams
  with NGINX OSS.
- Ran this test locally with 64 listeners, 10 routes and 10 upstreams with NGINX Plus.
- Ran this test on GKE with the default configuration with NGINX OSS.

Out of scope: ensuring this test runs successfully via GitHub pipeline.

Closes #1368

Largely based on work by Ciara in
#1804

Co-authored-by: Ciara Stacke <[email protected]>
@github-project-automation github-project-automation bot moved this from 👀 In Review to ✅ Done in NGINX Gateway Fabric May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pleshakov pleshakov

Labels
refined Requirements are refined and the issue is ready to be implemented. size/medium Estimated to be completed within a week tests Pull requests that update tests
Projects
NGINX Gateway Fabric
Archived in project
Milestone
v1.3.0
Development

Successfully merging a pull request may close this issue.

Automate scale test pleshakov/nginx-gateway-fabric
4 participants
@pleshakov @mpstefan @sjberman @ciarams87