Skip to content
This repository has been archived by the owner on Jul 11, 2023. It is now read-only.

ref(test): Improve e2e test framework using Fortio as load test tool #4631

Merged
merged 1 commit into from
Apr 20, 2022

Conversation

allenlsy
Copy link
Contributor

@allenlsy allenlsy commented Apr 5, 2022

Description:

Below is a list of services in the e2e tests that are used to test different web request protocols:

  • HTTP: curl
  • TCP: tcp-echo-server
  • gRPC: grpcurl, grpcbin

They are right now testing in single request mode. As our project develops, we need to a more customizable request sending solution to test various scenarios.

Fortio is a load testing tool that comes with both the server (web UI included) and the load generating client. They are packaged in a single Docker image.

In this PR:

  • Using Fortio, e2e test framework is improved by supporting simple load test in HTTP, TCP and gRPC(insecure) protocols. It exposes some common load test configurations such as test duration, number of calls and connections etc.
  • Migrated some e2e tests to use Fortio for demonstration and feature verification purposes. It's not the required to migrate all e2e tests to Fortio at this moment. These test cases are updated with Fortio being the Docker image of both client and server sides.

Partially resolves #4621 .

TODO after this PR:

  • Secured gRPC request and response is not implemented in this PR. It requires providing Fortio server with certificates in the test.
  • Fortio does not have a Windows compatible Docker image. e2e test in Windows is not supported yet

Testing done:

✅ Updated unit test

Affected area:

Functional Area
New Functionality [ ]
CI System [ ]
CLI Tool [ ]
Certificate Management [ ]
Control Plane [ ]
Demo [ ]
Documentation [ ]
Egress [ ]
Ingress [ ]
Install [ ]
Networking [ ]
Observability [ ]
Performance [ ]
SMI Policy [ ]
Security [ ]
Sidecar Injection [ ]
Tests [x]
Upgrade [ ]
Other [ ]

Please answer the following questions with yes/no.

  1. Does this change contain code from or inspired by another project?
    • Did you notify the maintainers and provide attribution?

No

  1. Is this a breaking change?

  2. Has documentation corresponding to this change been updated in the osm-docs repo (if applicable)?

@codecov-commenter
Copy link

codecov-commenter commented Apr 5, 2022

Codecov Report

Merging #4631 (07773d4) into main (30e5362) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #4631      +/-   ##
==========================================
+ Coverage   68.97%   68.99%   +0.02%     
==========================================
  Files         213      213              
  Lines       14861    14861              
==========================================
+ Hits        10250    10253       +3     
+ Misses       4558     4555       -3     
  Partials       53       53              
Flag Coverage Δ
unittests 68.99% <ø> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pkg/messaging/workqueue.go 100.00% <0.00%> (+10.71%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 30e5362...07773d4. Read the comment docs.

@allenlsy allenlsy added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 6, 2022
@allenlsy allenlsy force-pushed the e2e-fortio branch 27 times, most recently from ef850b5 to b1b3a60 Compare April 8, 2022 07:03
@allenlsy allenlsy force-pushed the e2e-fortio branch 3 times, most recently from 6a15408 to 9e37950 Compare April 11, 2022 17:22
@allenlsy allenlsy marked this pull request as ready for review April 11, 2022 17:30
@allenlsy allenlsy marked this pull request as draft April 11, 2022 17:44
@allenlsy allenlsy force-pushed the e2e-fortio branch 5 times, most recently from 2cabc53 to 5d867b2 Compare April 11, 2022 23:11
@allenlsy allenlsy marked this pull request as ready for review April 11, 2022 23:35
@allenlsy allenlsy changed the title ref(test): migrate to fortio image from curl in e2e test ref(test): Improve e2e test framework using Fortio as load test tool Apr 12, 2022
@allenlsy allenlsy removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 12, 2022
@allenlsy allenlsy marked this pull request as draft April 12, 2022 17:53
@allenlsy allenlsy force-pushed the e2e-fortio branch 2 times, most recently from e6f6929 to 777bc72 Compare April 12, 2022 18:02
* Improve e2e test framework to support running Fortio load tests
* Migrate testing application images to Fortio client & server
* Update some e2e test case for Fortio feature verification

Signed-off-by: Allen Leigh <[email protected]>
@allenlsy allenlsy marked this pull request as ready for review April 12, 2022 18:26
Copy link
Member

@shashankram shashankram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@allenlsy thanks for this significant enhancement. Great to see all the e2e tests pass.

@trstringer trstringer merged commit cf1395e into openservicemesh:main Apr 20, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add load testing support in e2e framework
4 participants