Skip to content

Commit

Permalink
[LoadTest] Revamp load test suite (#1002)
Browse files Browse the repository at this point in the history
## Summary

This pull request includes several updates to the load-testing
configuration and test files, as well as improvements to the relay
stress test suite. The most important changes include updating funding
account addresses, modifying gateway configurations, and refactoring
various test functions for better accuracy.

## Type of change

Select one or more from the following:

- [ ] New feature, functionality or library
- [ ] Consensus breaking; add the `consensus-breaking` label if so. See
#791 for details
- [x] Bug fix
- [ ] Code health or cleanup
- [ ] Documentation
- [ ] Other (specify)

## Testing

- [x] **Unit Tests**: `make go_develop_and_test`
- [x] **LocalNet E2E Tests**: `make test_e2e`
- [ ] **DevNet E2E Tests**: Add the `devnet-test-e2e` label to the PR.

## Sanity Checklist

- [x] I have tested my changes using the available tooling
- [ ] I have commented my code
- [x] I have performed a self-review of my own code; both comments &
source code
- [ ] I create and reference any new tickets, if applicable
- [ ] I have left TODOs throughout the codebase, if applicable
  • Loading branch information
red-0ne authored Jan 10, 2025
1 parent 92fbd03 commit 6680d3e
Show file tree
Hide file tree
Showing 19 changed files with 553 additions and 365 deletions.
3 changes: 3 additions & 0 deletions Tiltfile
Original file line number Diff line number Diff line change
Expand Up @@ -320,6 +320,9 @@ for x in range(localnet_config["path_gateways"]["count"]):
"--set=metrics.serviceMonitor.enabled=" + str(localnet_config["observability"]["enabled"]),
"--set=path.mountConfigMaps[0].name=path-config-" + str(actor_number),
"--set=path.mountConfigMaps[0].mountPath=/app/config/",
"--set=fullnameOverride=path" + str(actor_number),
"--set=nameOverride=path" + str(actor_number),
"--set=global.serviceAccount.name=path" + str(actor_number),
]

if localnet_config["path_local_repo"]["enabled"]:
Expand Down
10 changes: 7 additions & 3 deletions load-testing/config/load_test_manifest_reader.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ type LoadTestManifestYAML struct {
// IsEphemeralChain is a flag that indicates whether the test is expected to be
// run on LocalNet or long-living remote chain (i.e. TestNet/DevNet).
IsEphemeralChain bool `yaml:"is_ephemeral_chain"`
TestNetNode string `yaml:"testnet_node"`
RPCNode string `yaml:"rpc_node"`
ServiceId string `yaml:"service_id"`
Suppliers []ProvisionedActorConfig `yaml:"suppliers"`
Gateways []ProvisionedActorConfig `yaml:"gateways"`
Expand Down Expand Up @@ -67,6 +67,10 @@ func validatedEphemeralChainManifest(manifest *LoadTestManifestYAML) (*LoadTestM
return nil, ErrEphemeralChainLoadTestInvalidManifest.Wrap("empty funding account address")
}

if len(manifest.RPCNode) == 0 {
return nil, ErrEphemeralChainLoadTestInvalidManifest.Wrap("empty rpc node url")
}

for _, gateway := range manifest.Gateways {
if len(gateway.Address) == 0 {
return nil, ErrEphemeralChainLoadTestInvalidManifest.Wrap("empty gateway address")
Expand Down Expand Up @@ -107,8 +111,8 @@ func validatedNonEphemeralChainManifest(manifest *LoadTestManifestYAML) (*LoadTe
return nil, ErrNonEphemeralChainLoadTestInvalidManifest.Wrap("suppliers entry forbidden")
}

if len(manifest.TestNetNode) == 0 {
return nil, ErrNonEphemeralChainLoadTestInvalidManifest.Wrap("empty testnet node url")
if len(manifest.RPCNode) == 0 {
return nil, ErrNonEphemeralChainLoadTestInvalidManifest.Wrap("empty rpc node url")
}

if len(manifest.ServiceId) == 0 {
Expand Down
8 changes: 4 additions & 4 deletions load-testing/loadtest_manifest_example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@
# It is intended to target a remote environment, such as a devnet or testnet.
is_ephemeral_chain: false

# testnet_node is the URL of the node that the load test will use to query the
# rpc_node is the URL of the RPC node that the load test will use to query the
# chain and submit transactions.
testnet_node: https://devnet-sophon-validator-rpc.poktroll.com
rpc_node: https://devnet-sophon-validator-rpc.poktroll.com

# The service ID to request relays from.
service_id: "anvil"

# The address of the account that will be used to fund the application accounts
# so that they can stake on the network.
funding_account_address: pokt1awtlw5sjmw2f5lgj8ekdkaqezphgz88rdk93sk # address for faucet account
# so that they can stake on the local network.
funding_account_address: pokt1eeeksh2tvkh7wzmfrljnhw4wrhs55lcuvmekkw # address for faucet account

# In non-ephemeral chains, the gateways are identified by their address.
gateways:
Expand Down
14 changes: 9 additions & 5 deletions load-testing/loadtest_manifest_localnet.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,16 @@

is_ephemeral_chain: true # This should be `true` for LocalNet as it is an ephemeral network

# rpc_node is the URL of the RPC node that the load test will use to query the
# chain and submit transactions.
rpc_node: http://localhost:26657

# The service ID to use for the load test.
service_id: anvil

# The address of the account that will be used to fund the application,
# gateway and supplier accounts so that they can stake on the network.
funding_account_address: pokt1awtlw5sjmw2f5lgj8ekdkaqezphgz88rdk93sk # address for faucet account
# gateway and supplier accounts so that they can stake on the local network.
funding_account_address: pokt1eeeksh2tvkh7wzmfrljnhw4wrhs55lcuvmekkw # address for faucet account

# List of pre-provisioned suppliers used for load testing.
# These suppliers will be progressively staked during the load test, according
Expand Down Expand Up @@ -48,12 +52,12 @@ gateways:

# Gateway 1; http://localhost:10350/r/gateway1/overview
- address: pokt15vzxjqklzjtlz7lahe8z2dfe9nm5vxwwmscne4
exposed_url: http://anvil.localhost/v1:3000 # The gateway url that the user sends relays to (e.g. curl)
exposed_url: http://localhost:3000/v1/ # The gateway url that the user sends relays to (e.g. curl)

# Gateway 2; http://localhost:10350/r/gateway2/overview
- address: pokt15w3fhfyc0lttv7r585e2ncpf6t2kl9uh8rsnyz
exposed_url: http://anvil.localhost/v1:3001
exposed_url: http://localhost:3001/v1/

# Gateway 3; http://localhost:10350/r/gateway3/overview
- address: pokt1zhmkkd0rh788mc9prfq0m2h88t9ge0j83gnxya
exposed_url: http://anvil.localhost/v1:3002
exposed_url: http://localhost:3002/v1/
14 changes: 9 additions & 5 deletions load-testing/loadtest_manifest_localnet_single_supplier.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,16 @@

is_ephemeral_chain: true # This should be `true` for LocalNet as it is an ephemeral network

# rpc_node is the URL of the RPC node that the load test will use to query the
# chain and submit transactions.
rpc_node: http://localhost:26657

# The service ID to use for the load test.
service_id: anvil

# The address of the account that will be used to fund the application,
# gateway and supplier accounts so that they can stake on the network.
funding_account_address: pokt1awtlw5sjmw2f5lgj8ekdkaqezphgz88rdk93sk # address for faucet account
# gateway and supplier accounts so that they can stake on the local network.
funding_account_address: pokt1eeeksh2tvkh7wzmfrljnhw4wrhs55lcuvmekkw # address for faucet account

# List of pre-provisioned suppliers used for load testing.
# These suppliers will be progressively staked during the load test, according
Expand Down Expand Up @@ -40,12 +44,12 @@ gateways:

# Gateway 1; http://localhost:10350/r/gateway1/overview
- address: pokt15vzxjqklzjtlz7lahe8z2dfe9nm5vxwwmscne4
exposed_url: http://anvil.localhost/v1:3000 # The gateway url that the user sends relays to (e.g. curl)
exposed_url: http://localhost:3000/v1/ # The gateway url that the user sends relays to (e.g. curl)

# Gateway 2; http://localhost:10350/r/gateway2/overview
- address: pokt15w3fhfyc0lttv7r585e2ncpf6t2kl9uh8rsnyz
exposed_url: http://anvil.localhost/v1:3001
exposed_url: http://localhost:3001/v1/

# Gateway 3; http://localhost:10350/r/gateway3/overview
- address: pokt1zhmkkd0rh788mc9prfq0m2h88t9ge0j83gnxya
exposed_url: http://anvil.localhost/v1:3002
exposed_url: http://localhost:3002/v1/
10 changes: 9 additions & 1 deletion load-testing/tests/relays_stress.feature
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,12 @@ Feature: Loading gateway server with relays
| gateway | 1 | 10 | 3 |
| supplier | 1 | 10 | 3 |
When a load of concurrent relay requests are sent from the applications
Then the correct pairs count of claim and proof messages should be committed on-chain
Then the number of failed relay requests is "0"
# TODO_FOLLOWUP(@red-0ne): Implement the following steps
# Then "0" over servicing events are observed
# And "0" slashing events are observed
# And "0" expired claim events are observed
# And there is as many reimbursement requests as the number of settled claims
# And the number of claims submitted and claims settled is the same
# And the number of proofs submitted and proofs required is the same
# And the actors onchain balances are as expected
Loading

0 comments on commit 6680d3e

Please sign in to comment.