Loadtest producer consumer various fixes #1227

philipsu522 · 2024-01-10T23:15:49Z

Describe your changes and provide context

A few more fixes after noticing some performance issues:

only numProducer accounts were used, limiting the throughput
parallelize loading keys to make startup faster
increase numProducers
add a buffer for highTps tests to allow producers to populate the queue
fix hanging threads

Testing performed to validate your change

ran against loadtest cluster:

codecov · 2024-01-10T23:21:29Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (e696486) 63.44% compared to head (0ea591c) 63.43%.
Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1227      +/-   ##
==========================================
- Coverage   63.44%   63.43%   -0.02%     
==========================================
  Files         257      257              
  Lines       16739    16739              
==========================================
- Hits        10620    10618       -2     
  Misses       5613     5613              
- Partials      506      508       +2

Files	Coverage Δ
app/app.go	`77.86% <ø> (ø)`
app/upgrades.go	`40.74% <ø> (ø)`

... and 4 files with indirect coverage changes

loadtest/loadtest_client.go

loadtest/main.go

@@ -109,7 +109,7 @@
 	}

 	fmt.Printf("Starting loadtest consumers\n")
-	go client.SendTxs(txQueue, done, &sentCount, int(config.TargetTps))
+	go client.SendTxs(txQueue, done, &sentCount, int(config.TargetTps), &wg)


loadtest/loadtest_client.go

app/app.go

loadtest/main.go

loadtest/loadtest_client.go

+			go func(tx []byte) {
+				localCtx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
+				defer cancel()
+				defer wg.Done()
+				defer sem.Release(1)
+
+				if err := rateLimiter.Wait(localCtx); err != nil {
+					return
+				}
+				SendTx(tx, typestx.BroadcastMode_BROADCAST_MODE_BLOCK, false, *c, sentCount)
+			}(tx)


loadtest/main.go

 	for i := 0; i < numProducers; i++ {
 		wg.Add(1)
-		go client.BuildTxs(txQueue, i, &wg, done, &producedCount)
+		go client.BuildTxs(txQueue, i, keys, &wg, done, &producedCount)


loadtest/sign.go

+		go func(i int, fileName string) {
+			defer wg.Done()
+			key := sc.GetKey(fmt.Sprint(i), "test", filepath.Join(userHomeDir, "test_accounts", fileName))
+			keysChan <- key
+		}(i, file.Name())


* Initial commit for producer/consumer loadtest client (#1190) * Initial commit for producer/consumer loadtest client * update sei-cosmos * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * rm rounds * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * finalize * debug * debug * debug * debug * debug * debug * debug * debug * bump cosmos * gofmt * debug * debug * Bump version to v3.6.1 (#1222) * Bump version to v3.6.0 * Fix empty branch * Bump cosmos to fix upgrade migration * Update version * Fix changelogs readme * Bump tendermint version * Fix version * Add 3.6.1 upgrade handler * Supress lint * Add migration process readme for SeiDB (#1221) * Add seidb migration steps * Add migration process for SeiDB * Fix * Update seidb_migration.md * Loadtest producer consumer various fixes (#1227) * debug * debug * debug * set default broadcast to sync * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * grpc * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * lint * int * Add TransferFrom method to contract and bindings * Lt client fix (#1260) * Fix account loading in loadtest client * Fix account loading in loadtest client * debug * debug * debug * debug * debug * debug * debug * debug * update * base * base with error * comments * fix bug * old additions * revert merge * pre merge * issues * add tests for transfer and transferFrom * add approve functions * add send_from --------- Co-authored-by: Philip Su <[email protected]> Co-authored-by: Yiming Zang <[email protected]> Co-authored-by: Mingjun <[email protected]>

* Initial commit for producer/consumer loadtest client (#1190) * Initial commit for producer/consumer loadtest client * update sei-cosmos * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * rm rounds * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * finalize * debug * debug * debug * debug * debug * debug * debug * debug * bump cosmos * gofmt * debug * debug * Bump version to v3.6.1 (#1222) * Bump version to v3.6.0 * Fix empty branch * Bump cosmos to fix upgrade migration * Update version * Fix changelogs readme * Bump tendermint version * Fix version * Add 3.6.1 upgrade handler * Supress lint * Add migration process readme for SeiDB (#1221) * Add seidb migration steps * Add migration process for SeiDB * Fix * Update seidb_migration.md * Loadtest producer consumer various fixes (#1227) * debug * debug * debug * set default broadcast to sync * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * grpc * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * debug * lint * int * Lt client fix (#1260) * Fix account loading in loadtest client * Fix account loading in loadtest client * debug * debug * debug * debug * debug * debug * debug * debug * update * SeiDB State Store Integration Tests (#1242) * SeiDB State Store Integration Tests * Update to differences * Update num contracts/denoms + jq * change length * reduce sleeps * Check reverse * logging * Update block height logging * update seid command * Update * remove logging * Update to branch * Update bank tests * Point to latest seidb --------- Co-authored-by: kbhat1 <[email protected]> Co-authored-by: kbhat1 <[email protected]> * Update synchronous execution to set tx indices properly (#1283) * fix debug iavl dump (#1287) * Keep v prefix for seid version (#1311) keep v prefix for seid version * Release v3.6.3 (#1317) * [SeiDB] Add integration test for historical bank send (#1329) * Add integration test for historical bank send * Fix go mod * Ignore error for vote already exist (#1325) * Ignore error for vote already exist * Fix go mod * Add error log --------- Co-authored-by: Philip Su <[email protected]> Co-authored-by: Yiming Zang <[email protected]> Co-authored-by: Kartik Bhat <[email protected]> Co-authored-by: kbhat1 <[email protected]> Co-authored-by: kbhat1 <[email protected]> Co-authored-by: Uday Patil <[email protected]>