Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#21963] YSQL: New read restart workloads. #98

Merged
merged 1 commit into from
Oct 1, 2024

Conversation

pao214
Copy link
Contributor

@pao214 pao214 commented Sep 22, 2024

New Workloads

Workload Short Description
SqlEventCounter Track counters for events. Eg. clicks, views, etc.
SqlBankTransfers Transfer money between accounts and verify balances checkout.
SqlWarehouseStock New Orders decrement stock and a concurrent process restocks low stock items.
SqlMessageQueue Stock levels is disseminated by the publisher and the subscriber accumulates the total stock.
SqlConsistentHashing Find the node closest to key using consistent hashing in a dynamic node configuration.

Benchmark: Restart Read Requests

Workload Wall Clock Clockbound Clock
SqlEventCounter ~8000 ~110
SqlBankTransfers ~500 ~3
SqlWarehouseStock ~150 ~8
SqlMessageQueue ~50 ~5
SqlConsistentHashing ~1500 ~10

Security

Change the prometheus URL to http since we start yugabyted without TLS by default. Security is not the primary concern of this change.


Cluster Setup

  1. Create 3 AWS nodes on US N Viriginia with permissive networking.
  2. Copy/Download release tarball with LTO enabled onto the machines.
  3. Extract the tarball and cd into the directory.

For wall clock configuration,

./bin/yugabyted start [--join <first_node_private_ip>]

For clockbound clock configuration,

./bin/yugabyted configure clockbound
./bin/yugabyted start --enhance_time_sync_via_clockbound

Run each sample app for 3 times for benchmarking and confidence in the metrics.

@pao214 pao214 changed the title [#21963] YSQL: More read restart workloads. [#21963] YSQL: New read restart workloads. Sep 23, 2024
@pao214 pao214 force-pushed the new_workloads branch 5 times, most recently from 4d25cc7 to c50eabe Compare September 25, 2024 21:49
public long doRead() {
try (Connection connection = getPostgresConnection();
Statement statement = connection.createStatement()) {
statement.executeUpdate(String.format(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe in doRead workload do a read and check consistency? And this logic can be moved in doWrite method and activate on a coin flip?

Copy link
Contributor Author

@pao214 pao214 Sep 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I check if there are any stale reads now. The name doRead is unfortunate but it is better to report latency and throughput separately for different operations. If we use a coin flip, then the throughput, latency numbers mix across different operations.

@pao214 pao214 force-pushed the new_workloads branch 2 times, most recently from 42e8e4a to 51f23ed Compare September 26, 2024 21:35
@pao214 pao214 requested review from qvad and pilshchikov September 26, 2024 21:37
@pao214 pao214 force-pushed the new_workloads branch 2 times, most recently from 5a7f4c2 to b0c20ed Compare September 27, 2024 19:03
@pao214 pao214 merged commit d3dcc04 into yugabyte:master Oct 1, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants