roachtest: introduce multi tenant TPCH test #87084
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit unexports a couple of registration methods of the roachtests and then orders all such method lexicographically in the registry. Release justification: test-only change. Release note: None
|
This change is |
yuzefovich
force-pushed
the
tpch-multi-tenant
branch
from
b1852d6 to
b1b1a38
Compare
This commit introduces a new roachtest that runs TPCH benchmark on a single node in a single-tenant deployment first followed by another run in a multi-tenant deployment with a single SQL instance. It refactors the tpchvec roachtest a bit to extract a couple of helper methods for performing the latency analysis. In particular, it removes `queriesToRun` parameter since all variants now run all 22 queries. I ran into some difficulties around managing the certificates in different deployment models, so the test is structured that first the single-node cluster is used in a single-tenant deployment model, and then - after running all 22 queries - a tenant with a single SQL instance is created and all queries are run within the tenant. Here is the comparison of averages over 10 runs of single-tenant vs multi-tenant: ``` Q1: 6.34s 13.35s 110.39% Q2: 0.22s 0.49s 122.07% Q3: 4.88s 8.04s 64.67% Q4: 1.51s 4.75s 213.67% Q5: 2.33s 11.69s 400.90% Q6: 5.51s 35.49s 543.89% Q7: 5.75s 24.08s 318.42% Q8: 0.85s 3.82s 348.47% Q9: 7.34s 28.37s 286.25% Q10: 1.99s 5.00s 150.93% Q11: 0.55s 1.92s 249.00% Q12: 6.02s 34.76s 477.05% Q13: 1.88s 3.76s 100.00% Q14: 0.64s 1.10s 73.11% Q15: 3.33s 16.80s 404.23% Q16: 0.88s 1.29s 47.66% Q17: 0.24s 0.60s 145.49% Q18: 7.75s 30.13s 288.48% Q19: 5.20s 13.08s 151.69% Q20: 12.66s 55.30s 336.92% Q21: 6.98s 24.50s 250.77% Q22: 0.62s 1.17s 90.26% ``` Release justification: test-only change. Release note: None
yuzefovich
force-pushed
the
tpch-multi-tenant
branch
from
b1b1a38 to
26881f4
Compare
cucaroach
approved these changes
Sep 1, 2022
There was a problem hiding this comment.
Reviewed 3 of 3 files at r1, 8 of 8 files at r2, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @yuzefovich)
|
TFTR! bors r+ |
|
Build failed (retrying...): |
|
Build succeeded: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
roachtest: sort tests lexicographically
This commit unexports a couple of registration methods of the roachtests
and then orders all such method lexicographically in the registry.
Release justification: test-only change.
Release note: None
roachtest: introduce multi tenant TPCH test
This commit introduces a new roachtest that runs TPCH benchmark on
a single node in a single-tenant deployment first followed by another
run in a multi-tenant deployment with a single SQL instance. It
refactors the tpchvec roachtest a bit to extract a couple of helper
methods for performing the latency analysis. In particular, it removes
queriesToRunparameter since all variants now run all 22 queries.I ran into some difficulties around managing the certificates in
different deployment models, so the test is structured that first the
single-node cluster is used in a single-tenant deployment model, and
then - after running all 22 queries - a tenant with a single SQL
instance is created and all queries are run within the tenant.
Here is the comparison of averages over 10 runs of single-tenant vs
multi-tenant:
Fixes: #71528.
Release justification: test-only change.
Release note: None