Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preform performance and stability test on different client implementation #1421

Closed
mzitnik opened this issue Aug 14, 2023 · 8 comments · Fixed by #1439
Closed

Preform performance and stability test on different client implementation #1421

mzitnik opened this issue Aug 14, 2023 · 8 comments · Fixed by #1439

Comments

@mzitnik
Copy link
Contributor

mzitnik commented Aug 14, 2023

In clickhouse-http-client we support different Http Connection providers

  • APACHE_HTTP_CLIENT
  • HTTP_CLIENT
  • HTTP_URL_CONNECTION

We would like to test the performance and stability of those http connection providers

Verticals that we need to test

  • inserts
  • simple selects select
@mzitnik mzitnik self-assigned this Aug 14, 2023
@zhicwu
Copy link
Contributor

zhicwu commented Aug 15, 2023

Just some random ideas:

  • consider both small and large queries
  • I used to test in LAN but we try WAN as well(like the socket timeout issue)
  • make sure both Java client and JDBC are tested - JDBC is synchronous and has more overheads
  • not sure if okhttp is faster or not but we may want to check that later too

@mshustov mshustov added this to the v0.4.7 milestone Aug 15, 2023
@zhicwu zhicwu linked a pull request Sep 5, 2023 that will close this issue
3 tasks
@mzitnik
Copy link
Contributor Author

mzitnik commented Sep 7, 2023

@zhicwu is working to optimize it.

We are inserting 50K rows.

using DEFAULT_BUFFER_SIZE = 4096

Http Client - over wifi Query Insert - (20 byte) Insert - (64 byte) Insert - (124 byte)
Apache Http Client 70ms 271ms 564ms 960ms
HttpUrlConnection 211ms 770ms 1058ms 1310ms
Ratio 3X 2.8X 1.8X 1.3X

I will also run a test over LAN.

@mzitnik
Copy link
Contributor Author

mzitnik commented Sep 7, 2023

We are inserting 50K rows.

using BUFFER_SIZE = 8129

Http Client - over wifi Query Insert - (124 byte) Insert - (256 byte) Insert - (512 byte)
Apache Http Client 70ms 899ms 1639ms 3480
HttpUrlConnection 218ms 1429ms 2527ms 3862
Ratio 3X 1.5X 1.5X 1.1X

@zhicwu
Copy link
Contributor

zhicwu commented Sep 7, 2023

Is it single thread or multiple? And what's table structure and queries for the testing?

If the size here is bytes of an uncompressed row, then we're testing really small insert. It's just my opinion, since we usually create table with tens or hundreds of columns on ClickHouse, it might be better to try 1KB - 10KB row for small insert, and 100KB or more for large ones. Alternatively, we can test single-column table as in clickhouse-benchmark, because data type matters.

@mzitnik
Copy link
Contributor Author

mzitnik commented Sep 10, 2023

Planning to increase the row size in the test, we are inserting a batch of 50K rows (I will take a look on the benchmarking project )
@zhicwu what is the batch size in insert testing

@zhicwu
Copy link
Contributor

zhicwu commented Sep 10, 2023

what is the batch size in insert testing

It defaults to 5k but can be changed as needed.

private final int BATCH_SIZE = Integer.parseInt(System.getProperty("batchSize", "5000"));

@mzitnik
Copy link
Contributor Author

mzitnik commented Sep 11, 2023

Settings
prop.setProperty("buffer_size", "8192");
prop.setProperty("socket_rcvbuf", "804800");
prop.setProperty("socket_sndbuf", "804800");

Http Client - LAN Insert - (256 byte) Insert - (512 byte) Insert - (1024 byte) Insert - (2048 byte)
Apache Http Client 1853 2964 5323 10543
HttpUrlConnection 3153 5580 9634 14830
Ratio 1.7X 1.8X 1.8X 1.4X

@mzitnik
Copy link
Contributor Author

mzitnik commented Sep 11, 2023

Settings
prop.setProperty("buffer_size", "8192");

Http Client - LAN Insert - (256 byte) Insert - (512 byte) Insert - (1024 byte) Insert - (2048 byte)
Apache Http Client 1873 2773 5126 8497
HttpUrlConnection 2704 4843 8161 14397
Ratio 1.5X 1.7X 1.6X 1.7X

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants