refactor(perf): use libp2p-request-response #3646

mxinden · 2023-03-20T10:18:28Z

Description

Rewrite libp2p-perf protocol using libp2p-request-response. Additionally adjust to latest conventions, e.g. the final JSON output.

Notes & open questions

Change checklist

I have performed a self-review of my own code
I have made corresponding changes to the documentation
I have added tests that prove my fix is effective or that my feature works
A changelog entry has been made in the appropriate crates

- Enforces backpressure on stream creation on QUIC - Note that this does not sequentialize stream negotiation as we are using V1Lazy As an aside, with and without `V1Lazy`: ``` [2023-03-20T09:31:21Z INFO perf_client] Starting: single connection requests per second benchmark. [2023-03-20T09:31:28Z INFO perf_client] Finsihed: sent 10000 1 bytes requests with 1 bytes response each within 6.28 s thus 1591.963270172488 req/s. ``` ``` [2023-03-20T09:34:03Z INFO perf_client] Starting: single connection requests per second benchmark. [2023-03-20T09:37:07Z INFO perf_client] Finsihed: sent 10000 1 bytes requests with 1 bytes response each within 183.90 s thus 54.37835611842429 req/s. ```

``` [2023-03-20T10:09:29Z INFO perf_client] Initiating performance tests with /dns4/libp2p-perf.max-inden.de/udp/4001/quic-v1 [2023-03-20T10:09:29Z INFO perf_client] Connection to /dns4/libp2p-perf.max-inden.de/udp/4001/quic-v1 established. [2023-03-20T10:09:29Z INFO perf_client] Start benchmark: single connection single channel throughput [2023-03-20T10:09:41Z INFO perf_client] Finished: sent 10.00 MiB in 7.60 s with 10.53 MiBit/s and received 10.00 MiB in 3.92 s with 20.41 MiBit/s [2023-03-20T10:09:41Z INFO perf_client] Start benchmark: single connection requests per second [2023-03-20T10:09:48Z INFO perf_client] Finished: sent 10000 1 bytes requests with 1 bytes response each within 6.93 s thus 1442.81 req/s ```

protocols/perf/src/bin/perf-client.rs

mxinden · 2023-03-20T13:52:03Z

Sample output:

➜  perf git:(perf) cargo run --bin perf-client -- --server-address /dns4/libp2p-perf.max-inden.de/udp/4001/quic-v1
[2023-03-20T13:50:02Z INFO  perf_client] Start benchmark: round-trip-time latency
[2023-03-20T13:50:38Z INFO  perf_client] Finished: 1000 pings
[2023-03-20T13:50:38Z INFO  perf_client] - 0.0325 s median
[2023-03-20T13:50:38Z INFO  perf_client] - 0.0419 s 95th percentile
    
[2023-03-20T13:50:38Z INFO  perf_client] Start benchmark: single connection single channel throughput
[2023-03-20T13:50:48Z INFO  perf_client] Finished: sent 10.00 MiB in 7.38 s and received 10.00 MiB in 3.21 s
[2023-03-20T13:50:48Z INFO  perf_client] - 10.84 MiBit/s up
[2023-03-20T13:50:48Z INFO  perf_client] - 24.90 MiBit/s down
    
[2023-03-20T13:50:48Z INFO  perf_client] Start benchmark: single connection parallel requests per second
[2023-03-20T13:50:54Z INFO  perf_client] Finished: sent 10000 1 bytes requests with 1 bytes response each within 5.54 s
[2023-03-20T13:50:54Z INFO  perf_client] - 1803.94 req/s
    
[2023-03-20T13:50:54Z INFO  perf_client] Start benchmark: sequential connections with single request per second
[2023-03-20T13:51:01Z INFO  perf_client] Finished: established 100 connections with one 1 bytes request and one 1 bytes response within 7.23 s
[2023-03-20T13:51:01Z INFO  perf_client] - 0.0384 s 95th percentile connection establishment
[2023-03-20T13:51:01Z INFO  perf_client] - 0.0752 s 95th percentile connection establishment + one request

mergify · 2023-03-20T14:08:04Z

This pull request has merge conflicts. Could you please resolve them @mxinden? 🙏

See libp2p/test-plans#159 for schema discussion.

mergify · 2023-04-13T16:40:27Z

This pull request has merge conflicts. Could you please resolve them @mxinden? 🙏

mergify · 2023-05-14T11:00:36Z

This pull request has merge conflicts. Could you please resolve them @mxinden? 🙏

No longer needed. Output of perf binaries is much smaller now.

mxinden · 2023-05-26T07:46:46Z

@thomaseizinger can you give this pull request a review? Note that with #3990 we can easily do (breaking) changes in the future. Given that and given that it is not a production-environment component, I don't think it requires a thorough review.

In my eyes it makes sense to merge at this point in order to not accumulate even more changes into a single pull request. As is, it is a useful component. That said, I do expect more changes to come.

protocols/perf/src/bin/perf.rs

Makes it easier to compare with http performance

mxinden · 2023-05-28T05:33:19Z

Switched from Noise to TLS. That was as simple as going from libp2p_noise::Config to libp2p_tls::Config. Thanks for pushing all the consistency efforts @thomaseizinger!

Now that libp2p/rust-libp2p#3646 is merged.

thomaseizinger · 2023-05-30T15:31:00Z

Switched from Noise to TLS. That was as simple as going from libp2p_noise::Config to libp2p_tls::Config. Thanks for pushing all the consistency efforts @thomaseizinger!

It is coming together!

mxinden added 4 commits March 20, 2023 10:37

Implement single connection requests per second benchmark

1c5384d

Implement connections per second benchmark

9dffaf7

thomaseizinger reviewed Mar 20, 2023

View reviewed changes

protocols/perf/src/bin/perf-client.rs Outdated Show resolved Hide resolved

mxinden added 2 commits March 20, 2023 14:43

Add ping latency benchmark

bff4279

Fix units

dfa1e57

Print benchmark output according to json schema

ed6dd3f

See libp2p/test-plans#159 for schema discussion.

mxinden mentioned this pull request Mar 24, 2023

Add perf dashboard libp2p/test-plans#159

Closed

mxinden changed the title ~~feat: Enforce single pending stream, req/s benchmark and colored output~~ feat: iterate on perf protocol Mar 24, 2023

mxinden added 5 commits April 5, 2023 08:37

Merge branch 'master' of https://github.com/libp2p/rust-libp2p into perf

6778a6f

Update data schema

bebb7e7

Introduce custom benchmark

a01ff2c

Fix forwarding upload and download bytes

8cef2d6

Wait for remote to close read side

095ceb6

mxinden mentioned this pull request Apr 11, 2023

feat(perf): add (provision, build, run) tooling libp2p/test-plans#163

Merged

mxinden and others added 3 commits April 12, 2023 19:23

Provide option to pass secret key seed

0019366

Follow go implementation entry point convention

2a68630

Change to tokio runtime

d3c3ad4

mxinden added 8 commits April 13, 2023 19:44

Fix DNS imports

285a175

Use nodelay don't reuse port

cc064f5

Increase read buffer

4cde5b5

Set TCP nodelay on server

3aec5e5

Merge server and client into single entry point

cef0f23

Run n times on same connection

66a28b3

Fix info log

da5cc28

Use request-response and adjust to new flags

90964ff

mxinden added 8 commits May 1, 2023 18:58

Debug log level

e984af6

request-response closes for us

59dd784

Bump timeouts

9d49f26

Log on info

613e684

Fix compilation

f515473

Remove n-times flag

59d8f3f

Merge branch 'master' of https://github.com/libp2p/rust-libp2p into perf

1d18761

Provide ip:port

08e4276

mxinden added 6 commits May 19, 2023 12:29

Remove schema

647d76c

No longer needed. Output of perf binaries is much smaller now.

Differentiate by up- and download time

c4c84e7

Remove Benchmark trait

fa6886c

Use serde camelCase

488f007

Log with millis to debug RTT

ec52392

Merge branch 'master' of https://github.com/libp2p/rust-libp2p into perf

15e3642

mxinden changed the title ~~feat: iterate on perf protocol~~ refactor(perf): use libp2p-request-response May 26, 2023

thomaseizinger approved these changes May 26, 2023

View reviewed changes

protocols/perf/src/bin/perf.rs Outdated Show resolved Hide resolved

protocols/perf/src/bin/perf.rs Outdated Show resolved Hide resolved

mxinden added 4 commits May 28, 2023 14:22

Use swarm to contruct server

e343e5a

Fix clippy

539eb3d

Merge branch 'master' of https://github.com/libp2p/rust-libp2p into perf

095f920

Use tls instead of noise

6bb18a5

Makes it easier to compare with http performance

mxinden added the send-it label May 28, 2023

mergify bot merged commit 92af0d1 into libp2p:master May 28, 2023

mxinden added a commit to libp2p/test-plans that referenced this pull request May 29, 2023

fix(rust-libp2p): update to latest master

5fa246e

Now that libp2p/rust-libp2p#3646 is merged.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(perf): use libp2p-request-response #3646

refactor(perf): use libp2p-request-response #3646

mxinden commented Mar 20, 2023 •

edited

Loading

mxinden commented Mar 20, 2023

mergify bot commented Mar 20, 2023

mergify bot commented Apr 13, 2023

mergify bot commented May 14, 2023

mxinden commented May 26, 2023

mxinden commented May 28, 2023

thomaseizinger commented May 30, 2023

refactor(perf): use libp2p-request-response #3646

refactor(perf): use libp2p-request-response #3646

Conversation

mxinden commented Mar 20, 2023 • edited Loading

Description

Notes & open questions

Change checklist

mxinden commented Mar 20, 2023

mergify bot commented Mar 20, 2023

mergify bot commented Apr 13, 2023

mergify bot commented May 14, 2023

mxinden commented May 26, 2023

mxinden commented May 28, 2023

thomaseizinger commented May 30, 2023

mxinden commented Mar 20, 2023 •

edited

Loading