Benchmarking GNES on Network Latency

This repository tracks the network latency over different GNES versions. As a part of CICD pipeline, this repo gets automatically updated when the GNES master is updated or a new GNES version is released.

Please don't change the content of this file manually, as it will be overwritten during the update anyway.

Experimental Setup

In this benchmark, we setup multiple workflows to represent typical pipelines in the everyday usage of GNES. We then do a "load testing" to determine the system's behavior under normal/peak conditions.

All experiments use gnes/gnes:{version}-alpine as the base image. All microservices are simplified using BaseRouter and BlockRouter.

Run Test

For the purpose of evaluating this benchmark in CICD pipeline, some environment

For example, to run the third test case on GNES version latest-alpine:

export GNES_IMG_TAG=latest-alpine
export GNES_BENCHMARK_ID=3

make pull && make build && make test d=500 b=10 s=1000000 && make clean

The client will generate 500 documents with the batch size of 10, which yields 50 requests in total. Each document has the size of 1MB. Hence each request is 10MB.

Due to the memory limit of our CD node (t2-micro, 1GB memory), we better keep the number of document in this way.

Explanation of the Table

Time units are in seconds, the smaller the better. Numbers are the best average over three runs.

Time-related metrics

roundtrip: the average latency in seconds for a request travel from Frontend and through the whole workflow and finally back to Frontend.
MB/s: megabyte per second (MB/s) is a unit of data transfer rate over the whole workflow.
f:send: the average latency in seconds between sending every two requests at the Frontend.
f:recv: the average latency in seconds between receiving every two requests at the Frontend.
f->r1:send: the average latency in seconds for Router1 receiving a request sent from Frontend.
r1->r2:send: the average latency in seconds for Router2 receiving a request sent from Router1 (or all Router1 from the last layer).
r2->f:send: the average latency in seconds for Frontend receiving a request sent from Router2 (or all Router2 from the last layer).

Meta information

version_vcs: corresponds to the gnes-ai/gnes@ commit's SHA hash.
version_tag: corresponds to the version tag of a GNES docker image.
timestamp_build: timestamp when the docker image was built.
timestamp_eval: timestamp when the benchmark was evaluated.

Table results are sorted by timestamp_build with the most recent build at first.

Case 1: Non-blocking Flow

The workflow is as follows:

The ideal roundtrip latency is 0. The smaller the better.

Result

version_vcs	roundtrip	MB/s	f:send	f:recv	f->r1:send	r1->r2:send	r2->f:send	timestamp_build	timestamp_eval	version_tag
`f40d46a`	0.197	2760	0.134	0.130	0.043	0.052	0.099	2019-11-06 11:52	2019-11-06 12:29	`latest-alpine`
`14cffe3`	0.199	2733	0.135	0.131	0.043	0.052	0.099	2019-11-06 11:13	2019-11-06 11:40	`latest-alpine`
`ca20463`	0.197	2760	0.134	0.131	0.043	0.052	0.099	2019-11-06 10:35	2019-11-06 11:02	`v0.0.47-alpine`
`5484af3`	0.200	2747	0.135	0.131	0.043	0.047	0.101	2019-11-06 09:52	2019-11-06 10:24	`latest-alpine`
`6687b68`	0.199	2754	0.135	0.131	0.043	0.052	0.099	2019-11-06 08:48	2019-11-06 09:35	`latest-alpine`
`b4d2c8c`	0.198	2760	0.134	0.131	0.043	0.046	0.098	2019-10-24 05:32	2019-10-24 05:58	`latest-alpine`
`5de329d`	0.202	2723	0.134	0.130	0.042	0.048	0.102	2019-10-23 04:33	2019-10-23 05:01	`latest-alpine`
`44a54be`	0.200	2730	0.135	0.131	0.042	0.048	0.101	2019-10-23 03:55	2019-10-23 04:23	`latest-alpine`
`01fe27b`	0.200	2736	0.135	0.131	0.043	0.052	0.100	2019-10-23 03:18	2019-10-23 03:45	`latest-alpine`
`388d1ba`	0.200	2738	0.134	0.130	0.043	0.046	0.101	2019-10-23 02:41	2019-10-23 03:08	`latest-alpine`
`69ea3fd`	0.199	2753	0.134	0.130	0.042	0.046	0.100	2019-10-21 14:16	2019-10-21 14:42	`latest-alpine`
`97efb59`	0.199	2732	0.134	0.130	0.043	0.053	0.100	2019-10-21 10:46	2019-10-21 11:13	`latest-alpine`
`da50be0`	0.195	2830	0.132	0.128	0.042	0.051	0.098	2019-10-21 04:07	2019-10-21 04:33	`latest-alpine`
`7865b45`	0.196	2786	0.132	0.128	0.042	0.051	0.098	2019-10-18 08:04	2019-10-18 08:30	`latest-alpine`
`8b43398`	0.198	2774	0.133	0.130	0.043	0.047	0.100	2019-10-17 11:28	2019-10-17 11:54	`latest-alpine`
`15b940d`	0.201	2730	0.135	0.130	0.044	0.046	0.101	2019-10-17 10:51	2019-10-17 11:17	`v0.0.46-alpine`
`5fdf67a`	0.198	2766	0.133	0.129	0.043	0.052	0.100	2019-10-17 10:14	2019-10-17 10:41	`latest-alpine`
`394a007`	0.199	2771	0.133	0.130	0.042	0.048	0.099	2019-10-17 03:53	2019-10-17 04:20	`latest-alpine`
`c35b27e`	0.195	2828	0.132	0.128	0.042	0.046	0.098	2019-10-17 03:16	2019-10-17 03:43	`latest-alpine`
`ca9ad83`	0.199	2741	0.133	0.129	0.043	0.045	0.100	2019-10-16 05:52	2019-10-16 06:18	`latest-alpine`
`a419c34`	0.200	2771	0.133	0.130	0.041	0.046	0.100	2019-10-16 03:08	2019-10-16 03:31	`latest-alpine`
`5676ab8`	0.199	2771	0.135	0.131	0.043	0.047	0.100	2019-10-15 08:48	2019-10-15 09:15	`latest-alpine`
`cfee124`	0.197	2789	0.133	0.130	0.042	0.048	0.099	2019-10-15 08:11	2019-10-15 08:38	`latest-alpine`
`b6837b1`	0.261	2002	0.134	0.130	0.040	0.052	0.164	2019-10-15 06:04	2019-10-15 06:33	`v0.0.45-alpine`
`2d398c1`	0.260	2005	0.134	0.130	0.040	0.052	0.164	2019-10-15 04:53	2019-10-15 05:20	`latest-alpine`
`0ac92e4`	0.257	2027	0.133	0.128	0.039	0.051	0.162	2019-10-14 11:50	2019-10-14 12:17	`latest-alpine`
`5583462`	0.259	2011	0.133	0.129	0.039	0.045	0.164	2019-10-14 09:07	2019-10-14 09:34	`latest-alpine`
`e3ab1aa`	0.260	2017	0.134	0.130	0.040	0.045	0.165	2019-10-12 05:41	2019-10-12 06:07	`latest-alpine`
`e64bc7a`	0.254	2049	0.131	0.127	0.039	0.051	0.161	2019-10-11 13:11	2019-10-11 13:38	`latest-alpine`
`72c6d8f`	0.258	2023	0.133	0.129	0.039	0.051	0.163	2019-10-11 08:08	2019-10-11 08:35	`latest-alpine`
`a5b40c1`	0.259	2012	0.133	0.129	0.040	0.052	0.164	2019-10-11 07:31	2019-10-11 07:58	`v0.0.44-alpine`
`8ca5ef0`	0.256	2035	0.132	0.127	0.040	0.051	0.161	2019-10-11 05:48	2019-10-11 06:15	`latest-alpine`
`f7e7791`	3.300	228	0.124	0.010	0.054	0.053	3.189	2019-10-11 04:20	2019-10-11 04:47	`latest-alpine`
`9095bfa`	3.295	229	0.124	0.010	0.054	0.052	3.184	2019-10-10 11:45	2019-10-10 12:12	`latest-alpine`
`1878564`	3.317	226	0.124	0.010	0.054	0.046	3.206	2019-10-10 11:00	2019-10-10 11:34	`latest-alpine`
`49150fb`	3.314	227	0.123	0.010	0.053	0.046	3.204	2019-10-10 04:51	2019-10-10 05:18	`latest-alpine`
`17f9287`	0.161	3148	0.127	0.125	0.032	0.051	0.069	2019-10-10 00:33	2019-10-10 01:00	`latest-alpine`
`3aab341`	0.161	3150	0.129	0.127	0.032	0.052	0.068	2019-10-09 10:15	2019-10-09 10:39	`latest-alpine`
`fc546f0`	0.160	3166	0.128	0.126	0.032	0.047	0.068	2019-09-30 09:41	2019-09-30 10:08	`v0.0.43-alpine`
`83f96d9`	0.160	3171	0.127	0.126	0.032	0.048	0.068	2019-09-29 12:34	2019-09-29 13:01	`latest-alpine`
`4dcf6d0`	0.159	3185	0.127	0.125	0.031	0.050	0.068	2019-09-29 11:57	2019-09-29 12:24	`latest-alpine`
`0bb0082`	0.160	3172	0.128	0.126	0.031	0.048	0.068	2019-09-29 11:23	2019-09-29 11:47	`latest-alpine`
`cb4e46a`	0.159	3182	0.127	0.126	0.031	0.047	0.068	2019-09-29 04:07	2019-09-29 04:34	`latest-alpine`
`a087626`	2.534	727	1.114	1.052	0.537	0.478	1.491	2019-09-27 11:05	2019-09-27 11:35	`latest-alpine`
`51837cf`	0.160	3166	0.128	0.126	0.031	0.050	0.067	2019-09-27 06:36	2019-09-27 10:44	`latest-alpine`
`51837cf`	0.155	3258	0.124	0.123	0.032	0.047	0.064	2019-09-27 06:36	2019-09-27 10:29	`latest-alpine`

Case 2: Blocking Flow

The workflow is as follows:

It simulates a pipeline with uneven workload, Router2 block the pipeline for 1s.

Hence, a naive synchronized pipeline will take 50s to finish 50 requests.

Result

version_vcs	roundtrip	MB/s	f:send	f:recv	f->r1:send	r1->r2:send	r2->f:send	timestamp_build	timestamp_eval	version_tag
`f40d46a`	24.207	43	0.096	1.036	0.039	23.119	0.037	2019-11-06 11:52	2019-11-06 12:32	`latest-alpine`
`14cffe3`	24.220	43	0.096	1.035	0.038	23.133	0.037	2019-11-06 11:13	2019-11-06 11:44	`latest-alpine`
`ca20463`	24.176	43	0.095	1.036	0.039	23.092	0.036	2019-11-06 10:35	2019-11-06 11:06	`v0.0.47-alpine`
`5484af3`	24.161	43	0.096	1.035	0.038	23.077	0.037	2019-11-06 09:52	2019-11-06 10:28	`latest-alpine`
`6687b68`	24.168	43	0.096	1.035	0.040	23.082	0.038	2019-11-06 08:48	2019-11-06 09:39	`latest-alpine`
`b4d2c8c`	24.216	43	0.096	1.035	0.040	23.129	0.038	2019-10-24 05:32	2019-10-24 06:02	`latest-alpine`
`5de329d`	24.226	43	0.096	1.036	0.039	23.139	0.037	2019-10-23 04:33	2019-10-23 05:04	`latest-alpine`
`44a54be`	24.177	43	0.096	1.035	0.041	23.088	0.036	2019-10-23 03:55	2019-10-23 04:27	`latest-alpine`
`01fe27b`	24.160	43	0.096	1.035	0.040	23.075	0.036	2019-10-23 03:18	2019-10-23 03:48	`latest-alpine`
`388d1ba`	24.213	43	0.097	1.035	0.038	23.121	0.038	2019-10-23 02:41	2019-10-23 03:11	`latest-alpine`
`69ea3fd`	24.165	43	0.096	1.035	0.037	23.081	0.035	2019-10-21 14:16	2019-10-21 14:46	`latest-alpine`
`97efb59`	24.134	43	0.096	1.035	0.040	23.047	0.036	2019-10-21 10:46	2019-10-21 11:16	`latest-alpine`
`da50be0`	24.209	43	0.096	1.035	0.039	23.116	0.037	2019-10-21 04:07	2019-10-21 04:37	`latest-alpine`
`7865b45`	24.180	43	0.096	1.035	0.040	23.094	0.035	2019-10-18 08:04	2019-10-18 08:33	`latest-alpine`
`8b43398`	24.212	43	0.095	1.035	0.039	23.117	0.038	2019-10-17 11:28	2019-10-17 11:58	`latest-alpine`
`15b940d`	24.164	43	0.095	1.035	0.038	23.080	0.037	2019-10-17 10:51	2019-10-17 11:21	`v0.0.46-alpine`
`5fdf67a`	24.161	43	0.095	1.035	0.039	23.076	0.036	2019-10-17 10:14	2019-10-17 10:44	`latest-alpine`
`394a007`	24.218	43	0.095	1.035	0.038	23.134	0.036	2019-10-17 03:53	2019-10-17 04:23	`latest-alpine`
`c35b27e`	24.208	43	0.096	1.035	0.039	23.121	0.035	2019-10-17 03:16	2019-10-17 03:46	`latest-alpine`
`ca9ad83`	24.230	43	0.095	1.035	0.038	23.150	0.034	2019-10-16 05:52	2019-10-16 06:22	`latest-alpine`
`a419c34`	24.173	43	0.096	1.035	0.040	23.087	0.037	2019-10-16 03:08	2019-10-16 03:34	`latest-alpine`
`5676ab8`	24.209	43	0.097	1.035	0.040	23.117	0.036	2019-10-15 08:48	2019-10-15 09:18	`latest-alpine`
`cfee124`	24.164	43	0.095	1.035	0.039	23.079	0.035	2019-10-15 08:11	2019-10-15 08:41	`latest-alpine`
`b6837b1`	24.171	43	0.095	1.036	0.041	23.084	0.037	2019-10-15 06:04	2019-10-15 06:36	`v0.0.45-alpine`
`2d398c1`	24.246	43	0.095	1.036	0.039	23.161	0.036	2019-10-15 04:53	2019-10-15 05:24	`latest-alpine`
`0ac92e4`	24.160	43	0.094	1.035	0.040	23.074	0.036	2019-10-14 11:50	2019-10-14 12:20	`latest-alpine`
`5583462`	24.193	43	0.094	1.036	0.039	23.106	0.037	2019-10-14 09:07	2019-10-14 09:37	`latest-alpine`
`e3ab1aa`	24.251	43	0.094	1.035	0.041	23.165	0.037	2019-10-12 05:41	2019-10-12 06:11	`latest-alpine`
`e64bc7a`	24.223	43	0.093	1.035	0.038	23.140	0.035	2019-10-11 13:11	2019-10-11 13:41	`latest-alpine`
`72c6d8f`	24.242	43	0.094	1.036	0.040	23.151	0.036	2019-10-11 08:08	2019-10-11 08:39	`latest-alpine`
`a5b40c1`	24.221	43	0.096	1.036	0.040	23.131	0.036	2019-10-11 07:31	2019-10-11 08:01	`v0.0.44-alpine`
`8ca5ef0`	24.227	43	0.096	1.036	0.040	23.135	0.036	2019-10-11 05:48	2019-10-11 06:18	`latest-alpine`
`f7e7791`	24.435	34	0.093	0.967	0.041	23.201	0.183	2019-10-11 04:20	2019-10-11 04:50	`latest-alpine`
`9095bfa`	24.432	34	0.093	0.965	0.042	23.196	0.180	2019-10-10 11:45	2019-10-10 12:15	`latest-alpine`
`1878564`	24.456	34	0.093	0.967	0.041	23.222	0.182	2019-10-10 11:00	2019-10-10 11:38	`latest-alpine`
`49150fb`	24.423	34	0.094	0.966	0.043	23.185	0.183	2019-10-10 04:51	2019-10-10 05:21	`latest-alpine`
`17f9287`	24.236	43	0.092	1.034	0.037	23.152	0.035	2019-10-10 00:33	2019-10-10 01:03	`latest-alpine`
`3aab341`	24.280	43	0.092	1.035	0.038	23.198	0.036	2019-10-09 10:15	2019-10-09 10:43	`latest-alpine`
`fc546f0`	24.227	43	0.092	1.035	0.038	23.132	0.036	2019-09-30 09:41	2019-09-30 10:11	`v0.0.43-alpine`
`83f96d9`	24.250	43	0.093	1.034	0.038	23.166	0.037	2019-09-29 12:34	2019-09-29 13:05	`latest-alpine`
`4dcf6d0`	24.220	43	0.092	1.035	0.037	23.138	0.034	2019-09-29 11:57	2019-09-29 12:27	`latest-alpine`
`0bb0082`	24.203	43	0.092	1.033	0.038	23.111	0.037	2019-09-29 11:23	2019-09-29 11:50	`latest-alpine`
`cb4e46a`	24.255	43	0.092	1.034	0.037	23.170	0.037	2019-09-29 04:07	2019-09-29 04:37	`latest-alpine`
`a087626`	5.928	86	0.184	1.265	0.099	2.134	0.341	2019-09-27 11:05	2019-09-27 11:39	`latest-alpine`
`51837cf`	24.246	43	0.092	1.035	0.037	23.164	0.035	2019-09-27 06:36	2019-09-27 10:47	`latest-alpine`
`51837cf`	24.304	43	0.091	1.035	0.037	23.220	0.036	2019-09-27 06:36	2019-09-27 10:32	`latest-alpine`

Case 3: Parallel Non-blocking Flow

The workflow is as follows:

The ideal roundtrip latency is 0. The smaller the better.

Result

version_vcs	roundtrip	MB/s	f:send	f:recv	f->r1:send	r1->r2:send	r2->f:send	timestamp_build	timestamp_eval	version_tag
`f40d46a`	0.202	2706	0.136	0.132	0.045	0.053	0.099	2019-11-06 11:52	2019-11-06 12:34	`latest-alpine`
`14cffe3`	0.204	2691	0.137	0.133	0.046	0.053	0.100	2019-11-06 11:13	2019-11-06 11:45	`latest-alpine`
`ca20463`	0.203	2673	0.136	0.132	0.045	0.048	0.101	2019-11-06 10:35	2019-11-06 11:07	`v0.0.47-alpine`
`5484af3`	0.206	2646	0.138	0.134	0.046	0.053	0.101	2019-11-06 09:52	2019-11-06 10:29	`latest-alpine`
`6687b68`	0.202	2689	0.137	0.133	0.046	0.053	0.099	2019-11-06 08:48	2019-11-06 09:40	`latest-alpine`
`b4d2c8c`	0.205	2633	0.137	0.133	0.045	0.048	0.102	2019-10-24 05:32	2019-10-24 06:03	`latest-alpine`
`5de329d`	0.205	2668	0.137	0.133	0.046	0.049	0.101	2019-10-23 04:33	2019-10-23 05:06	`latest-alpine`
`44a54be`	0.205	2650	0.137	0.133	0.046	0.053	0.101	2019-10-23 03:55	2019-10-23 04:28	`latest-alpine`
`01fe27b`	0.205	2662	0.137	0.132	0.046	0.048	0.101	2019-10-23 03:18	2019-10-23 03:50	`latest-alpine`
`388d1ba`	0.208	2612	0.138	0.134	0.045	0.050	0.102	2019-10-23 02:41	2019-10-23 03:12	`latest-alpine`
`69ea3fd`	0.206	2635	0.137	0.134	0.046	0.054	0.102	2019-10-21 14:16	2019-10-21 14:47	`latest-alpine`
`97efb59`	0.205	2659	0.137	0.133	0.046	0.054	0.100	2019-10-21 10:46	2019-10-21 11:18	`latest-alpine`
`da50be0`	0.201	2730	0.135	0.131	0.045	0.053	0.098	2019-10-21 04:07	2019-10-21 04:38	`latest-alpine`
`7865b45`	0.203	2666	0.136	0.132	0.045	0.047	0.101	2019-10-18 08:04	2019-10-18 08:35	`latest-alpine`
`8b43398`	0.206	2634	0.137	0.133	0.046	0.048	0.101	2019-10-17 11:28	2019-10-17 11:59	`latest-alpine`
`15b940d`	0.206	2642	0.137	0.133	0.045	0.049	0.101	2019-10-17 10:51	2019-10-17 11:22	`v0.0.46-alpine`
`5fdf67a`	0.203	2702	0.137	0.132	0.045	0.053	0.100	2019-10-17 10:14	2019-10-17 10:45	`latest-alpine`
`394a007`	0.203	2675	0.135	0.132	0.045	0.047	0.100	2019-10-17 03:53	2019-10-17 04:25	`latest-alpine`
`c35b27e`	0.204	2656	0.136	0.132	0.045	0.049	0.100	2019-10-17 03:16	2019-10-17 03:48	`latest-alpine`
`ca9ad83`	0.208	2614	0.137	0.133	0.045	0.048	0.103	2019-10-16 05:52	2019-10-16 06:23	`latest-alpine`
`a419c34`	0.199	2727	0.135	0.131	0.045	0.051	0.098	2019-10-16 03:08	2019-10-16 03:36	`latest-alpine`
`5676ab8`	0.205	2686	0.138	0.133	0.046	0.051	0.101	2019-10-15 08:48	2019-10-15 09:20	`latest-alpine`
`cfee124`	0.203	2680	0.135	0.131	0.045	0.053	0.100	2019-10-15 08:11	2019-10-15 08:43	`latest-alpine`
`b6837b1`	0.266	1962	0.137	0.133	0.042	0.047	0.167	2019-10-15 06:04	2019-10-15 06:37	`v0.0.45-alpine`
`2d398c1`	0.266	1956	0.137	0.133	0.042	0.047	0.167	2019-10-15 04:53	2019-10-15 05:25	`latest-alpine`
`0ac92e4`	0.262	1986	0.135	0.131	0.041	0.052	0.163	2019-10-14 11:50	2019-10-14 12:21	`latest-alpine`
`5583462`	0.264	1974	0.136	0.131	0.042	0.048	0.164	2019-10-14 09:07	2019-10-14 09:39	`latest-alpine`
`e3ab1aa`	0.263	1983	0.135	0.131	0.041	0.049	0.164	2019-10-12 05:41	2019-10-12 06:12	`latest-alpine`
`e64bc7a`	0.264	1977	0.136	0.132	0.042	0.053	0.164	2019-10-11 13:11	2019-10-11 13:42	`latest-alpine`
`72c6d8f`	0.266	1958	0.137	0.133	0.041	0.053	0.166	2019-10-11 08:08	2019-10-11 08:40	`latest-alpine`
`a5b40c1`	0.263	1974	0.135	0.131	0.042	0.052	0.164	2019-10-11 07:31	2019-10-11 08:02	`v0.0.44-alpine`
`8ca5ef0`	0.265	1962	0.136	0.132	0.042	0.054	0.165	2019-10-11 05:48	2019-10-11 06:19	`latest-alpine`
`f7e7791`	3.383	220	0.126	0.010	0.058	0.052	3.268	2019-10-11 04:20	2019-10-11 04:51	`latest-alpine`
`9095bfa`	3.430	219	0.128	0.010	0.058	0.054	3.313	2019-10-10 11:45	2019-10-10 12:16	`latest-alpine`
`1878564`	3.411	222	0.127	0.010	0.059	0.054	3.293	2019-10-10 11:00	2019-10-10 11:39	`latest-alpine`
`49150fb`	3.398	221	0.127	0.010	0.058	0.047	3.281	2019-10-10 04:51	2019-10-10 05:22	`latest-alpine`
`17f9287`	0.157	3232	0.126	0.124	0.031	0.047	0.063	2019-10-10 00:33	2019-10-10 01:05	`latest-alpine`
`3aab341`	0.158	3213	0.127	0.125	0.031	0.052	0.063	2019-10-09 10:15	2019-10-09 10:44	`latest-alpine`
`fc546f0`	0.159	3195	0.127	0.123	0.031	0.052	0.064	2019-09-30 09:41	2019-09-30 10:12	`v0.0.43-alpine`
`83f96d9`	0.160	3172	0.128	0.127	0.032	0.053	0.063	2019-09-29 12:34	2019-09-29 13:06	`latest-alpine`
`4dcf6d0`	0.156	3245	0.126	0.124	0.031	0.053	0.062	2019-09-29 11:57	2019-09-29 12:29	`latest-alpine`
`0bb0082`	0.155	3261	0.125	0.124	0.030	0.050	0.064	2019-09-29 11:23	2019-09-29 11:51	`latest-alpine`
`cb4e46a`	0.156	3255	0.125	0.124	0.031	0.052	0.063	2019-09-29 04:07	2019-09-29 04:38	`latest-alpine`
`a087626`	1.623	601	0.688	0.703	0.351	0.300	0.953	2019-09-27 11:05	2019-09-27 11:43	`latest-alpine`
`51837cf`	0.159	3181	0.128	0.126	0.032	0.055	0.063	2019-09-27 06:36	2019-09-27 10:48	`latest-alpine`
`51837cf`	0.157	3221	0.126	0.125	0.032	0.053	0.062	2019-09-27 06:36	2019-09-27 10:34	`latest-alpine`

Case 4: Parallel Blocking Flow

The workflow is as follows:

It simulates a parallel pipeline with heavy workload. Both Router1 and Router2 will block the pipeline for 1s.

As Router1 and Router2 are parallel, a naive synchronized implementation will take 25s to finish 50 requests.

Result

version_vcs	roundtrip	MB/s	f:send	f:recv	f->r1:send	r1->r2:send	r2->f:send	timestamp_build	timestamp_eval	version_tag
`f40d46a`	11.707	75	0.079	0.508	10.630	0.030	0.033	2019-11-06 11:52	2019-11-06 12:36	`latest-alpine`
`14cffe3`	11.719	76	0.077	0.508	10.642	0.031	0.035	2019-11-06 11:13	2019-11-06 11:47	`latest-alpine`
`ca20463`	11.722	75	0.078	0.508	10.646	0.030	0.036	2019-11-06 10:35	2019-11-06 11:09	`v0.0.47-alpine`
`5484af3`	11.737	75	0.078	0.508	10.662	0.030	0.034	2019-11-06 09:52	2019-11-06 10:31	`latest-alpine`
`6687b68`	11.715	75	0.079	0.508	10.638	0.031	0.033	2019-11-06 08:48	2019-11-06 09:42	`latest-alpine`
`b4d2c8c`	11.691	76	0.078	0.508	10.613	0.032	0.037	2019-10-24 05:32	2019-10-24 06:05	`latest-alpine`
`5de329d`	11.708	75	0.078	0.507	10.633	0.030	0.036	2019-10-23 04:33	2019-10-23 05:08	`latest-alpine`
`44a54be`	11.722	75	0.077	0.507	10.646	0.030	0.036	2019-10-23 03:55	2019-10-23 04:30	`latest-alpine`
`01fe27b`	11.716	75	0.079	0.508	10.639	0.030	0.033	2019-10-23 03:18	2019-10-23 03:52	`latest-alpine`
`388d1ba`	11.692	76	0.079	0.508	10.617	0.030	0.036	2019-10-23 02:41	2019-10-23 03:15	`latest-alpine`
`69ea3fd`	11.744	75	0.079	0.509	10.668	0.031	0.036	2019-10-21 14:16	2019-10-21 14:49	`latest-alpine`
`97efb59`	11.706	75	0.078	0.507	10.632	0.030	0.036	2019-10-21 10:46	2019-10-21 11:20	`latest-alpine`
`da50be0`	11.718	75	0.078	0.507	10.643	0.030	0.034	2019-10-21 04:07	2019-10-21 04:40	`latest-alpine`
`7865b45`	11.726	76	0.077	0.508	10.653	0.030	0.034	2019-10-18 08:04	2019-10-18 08:37	`latest-alpine`
`8b43398`	11.715	75	0.078	0.508	10.640	0.029	0.035	2019-10-17 11:28	2019-10-17 12:01	`latest-alpine`
`15b940d`	11.746	75	0.077	0.507	10.669	0.031	0.036	2019-10-17 10:51	2019-10-17 11:24	`v0.0.46-alpine`
`5fdf67a`	11.710	75	0.078	0.507	10.634	0.030	0.035	2019-10-17 10:14	2019-10-17 10:47	`latest-alpine`
`394a007`	11.713	76	0.078	0.507	10.638	0.030	0.032	2019-10-17 03:53	2019-10-17 04:27	`latest-alpine`
`c35b27e`	11.710	75	0.077	0.507	10.635	0.030	0.036	2019-10-17 03:16	2019-10-17 03:50	`latest-alpine`
`ca9ad83`	11.721	75	0.079	0.508	10.643	0.031	0.034	2019-10-16 05:52	2019-10-16 06:25	`latest-alpine`
`a419c34`	11.727	75	0.077	0.508	10.652	0.031	0.036	2019-10-16 03:08	2019-10-16 03:38	`latest-alpine`
`5676ab8`	11.722	76	0.078	0.508	10.647	0.030	0.033	2019-10-15 08:48	2019-10-15 09:22	`latest-alpine`
`cfee124`	11.727	75	0.078	0.508	10.650	0.031	0.035	2019-10-15 08:11	2019-10-15 08:45	`latest-alpine`
`b6837b1`	11.705	75	0.078	0.507	10.626	0.030	0.037	2019-10-15 06:04	2019-10-15 06:40	`v0.0.45-alpine`
`2d398c1`	11.731	75	0.077	0.507	10.650	0.030	0.039	2019-10-15 04:53	2019-10-15 05:27	`latest-alpine`
`0ac92e4`	11.721	75	0.077	0.508	10.641	0.029	0.034	2019-10-14 11:50	2019-10-14 12:24	`latest-alpine`
`5583462`	11.741	75	0.076	0.508	10.666	0.031	0.036	2019-10-14 09:07	2019-10-14 09:41	`latest-alpine`
`e3ab1aa`	11.720	75	0.076	0.508	10.640	0.030	0.039	2019-10-12 05:41	2019-10-12 06:14	`latest-alpine`
`e64bc7a`	11.745	75	0.078	0.508	10.665	0.030	0.032	2019-10-11 13:11	2019-10-11 13:44	`latest-alpine`
`72c6d8f`	11.723	75	0.078	0.508	10.641	0.030	0.032	2019-10-11 08:08	2019-10-11 08:42	`latest-alpine`
`a5b40c1`	11.721	75	0.076	0.508	10.644	0.030	0.037	2019-10-11 07:31	2019-10-11 08:04	`v0.0.44-alpine`
`8ca5ef0`	11.759	75	0.077	0.508	10.681	0.030	0.032	2019-10-11 05:48	2019-10-11 06:21	`latest-alpine`
`f7e7791`	11.948	59	0.076	0.456	10.693	0.031	0.210	2019-10-11 04:20	2019-10-11 04:53	`latest-alpine`
`9095bfa`	11.954	60	0.077	0.457	10.701	0.032	0.208	2019-10-10 11:45	2019-10-10 12:18	`latest-alpine`
`1878564`	11.964	60	0.076	0.457	10.718	0.031	0.206	2019-10-10 11:00	2019-10-10 11:41	`latest-alpine`
`49150fb`	11.946	60	0.075	0.457	10.694	0.030	0.206	2019-10-10 04:51	2019-10-10 05:24	`latest-alpine`
`17f9287`	11.845	75	0.075	0.509	10.767	0.030	0.032	2019-10-10 00:33	2019-10-10 01:07	`latest-alpine`
`3aab341`	11.841	75	0.075	0.507	10.755	0.031	0.030	2019-10-09 10:15	2019-10-09 10:46	`latest-alpine`
`fc546f0`	11.820	75	0.074	0.508	10.741	0.032	0.036	2019-09-30 09:41	2019-09-30 10:14	`v0.0.43-alpine`
`83f96d9`	11.824	75	0.075	0.508	10.750	0.030	0.032	2019-09-29 12:34	2019-09-29 13:08	`latest-alpine`
`4dcf6d0`	11.837	75	0.075	0.508	10.761	0.032	0.034	2019-09-29 11:57	2019-09-29 12:31	`latest-alpine`
`0bb0082`	11.834	75	0.074	0.508	10.757	0.030	0.035	2019-09-29 11:23	2019-09-29 11:54	`latest-alpine`
`cb4e46a`	11.820	75	0.073	0.507	10.742	0.031	0.038	2019-09-29 04:07	2019-09-29 04:41	`latest-alpine`
`a087626`	5.022	103	0.081	0.633	0.837	0.344	0.289	2019-09-27 11:05	2019-09-27 11:48	`latest-alpine`
`51837cf`	11.856	75	0.075	0.509	10.783	0.031	0.033	2019-09-27 06:36	2019-09-27 10:51	`latest-alpine`
`51837cf`	11.834	76	0.074	0.508	10.759	0.030	0.033	2019-09-27 06:36	2019-09-27 10:36	`latest-alpine`

Name		Name	Last commit message	Last commit date
Latest commit History 81 Commits
.data		.data
.github		.github
client		client
docker-compose		docker-compose
summary		summary
.gitignore		.gitignore
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
run-dind.sh		run-dind.sh
run.sh		run.sh

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Benchmarking GNES on Network Latency

Experimental Setup

Run Test

Explanation of the Table

Time-related metrics

Meta information

Case 1: Non-blocking Flow

Result

Case 2: Blocking Flow

Result

Case 3: Parallel Non-blocking Flow

Result

Case 4: Parallel Blocking Flow

Result

About

Releases

Packages

Languages

License

hanxiao/benchmark

Folders and files

Latest commit

History

Repository files navigation

Benchmarking GNES on Network Latency

Experimental Setup

Run Test

Explanation of the Table

Time-related metrics

Meta information

Case 1: Non-blocking Flow

Result

Case 2: Blocking Flow

Result

Case 3: Parallel Non-blocking Flow

Result

Case 4: Parallel Blocking Flow

Result

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages