Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](cloud) fix be core when using stream load record in cloud mode #37903

Merged
merged 1 commit into from
Jul 20, 2024

Conversation

sollhui
Copy link
Contributor

@sollhui sollhui commented Jul 16, 2024

when set enable_stream_load_record=true in cloud mode, meet core:

F20240711 00:02:05.733184  2316 storage_engine.cpp:123] Check failed: _type == Type::LOCAL (^A vs. ^@) 
*** Check failure stack trace: ***
    @     0x557f1becd646  google::LogMessageFatal::~LogMessageFatal()
    @     0x557f128ebb9a  doris::BaseStorageEngine::to_local()
    @     0x557f13413005  doris::StreamLoadAction::_save_stream_load_record()
    @     0x557f134125c9  doris::StreamLoadAction::handle()
    @     0x557f1c6399d6  (unknown)
    @     0x557f1c638ee2  (unknown)
    @     0x557f1c61e626  bufferevent_run_readcb_
    @     0x557f1c639f3f  (unknown)
    @     0x557f1c62a679  (unknown)
    @     0x557f1c626e7f  event_base_loop
    @     0x557f13436087  std::_Function_handler<>::_M_invoke()
    @     0x557f12da2148  doris::ThreadPool::dispatch_thread()
    @     0x557f12d970d1  doris::Thread::supervise_thread()
    @     0x7ffb1a784ac3  (unknown)
    @     0x7ffb1a815a04  clone
    @              (nil)  (unknown)

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@sollhui
Copy link
Contributor Author

sollhui commented Jul 16, 2024

run buildall

@sollhui sollhui marked this pull request as draft July 16, 2024 08:05
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40022 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1803f24b51b0073ef0c0a786f4a72b7972f00afd, data reload: false

------ Round 1 ----------------------------------
q1	17625	4790	4409	4409
q2	2012	190	197	190
q3	10484	1164	1127	1127
q4	10197	854	826	826
q5	7570	2682	2702	2682
q6	218	139	140	139
q7	962	601	602	601
q8	9220	2076	2058	2058
q9	8933	6572	6555	6555
q10	8823	3819	3767	3767
q11	498	237	241	237
q12	395	227	221	221
q13	17772	2960	2982	2960
q14	284	233	246	233
q15	518	493	494	493
q16	506	399	376	376
q17	979	803	762	762
q18	8094	7526	7331	7331
q19	8073	1344	1468	1344
q20	697	325	315	315
q21	4854	3118	3205	3118
q22	354	286	278	278
Total cold run time: 119068 ms
Total hot run time: 40022 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4380	4278	4241	4241
q2	383	269	264	264
q3	3072	2898	2932	2898
q4	1978	1738	1753	1738
q5	5597	5525	5466	5466
q6	240	138	145	138
q7	2260	1834	1804	1804
q8	3273	3437	3396	3396
q9	8807	8900	8832	8832
q10	4156	3761	3896	3761
q11	609	512	530	512
q12	814	634	611	611
q13	17253	3176	3181	3176
q14	310	285	300	285
q15	513	504	498	498
q16	483	450	434	434
q17	1817	1511	1530	1511
q18	8114	8099	7858	7858
q19	1755	1662	1535	1535
q20	2333	1865	1862	1862
q21	10848	4842	4829	4829
q22	559	502	511	502
Total cold run time: 79554 ms
Total hot run time: 56151 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173843 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1803f24b51b0073ef0c0a786f4a72b7972f00afd, data reload: false

query1	905	370	363	363
query2	6590	1887	1908	1887
query3	6641	207	215	207
query4	28250	17418	17540	17418
query5	3650	488	472	472
query6	256	170	168	168
query7	4583	287	282	282
query8	241	196	198	196
query9	8397	2385	2388	2385
query10	449	290	292	290
query11	11904	10074	10097	10074
query12	110	82	85	82
query13	1669	365	364	364
query14	9887	7823	6924	6924
query15	213	167	165	165
query16	7119	308	304	304
query17	1374	568	542	542
query18	1828	284	273	273
query19	198	148	146	146
query20	88	80	81	80
query21	204	131	123	123
query22	4451	4081	3977	3977
query23	33998	33817	33596	33596
query24	9361	2975	2890	2890
query25	618	394	406	394
query26	702	146	147	146
query27	2235	274	277	274
query28	6053	2049	2053	2049
query29	902	657	645	645
query30	264	153	158	153
query31	983	787	734	734
query32	98	55	56	55
query33	672	293	295	293
query34	920	489	490	489
query35	685	602	586	586
query36	1093	1000	971	971
query37	154	89	93	89
query38	2993	2878	2796	2796
query39	912	845	841	841
query40	203	118	120	118
query41	50	43	43	43
query42	114	95	99	95
query43	508	489	479	479
query44	1094	737	752	737
query45	199	161	161	161
query46	1084	737	747	737
query47	1826	1797	1786	1786
query48	367	297	290	290
query49	822	402	449	402
query50	781	391	393	391
query51	6884	6822	6690	6690
query52	107	96	93	93
query53	359	298	293	293
query54	786	450	439	439
query55	74	73	72	72
query56	279	260	267	260
query57	1151	1061	1073	1061
query58	248	246	248	246
query59	2904	2815	2968	2815
query60	297	277	274	274
query61	97	93	101	93
query62	792	662	635	635
query63	318	287	289	287
query64	9185	2318	1658	1658
query65	3158	3094	3066	3066
query66	794	328	332	328
query67	15493	15314	14986	14986
query68	4532	548	523	523
query69	462	341	331	331
query70	1144	1136	1152	1136
query71	367	287	272	272
query72	7517	6058	5478	5478
query73	734	327	320	320
query74	6155	5617	5697	5617
query75	3393	2713	2711	2711
query76	2210	958	932	932
query77	611	304	296	296
query78	9508	9808	11233	9808
query79	1682	526	512	512
query80	1104	464	469	464
query81	601	216	218	216
query82	298	127	136	127
query83	308	163	159	159
query84	252	87	87	87
query85	1611	317	299	299
query86	485	328	303	303
query87	3374	3082	3062	3062
query88	3433	2458	2458	2458
query89	470	400	388	388
query90	1773	199	194	194
query91	128	100	97	97
query92	59	51	47	47
query93	1006	499	503	499
query94	1153	222	218	218
query95	414	314	323	314
query96	589	277	276	276
query97	3234	3032	3054	3032
query98	224	200	200	200
query99	1506	1209	1217	1209
Total cold run time: 271961 ms
Total hot run time: 173843 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.51 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1803f24b51b0073ef0c0a786f4a72b7972f00afd, data reload: false

query1	0.04	0.03	0.04
query2	0.09	0.04	0.04
query3	0.23	0.04	0.05
query4	1.69	0.06	0.06
query5	0.49	0.47	0.50
query6	1.14	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.50	0.50
query10	0.55	0.54	0.53
query11	0.16	0.11	0.11
query12	0.15	0.12	0.13
query13	0.59	0.59	0.59
query14	0.76	0.77	0.77
query15	0.85	0.82	0.82
query16	0.36	0.36	0.36
query17	0.94	0.96	0.97
query18	0.23	0.21	0.22
query19	1.74	1.68	1.70
query20	0.01	0.01	0.01
query21	15.39	0.74	0.66
query22	3.98	7.46	1.97
query23	18.31	1.36	1.25
query24	2.07	0.26	0.23
query25	0.15	0.08	0.09
query26	0.30	0.20	0.20
query27	0.45	0.23	0.22
query28	13.21	1.00	1.00
query29	12.59	3.29	3.35
query30	0.26	0.06	0.05
query31	2.88	0.38	0.38
query32	3.31	0.47	0.47
query33	2.94	2.88	2.93
query34	16.98	4.34	4.38
query35	4.46	4.44	4.39
query36	0.65	0.48	0.46
query37	0.19	0.15	0.16
query38	0.16	0.15	0.14
query39	0.05	0.03	0.04
query40	0.16	0.12	0.11
query41	0.09	0.05	0.05
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.31 s
Total hot run time: 30.51 s

@sollhui sollhui marked this pull request as ready for review July 16, 2024 10:45
@sollhui
Copy link
Contributor Author

sollhui commented Jul 16, 2024

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40050 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 98b7f5f57c21001517bd5707c01e757cd38f221f, data reload: false

------ Round 1 ----------------------------------
q1	17602	4401	4359	4359
q2	2013	192	187	187
q3	10453	1147	1157	1147
q4	10190	830	840	830
q5	7564	2697	2638	2638
q6	225	139	140	139
q7	960	596	593	593
q8	9228	2065	2079	2065
q9	8813	6567	6581	6567
q10	8858	3766	3774	3766
q11	459	239	239	239
q12	460	225	222	222
q13	17813	3012	2952	2952
q14	290	239	247	239
q15	525	491	483	483
q16	505	394	378	378
q17	981	594	742	594
q18	7976	7508	7445	7445
q19	7859	1414	1449	1414
q20	685	314	309	309
q21	4881	3203	3444	3203
q22	353	289	281	281
Total cold run time: 118693 ms
Total hot run time: 40050 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4421	4271	4279	4271
q2	367	275	272	272
q3	3025	2908	2972	2908
q4	2027	1794	1742	1742
q5	5628	5512	5512	5512
q6	239	138	139	138
q7	2286	1861	1784	1784
q8	3320	3463	3395	3395
q9	8796	8929	8832	8832
q10	4146	3741	3829	3741
q11	589	495	521	495
q12	842	651	625	625
q13	16935	3187	3190	3187
q14	307	295	317	295
q15	529	506	486	486
q16	495	443	436	436
q17	1799	1536	1530	1530
q18	8092	8029	7784	7784
q19	1746	1403	1599	1403
q20	3047	1899	1859	1859
q21	10909	4768	4944	4768
q22	580	498	496	496
Total cold run time: 80125 ms
Total hot run time: 55959 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173535 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 98b7f5f57c21001517bd5707c01e757cd38f221f, data reload: false

query1	902	394	364	364
query2	6555	1785	1742	1742
query3	6636	213	224	213
query4	28261	17501	17454	17454
query5	3718	480	474	474
query6	259	164	163	163
query7	4606	286	285	285
query8	239	196	206	196
query9	8520	2409	2406	2406
query10	432	281	276	276
query11	12128	9938	10094	9938
query12	115	93	85	85
query13	1652	375	370	370
query14	10202	7700	7746	7700
query15	218	164	172	164
query16	7148	331	313	313
query17	1345	554	550	550
query18	1837	282	279	279
query19	199	154	151	151
query20	92	85	83	83
query21	204	130	126	126
query22	4475	4176	4026	4026
query23	33934	33853	33714	33714
query24	11027	2819	2882	2819
query25	626	429	416	416
query26	715	155	155	155
query27	2334	275	279	275
query28	6167	2080	2078	2078
query29	999	619	647	619
query30	249	152	156	152
query31	953	762	737	737
query32	100	55	56	55
query33	658	311	298	298
query34	883	505	501	501
query35	682	585	609	585
query36	1142	958	986	958
query37	143	78	82	78
query38	2957	2860	2831	2831
query39	925	807	808	807
query40	214	119	117	117
query41	46	42	42	42
query42	121	95	101	95
query43	508	454	452	452
query44	1056	721	725	721
query45	194	165	165	165
query46	1084	754	735	735
query47	1855	1753	1777	1753
query48	354	296	285	285
query49	832	399	410	399
query50	766	401	377	377
query51	6856	6815	6769	6769
query52	99	100	88	88
query53	353	284	289	284
query54	864	446	442	442
query55	75	74	73	73
query56	287	268	268	268
query57	1163	1040	1079	1040
query58	255	267	257	257
query59	2819	2643	2561	2561
query60	323	279	278	278
query61	97	96	94	94
query62	794	649	611	611
query63	317	280	281	280
query64	9126	2198	1612	1612
query65	3266	3110	3103	3103
query66	777	345	324	324
query67	15553	14922	15127	14922
query68	4630	531	541	531
query69	610	475	367	367
query70	1211	1121	1181	1121
query71	424	278	280	278
query72	8786	5551	5747	5551
query73	746	325	320	320
query74	6066	5756	5703	5703
query75	4132	2698	2706	2698
query76	2966	977	908	908
query77	684	296	298	296
query78	9759	10141	9077	9077
query79	7943	525	516	516
query80	1473	473	475	473
query81	594	218	223	218
query82	1497	137	133	133
query83	347	168	165	165
query84	278	83	87	83
query85	1309	336	290	290
query86	458	317	306	306
query87	3280	3080	3085	3080
query88	4712	2356	2381	2356
query89	490	394	377	377
query90	1866	191	189	189
query91	141	101	98	98
query92	65	47	49	47
query93	6439	507	504	504
query94	1229	218	210	210
query95	399	321	312	312
query96	606	281	267	267
query97	3156	3039	3048	3039
query98	215	204	205	204
query99	1590	1235	1251	1235
Total cold run time: 292392 ms
Total hot run time: 173535 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.16 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 98b7f5f57c21001517bd5707c01e757cd38f221f, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.69	0.08	0.07
query5	0.52	0.47	0.47
query6	1.14	0.72	0.71
query7	0.02	0.02	0.01
query8	0.06	0.04	0.04
query9	0.56	0.49	0.49
query10	0.54	0.54	0.53
query11	0.14	0.11	0.12
query12	0.15	0.12	0.12
query13	0.59	0.58	0.59
query14	0.76	0.78	0.78
query15	0.85	0.80	0.82
query16	0.37	0.37	0.38
query17	1.00	1.03	1.03
query18	0.24	0.21	0.21
query19	1.91	1.82	1.67
query20	0.01	0.01	0.01
query21	15.39	0.73	0.65
query22	3.93	6.73	2.42
query23	18.27	1.39	1.34
query24	2.21	0.23	0.22
query25	0.15	0.08	0.08
query26	0.29	0.21	0.21
query27	0.45	0.22	0.22
query28	13.22	1.01	1.00
query29	12.65	3.30	3.32
query30	0.26	0.05	0.05
query31	2.91	0.39	0.40
query32	3.25	0.48	0.47
query33	2.90	2.92	2.93
query34	17.02	4.35	4.43
query35	4.39	4.41	4.41
query36	0.65	0.45	0.48
query37	0.18	0.15	0.17
query38	0.15	0.14	0.15
query39	0.05	0.03	0.04
query40	0.15	0.12	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.04
query43	0.05	0.04	0.03
Total cold run time: 109.55 s
Total hot run time: 31.16 s

@sollhui
Copy link
Contributor Author

sollhui commented Jul 17, 2024

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39739 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 020e6400b73d5f37eaa17cb1062be3573b3f4927, data reload: false

------ Round 1 ----------------------------------
q1	17618	4461	4259	4259
q2	2017	187	191	187
q3	10445	1284	1096	1096
q4	10187	856	750	750
q5	7536	2712	2676	2676
q6	220	136	136	136
q7	1039	619	621	619
q8	9215	2049	2090	2049
q9	8772	6578	6596	6578
q10	8723	3746	3733	3733
q11	457	235	230	230
q12	455	230	233	230
q13	17763	2938	2965	2938
q14	282	223	237	223
q15	518	488	476	476
q16	500	371	378	371
q17	957	632	642	632
q18	8165	7515	7484	7484
q19	6131	1367	1384	1367
q20	682	326	324	324
q21	4890	3101	3308	3101
q22	355	280	291	280
Total cold run time: 116927 ms
Total hot run time: 39739 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4386	4240	4212	4212
q2	377	276	258	258
q3	3026	2782	2891	2782
q4	1985	1748	1712	1712
q5	5701	5519	5453	5453
q6	224	136	133	133
q7	2210	1891	1828	1828
q8	3292	3409	3412	3409
q9	8699	8682	8888	8682
q10	4036	3899	3739	3739
q11	599	514	503	503
q12	832	622	640	622
q13	17187	3165	3207	3165
q14	300	299	277	277
q15	525	500	484	484
q16	500	438	427	427
q17	1830	1504	1513	1504
q18	8345	8008	7828	7828
q19	1729	1513	1616	1513
q20	2063	1877	1868	1868
q21	8506	4789	4728	4728
q22	600	493	498	493
Total cold run time: 76952 ms
Total hot run time: 55620 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173953 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 020e6400b73d5f37eaa17cb1062be3573b3f4927, data reload: false

query1	910	367	372	367
query2	6463	1806	1861	1806
query3	6655	211	229	211
query4	24679	17497	17332	17332
query5	3753	486	485	485
query6	264	169	169	169
query7	4578	287	283	283
query8	241	189	186	186
query9	8550	2428	2412	2412
query10	441	284	277	277
query11	10482	10039	9984	9984
query12	114	83	86	83
query13	1648	370	374	370
query14	10101	8086	7745	7745
query15	222	169	169	169
query16	7400	320	321	320
query17	1798	575	554	554
query18	1840	292	284	284
query19	205	152	156	152
query20	93	83	90	83
query21	210	132	123	123
query22	4324	4098	4018	4018
query23	33883	33539	33639	33539
query24	10603	2893	2892	2892
query25	588	392	422	392
query26	716	147	148	147
query27	2284	283	283	283
query28	5821	2086	2075	2075
query29	897	673	629	629
query30	261	156	155	155
query31	992	768	784	768
query32	94	54	53	53
query33	646	306	285	285
query34	883	494	503	494
query35	687	586	578	578
query36	1076	981	998	981
query37	147	82	93	82
query38	2996	2894	2824	2824
query39	893	856	839	839
query40	211	123	121	121
query41	44	44	50	44
query42	109	102	98	98
query43	512	463	473	463
query44	1094	727	726	726
query45	187	162	159	159
query46	1088	730	736	730
query47	1816	1764	1764	1764
query48	366	292	290	290
query49	828	406	406	406
query50	774	388	397	388
query51	6868	6756	6870	6756
query52	100	100	91	91
query53	357	289	286	286
query54	867	443	445	443
query55	74	74	76	74
query56	292	278	260	260
query57	1104	1054	1054	1054
query58	261	263	250	250
query59	2901	2604	2647	2604
query60	315	266	281	266
query61	96	92	99	92
query62	797	637	658	637
query63	317	283	292	283
query64	9112	2215	7469	2215
query65	3117	3140	3160	3140
query66	714	329	334	329
query67	15519	15032	14951	14951
query68	4607	543	539	539
query69	591	391	344	344
query70	1177	1130	1139	1130
query71	401	280	292	280
query72	8345	5252	5686	5252
query73	737	325	331	325
query74	6208	5662	5628	5628
query75	3495	2698	2685	2685
query76	2572	943	986	943
query77	648	304	303	303
query78	11169	9187	9007	9007
query79	3583	525	533	525
query80	1682	466	466	466
query81	593	221	222	221
query82	839	134	130	130
query83	343	165	163	163
query84	280	92	88	88
query85	1754	314	295	295
query86	475	324	321	321
query87	3283	3129	3124	3124
query88	4058	2380	2404	2380
query89	488	385	398	385
query90	1902	194	195	194
query91	126	98	98	98
query92	63	48	50	48
query93	4940	576	508	508
query94	1273	210	208	208
query95	415	313	322	313
query96	621	271	274	271
query97	3245	2980	3061	2980
query98	209	204	191	191
query99	1535	1285	1259	1259
Total cold run time: 279935 ms
Total hot run time: 173953 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 020e6400b73d5f37eaa17cb1062be3573b3f4927, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.03
query3	0.23	0.05	0.04
query4	1.67	0.07	0.07
query5	0.50	0.50	0.50
query6	1.16	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.48	0.49
query10	0.54	0.53	0.53
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.60	0.58
query14	0.75	0.77	0.77
query15	0.86	0.81	0.81
query16	0.36	0.36	0.36
query17	0.95	1.02	1.03
query18	0.23	0.22	0.22
query19	1.82	1.66	1.71
query20	0.01	0.01	0.01
query21	15.39	0.76	0.66
query22	4.52	6.95	1.82
query23	18.32	1.25	1.22
query24	2.14	0.22	0.22
query25	0.16	0.09	0.08
query26	0.29	0.22	0.21
query27	0.46	0.23	0.23
query28	13.27	1.02	0.99
query29	12.59	3.26	3.24
query30	0.25	0.06	0.05
query31	2.86	0.39	0.38
query32	3.29	0.48	0.46
query33	2.88	2.93	2.87
query34	17.07	4.35	4.33
query35	4.42	4.39	4.39
query36	0.66	0.49	0.46
query37	0.20	0.16	0.15
query38	0.15	0.14	0.15
query39	0.04	0.03	0.03
query40	0.15	0.12	0.12
query41	0.09	0.05	0.04
query42	0.05	0.05	0.05
query43	0.04	0.05	0.05
Total cold run time: 109.99 s
Total hot run time: 30.31 s

@sollhui
Copy link
Contributor Author

sollhui commented Jul 17, 2024

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40315 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aa070deeb9d723fc44ae9b4dffd38d7b0996b5ba, data reload: false

------ Round 1 ----------------------------------
q1	18223	4463	4369	4369
q2	2321	195	200	195
q3	11702	1210	1158	1158
q4	10300	912	882	882
q5	7898	2710	2804	2710
q6	220	138	141	138
q7	976	616	604	604
q8	9549	2055	2078	2055
q9	8683	6542	6501	6501
q10	8637	3786	3833	3786
q11	475	246	247	246
q12	395	232	229	229
q13	17754	2989	2993	2989
q14	273	231	239	231
q15	512	497	489	489
q16	491	385	378	378
q17	972	659	650	650
q18	8313	7435	7414	7414
q19	6754	1448	1436	1436
q20	681	316	328	316
q21	4877	3253	3311	3253
q22	352	286	290	286
Total cold run time: 120358 ms
Total hot run time: 40315 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4354	4240	4205	4205
q2	363	266	275	266
q3	3024	2791	2786	2786
q4	1885	1578	1618	1578
q5	5275	5319	5302	5302
q6	222	134	138	134
q7	2122	1765	1688	1688
q8	3172	3334	3296	3296
q9	8431	8376	8371	8371
q10	3904	3665	3709	3665
q11	591	497	483	483
q12	771	590	593	590
q13	16371	3009	3018	3009
q14	312	280	276	276
q15	512	478	478	478
q16	454	420	426	420
q17	1746	1495	1496	1495
q18	7621	7492	7394	7394
q19	1689	1549	1490	1490
q20	2035	1762	1789	1762
q21	4825	4742	4661	4661
q22	588	486	508	486
Total cold run time: 70267 ms
Total hot run time: 53835 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172641 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aa070deeb9d723fc44ae9b4dffd38d7b0996b5ba, data reload: false

query1	924	369	362	362
query2	6471	1936	1826	1826
query3	6654	220	221	220
query4	28431	17396	17308	17308
query5	4159	490	478	478
query6	256	165	163	163
query7	4588	299	297	297
query8	228	189	197	189
query9	8623	2357	2352	2352
query10	433	278	270	270
query11	11084	10085	10069	10069
query12	132	85	86	85
query13	1642	396	362	362
query14	9576	7558	7708	7558
query15	238	164	162	162
query16	7819	343	312	312
query17	1797	557	524	524
query18	1913	278	271	271
query19	199	146	150	146
query20	90	81	106	81
query21	206	128	133	128
query22	4286	4007	4013	4007
query23	33841	33010	33147	33010
query24	11231	2807	2849	2807
query25	633	370	376	370
query26	1478	152	147	147
query27	2937	292	271	271
query28	7499	1991	1974	1974
query29	985	629	600	600
query30	287	146	151	146
query31	959	745	771	745
query32	102	51	55	51
query33	770	304	289	289
query34	952	470	505	470
query35	689	594	590	590
query36	1103	940	933	933
query37	163	79	78	78
query38	2866	2738	2754	2738
query39	859	829	827	827
query40	276	121	121	121
query41	49	48	46	46
query42	124	99	104	99
query43	497	478	475	475
query44	1188	731	726	726
query45	192	163	161	161
query46	1096	736	752	736
query47	1871	1756	1771	1756
query48	376	288	292	288
query49	1163	419	448	419
query50	776	395	388	388
query51	6915	6860	6918	6860
query52	104	93	94	93
query53	361	284	290	284
query54	1065	450	444	444
query55	76	76	75	75
query56	289	263	264	263
query57	1155	1062	1058	1058
query58	255	235	264	235
query59	2816	2543	2609	2543
query60	299	271	271	271
query61	96	90	92	90
query62	840	632	650	632
query63	355	285	288	285
query64	10507	2215	1700	1700
query65	3179	3109	3093	3093
query66	1356	324	327	324
query67	15541	14975	14972	14972
query68	4533	538	539	538
query69	480	326	319	319
query70	1157	1151	1126	1126
query71	395	287	277	277
query72	7634	5641	5657	5641
query73	743	325	323	323
query74	6092	5657	5690	5657
query75	3427	2744	2721	2721
query76	2477	995	909	909
query77	471	304	296	296
query78	9527	9049	9131	9049
query79	2625	524	528	524
query80	2138	461	488	461
query81	583	220	225	220
query82	811	140	129	129
query83	309	166	162	162
query84	274	89	83	83
query85	2006	307	298	298
query86	499	315	317	315
query87	3359	3088	3135	3088
query88	3790	2359	2348	2348
query89	475	397	381	381
query90	1844	195	192	192
query91	125	99	105	99
query92	64	50	50	50
query93	2773	499	496	496
query94	1270	220	287	220
query95	404	318	322	318
query96	594	274	275	274
query97	3181	3054	3040	3040
query98	226	199	191	191
query99	1771	1272	1243	1243
Total cold run time: 285977 ms
Total hot run time: 172641 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.72 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit aa070deeb9d723fc44ae9b4dffd38d7b0996b5ba, data reload: false

query1	0.04	0.04	0.02
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.51	0.50	0.49
query6	1.13	0.72	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.05
query9	0.54	0.48	0.50
query10	0.53	0.53	0.54
query11	0.15	0.11	0.11
query12	0.14	0.12	0.13
query13	0.61	0.59	0.59
query14	0.74	0.79	0.77
query15	0.85	0.81	0.81
query16	0.34	0.36	0.36
query17	0.95	1.02	0.99
query18	0.22	0.22	0.22
query19	1.79	1.67	1.69
query20	0.02	0.01	0.01
query21	15.39	0.77	0.67
query22	3.94	7.82	1.96
query23	18.29	1.41	1.34
query24	2.04	0.24	0.22
query25	0.16	0.09	0.09
query26	0.31	0.21	0.21
query27	0.45	0.24	0.23
query28	13.29	1.02	1.00
query29	12.64	3.30	3.26
query30	0.25	0.06	0.05
query31	2.88	0.41	0.39
query32	3.25	0.48	0.47
query33	2.91	2.93	2.96
query34	16.85	4.31	4.40
query35	4.41	4.42	4.42
query36	0.64	0.46	0.49
query37	0.19	0.16	0.16
query38	0.16	0.15	0.14
query39	0.04	0.04	0.03
query40	0.15	0.12	0.12
query41	0.09	0.05	0.05
query42	0.06	0.06	0.06
query43	0.05	0.04	0.04
Total cold run time: 109.05 s
Total hot run time: 30.72 s

@doris-robot
Copy link

TPC-H: Total hot run time: 39667 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1b0ac5b2d3b31b801bd9e34384e68a0287990724, data reload: false

------ Round 1 ----------------------------------
q1	17625	4326	4263	4263
q2	2019	192	186	186
q3	10446	1205	1085	1085
q4	10198	849	760	760
q5	7535	2713	2626	2626
q6	217	138	137	137
q7	963	596	601	596
q8	9216	2088	2049	2049
q9	8865	6573	6573	6573
q10	8796	3741	3766	3741
q11	442	234	239	234
q12	431	226	217	217
q13	17773	3013	2980	2980
q14	268	228	241	228
q15	520	473	481	473
q16	490	376	380	376
q17	960	714	732	714
q18	8041	7362	7376	7362
q19	6949	1433	1277	1277
q20	658	329	328	328
q21	4852	3213	3175	3175
q22	350	287	293	287
Total cold run time: 117614 ms
Total hot run time: 39667 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4330	4238	4216	4216
q2	374	256	263	256
q3	2996	2806	2950	2806
q4	1969	1756	1705	1705
q5	5630	5471	5463	5463
q6	222	129	132	129
q7	2268	1846	1821	1821
q8	3276	3390	3439	3390
q9	8708	8715	8837	8715
q10	4069	3844	3684	3684
q11	578	491	511	491
q12	826	669	625	625
q13	16069	3167	3174	3167
q14	311	286	283	283
q15	529	484	494	484
q16	484	429	442	429
q17	1816	1563	1488	1488
q18	8170	7861	7772	7772
q19	1819	1702	1609	1609
q20	2168	1896	1863	1863
q21	5055	4949	4768	4768
q22	615	492	503	492
Total cold run time: 72282 ms
Total hot run time: 55656 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174063 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1b0ac5b2d3b31b801bd9e34384e68a0287990724, data reload: false

query1	927	366	362	362
query2	6449	1892	1869	1869
query3	6635	206	216	206
query4	28298	17381	17540	17381
query5	3698	467	487	467
query6	287	175	169	169
query7	4593	292	294	292
query8	250	206	192	192
query9	8628	2505	2487	2487
query10	429	293	298	293
query11	11820	9912	10148	9912
query12	119	84	85	84
query13	1634	367	365	365
query14	10250	7577	7523	7523
query15	224	170	171	170
query16	7716	315	305	305
query17	1572	547	523	523
query18	1894	270	272	270
query19	191	141	143	141
query20	89	83	82	82
query21	203	127	123	123
query22	4301	3933	3960	3933
query23	33779	34769	33850	33850
query24	10980	2940	3004	2940
query25	515	370	392	370
query26	687	155	150	150
query27	2405	282	282	282
query28	6149	2123	2096	2096
query29	875	640	626	626
query30	254	159	153	153
query31	929	801	806	801
query32	101	54	64	54
query33	710	327	303	303
query34	928	505	500	500
query35	682	568	601	568
query36	1145	957	971	957
query37	141	92	80	80
query38	2964	2848	2833	2833
query39	903	801	812	801
query40	200	115	118	115
query41	45	43	44	43
query42	116	103	94	94
query43	502	481	481	481
query44	1069	725	723	723
query45	189	159	158	158
query46	1078	737	761	737
query47	1863	1768	1754	1754
query48	364	296	302	296
query49	826	401	427	401
query50	786	395	400	395
query51	6932	6853	6728	6728
query52	104	95	89	89
query53	357	289	294	289
query54	892	476	468	468
query55	77	75	72	72
query56	300	280	290	280
query57	1112	1052	1057	1052
query58	266	255	271	255
query59	2959	2925	2691	2691
query60	326	301	298	298
query61	127	120	113	113
query62	840	657	651	651
query63	314	290	294	290
query64	9207	2281	1709	1709
query65	3178	3103	3113	3103
query66	757	339	346	339
query67	15485	15002	14912	14912
query68	4636	560	560	560
query69	725	509	351	351
query70	1129	1124	1169	1124
query71	460	308	284	284
query72	9022	6141	5547	5547
query73	757	334	328	328
query74	6161	5687	5676	5676
query75	3651	2713	2667	2667
query76	3274	960	967	960
query77	642	295	309	295
query78	10285	8962	8998	8962
query79	6270	543	528	528
query80	1999	496	480	480
query81	585	224	218	218
query82	929	129	128	128
query83	326	167	163	163
query84	279	93	86	86
query85	1769	311	294	294
query86	460	329	316	316
query87	3337	3106	3170	3106
query88	5195	2493	2454	2454
query89	547	375	378	375
query90	1867	196	200	196
query91	137	103	99	99
query92	60	49	48	48
query93	6003	525	524	524
query94	1238	216	221	216
query95	411	326	324	324
query96	627	274	272	272
query97	3200	3030	3004	3004
query98	218	197	192	192
query99	1611	1277	1237	1237
Total cold run time: 292435 ms
Total hot run time: 174063 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1b0ac5b2d3b31b801bd9e34384e68a0287990724, data reload: false

query1	0.03	0.03	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.04
query4	1.68	0.07	0.07
query5	0.50	0.49	0.49
query6	1.13	0.72	0.71
query7	0.02	0.01	0.01
query8	0.04	0.05	0.05
query9	0.55	0.49	0.49
query10	0.53	0.54	0.55
query11	0.15	0.11	0.12
query12	0.14	0.12	0.12
query13	0.60	0.59	0.58
query14	0.76	0.79	0.78
query15	0.86	0.82	0.82
query16	0.37	0.37	0.37
query17	1.05	1.05	0.97
query18	0.23	0.22	0.22
query19	1.83	1.69	1.70
query20	0.02	0.01	0.01
query21	15.43	0.76	0.65
query22	4.79	6.90	1.74
query23	18.25	1.32	1.24
query24	2.12	0.22	0.23
query25	0.16	0.08	0.08
query26	0.31	0.22	0.22
query27	0.46	0.24	0.23
query28	13.40	1.02	1.00
query29	12.92	3.32	3.29
query30	0.26	0.05	0.06
query31	2.86	0.39	0.39
query32	3.25	0.47	0.46
query33	2.83	2.92	2.88
query34	17.02	4.28	4.36
query35	4.41	4.46	4.41
query36	0.66	0.46	0.46
query37	0.18	0.15	0.15
query38	0.16	0.15	0.14
query39	0.04	0.03	0.03
query40	0.15	0.12	0.12
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.03	0.04
Total cold run time: 110.66 s
Total hot run time: 30.31 s

liaoxin01
liaoxin01 previously approved these changes Jul 18, 2024
Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 18, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

dataroaring
dataroaring previously approved these changes Jul 18, 2024
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sollhui
Copy link
Contributor Author

sollhui commented Jul 19, 2024

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 19, 2024
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39981 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 52d06c35a31b40c713b8b39b30a99aa2f083cdd8, data reload: false

------ Round 1 ----------------------------------
q1	17842	4364	4300	4300
q2	2034	196	200	196
q3	10497	1178	1127	1127
q4	10231	786	830	786
q5	7528	2682	2640	2640
q6	220	140	140	140
q7	970	596	610	596
q8	9217	2074	2087	2074
q9	8749	6557	6560	6557
q10	8800	3762	3766	3762
q11	446	249	249	249
q12	401	233	229	229
q13	19034	2940	3007	2940
q14	286	239	237	237
q15	523	481	470	470
q16	520	384	374	374
q17	983	686	743	686
q18	8125	7625	7537	7537
q19	6458	1349	1315	1315
q20	660	332	343	332
q21	4929	3154	3344	3154
q22	356	285	280	280
Total cold run time: 118809 ms
Total hot run time: 39981 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4397	4245	4291	4245
q2	370	269	255	255
q3	3085	2892	2933	2892
q4	2025	1737	1710	1710
q5	5610	5544	5485	5485
q6	225	143	134	134
q7	2285	1855	1862	1855
q8	3279	3459	3377	3377
q9	8712	8917	8771	8771
q10	4079	3695	3893	3695
q11	592	498	493	493
q12	827	660	666	660
q13	15982	3205	3189	3189
q14	323	298	285	285
q15	517	488	494	488
q16	485	441	450	441
q17	1807	1505	1497	1497
q18	8233	7951	7879	7879
q19	1765	1580	1686	1580
q20	3490	1893	1859	1859
q21	10287	4788	4971	4788
q22	577	519	509	509
Total cold run time: 78952 ms
Total hot run time: 56087 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173514 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 52d06c35a31b40c713b8b39b30a99aa2f083cdd8, data reload: false

query1	911	374	366	366
query2	6451	1902	1849	1849
query3	6633	205	213	205
query4	27984	17544	17392	17392
query5	3748	485	469	469
query6	258	188	173	173
query7	4586	288	289	288
query8	241	205	206	205
query9	8530	2496	2455	2455
query10	438	279	275	275
query11	11779	10061	10110	10061
query12	119	94	80	80
query13	1650	375	367	367
query14	10137	8040	7315	7315
query15	218	172	167	167
query16	7277	311	301	301
query17	1388	536	536	536
query18	1705	270	275	270
query19	193	151	151	151
query20	87	81	85	81
query21	211	130	123	123
query22	4313	4283	3994	3994
query23	33942	33620	33729	33620
query24	11080	2885	2847	2847
query25	588	409	387	387
query26	714	156	150	150
query27	2338	274	277	274
query28	6059	2115	2076	2076
query29	915	640	662	640
query30	253	157	149	149
query31	948	752	765	752
query32	94	58	58	58
query33	656	315	290	290
query34	893	501	505	501
query35	694	588	565	565
query36	1116	975	962	962
query37	147	84	86	84
query38	2986	2824	2887	2824
query39	891	806	808	806
query40	197	121	123	121
query41	46	46	46	46
query42	127	100	98	98
query43	494	475	473	473
query44	1091	740	746	740
query45	192	162	161	161
query46	1087	745	768	745
query47	1848	1784	1792	1784
query48	363	288	288	288
query49	823	403	409	403
query50	777	386	395	386
query51	6856	6864	6776	6776
query52	109	90	94	90
query53	350	279	278	278
query54	846	455	443	443
query55	75	76	75	75
query56	285	261	263	261
query57	1117	1062	1053	1053
query58	235	243	242	242
query59	2757	2746	2777	2746
query60	308	275	280	275
query61	97	97	92	92
query62	804	660	649	649
query63	322	281	283	281
query64	9125	2232	1622	1622
query65	3139	3128	3100	3100
query66	734	323	330	323
query67	15568	15061	15064	15061
query68	6190	546	545	545
query69	707	428	333	333
query70	1117	1078	1080	1078
query71	431	284	281	281
query72	8036	5752	5612	5612
query73	797	329	327	327
query74	6269	5742	5664	5664
query75	3422	2698	2675	2675
query76	3200	933	965	933
query77	674	312	307	307
query78	10125	9002	8894	8894
query79	3342	520	519	519
query80	2712	489	471	471
query81	581	222	225	222
query82	1420	141	143	141
query83	309	184	169	169
query84	278	89	89	89
query85	1438	301	294	294
query86	459	286	305	286
query87	3293	3097	3083	3083
query88	4004	2369	2371	2369
query89	472	389	377	377
query90	1718	191	188	188
query91	131	98	103	98
query92	60	53	49	49
query93	4785	516	527	516
query94	1150	221	214	214
query95	406	320	332	320
query96	614	289	271	271
query97	3218	3041	3057	3041
query98	236	205	197	197
query99	1532	1231	1302	1231
Total cold run time: 286069 ms
Total hot run time: 173514 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.46 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 52d06c35a31b40c713b8b39b30a99aa2f083cdd8, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.04
query3	0.23	0.05	0.06
query4	1.66	0.09	0.08
query5	0.50	0.50	0.48
query6	1.15	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.55	0.49	0.49
query10	0.54	0.54	0.55
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.59	0.59	0.59
query14	0.76	0.78	0.78
query15	0.84	0.82	0.81
query16	0.37	0.37	0.37
query17	1.05	0.97	0.97
query18	0.22	0.22	0.21
query19	1.83	1.70	1.70
query20	0.01	0.01	0.01
query21	15.40	0.74	0.65
query22	4.21	7.80	1.83
query23	18.28	1.45	1.29
query24	2.22	0.22	0.21
query25	0.16	0.08	0.08
query26	0.29	0.21	0.21
query27	0.45	0.23	0.24
query28	13.22	1.00	1.00
query29	12.61	3.24	3.25
query30	0.24	0.06	0.05
query31	2.87	0.40	0.39
query32	3.26	0.47	0.48
query33	2.89	2.90	2.93
query34	16.94	4.32	4.35
query35	4.47	4.39	4.39
query36	0.65	0.48	0.46
query37	0.18	0.16	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.16	0.12	0.13
query41	0.10	0.05	0.04
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.7 s
Total hot run time: 30.46 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 20, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@liaoxin01 liaoxin01 merged commit cbbc15e into apache:master Jul 20, 2024
28 of 31 checks passed
liaoxin01 pushed a commit that referenced this pull request Aug 30, 2024
dataroaring pushed a commit that referenced this pull request Aug 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants