Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement][Audit] add sql type for query audit #37790

Merged
merged 12 commits into from
Jul 26, 2024

Conversation

qzsee
Copy link
Contributor

@qzsee qzsee commented Jul 15, 2024

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@qzsee
Copy link
Contributor Author

qzsee commented Jul 15, 2024

run buildall

@qzsee qzsee force-pushed the add-sql-type branch 2 times, most recently from 5046679 to e4fd3c4 Compare July 16, 2024 06:28
@qzsee
Copy link
Contributor Author

qzsee commented Jul 17, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40364 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2b1bbeb00ad28579b8efe7bf27b46ea88d47d5fa, data reload: false

------ Round 1 ----------------------------------
q1	17609	4483	4346	4346
q2	2025	193	201	193
q3	10424	1236	1170	1170
q4	10190	859	826	826
q5	7550	2729	2727	2727
q6	224	145	140	140
q7	965	604	619	604
q8	9222	2102	2131	2102
q9	8938	6581	6608	6581
q10	8837	3867	3805	3805
q11	465	235	240	235
q12	398	235	234	234
q13	17752	3023	2990	2990
q14	285	241	225	225
q15	541	490	494	490
q16	509	386	378	378
q17	977	671	678	671
q18	8244	7515	7519	7515
q19	7396	1395	1275	1275
q20	721	332	345	332
q21	4921	3298	3242	3242
q22	350	283	301	283
Total cold run time: 118543 ms
Total hot run time: 40364 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4433	4307	4372	4307
q2	391	276	255	255
q3	3115	2932	2993	2932
q4	1990	1755	1729	1729
q5	5602	5560	5546	5546
q6	226	142	135	135
q7	2293	1859	1868	1859
q8	3290	3496	3406	3406
q9	8815	8848	8828	8828
q10	4174	3775	3953	3775
q11	596	497	503	497
q12	841	654	663	654
q13	17357	3178	3201	3178
q14	307	280	286	280
q15	536	487	495	487
q16	489	422	422	422
q17	1823	1496	1528	1496
q18	8078	8013	7941	7941
q19	1779	1574	1707	1574
q20	2123	1891	1874	1874
q21	5103	4836	4996	4836
q22	591	538	516	516
Total cold run time: 73952 ms
Total hot run time: 56527 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174639 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2b1bbeb00ad28579b8efe7bf27b46ea88d47d5fa, data reload: false

query1	908	368	369	368
query2	6423	1936	1957	1936
query3	6650	214	221	214
query4	28113	17614	17568	17568
query5	3770	487	481	481
query6	270	186	168	168
query7	4593	310	308	308
query8	267	205	203	203
query9	8459	2417	2404	2404
query10	442	282	272	272
query11	10562	10169	10056	10056
query12	120	88	90	88
query13	1660	373	373	373
query14	10149	7921	7762	7762
query15	232	166	173	166
query16	7755	318	320	318
query17	1799	554	524	524
query18	1964	292	271	271
query19	201	149	155	149
query20	91	83	83	83
query21	204	130	129	129
query22	4315	4094	4210	4094
query23	34199	33831	33685	33685
query24	10975	2940	2977	2940
query25	619	371	378	371
query26	1495	159	155	155
query27	2812	282	278	278
query28	7792	2070	2052	2052
query29	987	653	655	653
query30	250	168	159	159
query31	999	772	774	772
query32	94	56	57	56
query33	759	300	289	289
query34	999	487	501	487
query35	709	600	584	584
query36	1148	1006	993	993
query37	157	86	88	86
query38	2984	2831	2809	2809
query39	915	878	841	841
query40	269	133	131	131
query41	49	50	55	50
query42	118	104	101	101
query43	501	492	483	483
query44	1160	719	716	716
query45	188	156	164	156
query46	1100	711	735	711
query47	1862	1781	1795	1781
query48	367	295	287	287
query49	910	409	415	409
query50	781	391	393	391
query51	6951	6862	6849	6849
query52	104	96	93	93
query53	351	292	290	290
query54	885	442	441	441
query55	74	75	73	73
query56	288	262	284	262
query57	1167	1039	1051	1039
query58	242	253	253	253
query59	2954	2712	2664	2664
query60	300	274	280	274
query61	99	93	95	93
query62	854	660	655	655
query63	313	296	290	290
query64	10123	2192	1675	1675
query65	3205	3122	3093	3093
query66	915	328	330	328
query67	15380	14973	15194	14973
query68	4622	541	547	541
query69	465	333	330	330
query70	1210	1159	1068	1068
query71	415	283	278	278
query72	6908	5550	6061	5550
query73	743	325	326	325
query74	6131	5680	5675	5675
query75	3459	2686	2674	2674
query76	2738	942	943	942
query77	458	304	378	304
query78	9743	9319	9023	9023
query79	2647	525	517	517
query80	2185	494	475	475
query81	588	229	222	222
query82	843	141	131	131
query83	283	180	168	168
query84	273	90	84	84
query85	2053	319	306	306
query86	496	298	309	298
query87	3338	3134	3106	3106
query88	3893	2376	2379	2376
query89	488	399	391	391
query90	1816	197	193	193
query91	130	102	100	100
query92	61	53	53	53
query93	2771	513	500	500
query94	1176	209	219	209
query95	407	316	322	316
query96	590	274	280	274
query97	3271	3021	3023	3021
query98	230	203	270	203
query99	1666	1295	1280	1280
Total cold run time: 284447 ms
Total hot run time: 174639 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.33 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2b1bbeb00ad28579b8efe7bf27b46ea88d47d5fa, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.05
query3	0.22	0.05	0.05
query4	1.68	0.09	0.08
query5	0.50	0.47	0.48
query6	1.14	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.05	0.05
query9	0.54	0.50	0.49
query10	0.55	0.54	0.55
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.60	0.59	0.59
query14	0.76	0.78	0.79
query15	0.86	0.81	0.81
query16	0.38	0.36	0.36
query17	0.94	1.01	1.00
query18	0.23	0.22	0.22
query19	1.80	1.69	1.83
query20	0.02	0.01	0.01
query21	15.39	0.77	0.67
query22	3.71	6.40	2.57
query23	18.29	1.48	1.27
query24	2.12	0.24	0.22
query25	0.17	0.08	0.08
query26	0.31	0.21	0.21
query27	0.45	0.23	0.24
query28	13.23	1.02	1.00
query29	12.61	3.39	3.27
query30	0.26	0.07	0.05
query31	2.89	0.38	0.38
query32	3.27	0.49	0.48
query33	2.88	2.91	2.94
query34	17.09	4.40	4.34
query35	4.48	4.44	4.46
query36	0.66	0.47	0.47
query37	0.18	0.16	0.15
query38	0.16	0.16	0.15
query39	0.04	0.03	0.04
query40	0.14	0.12	0.13
query41	0.09	0.04	0.05
query42	0.06	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.25 s
Total hot run time: 31.33 s

@morningman morningman self-assigned this Jul 17, 2024
morningman
morningman previously approved these changes Jul 18, 2024
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 18, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@@ -107,4 +108,8 @@ default DataTrait computeDataTrait() {
void computeEqualSet(DataTrait.Builder builder);

void computeFd(DataTrait.Builder builder);

default StmtType stmtType() {
return StmtType.OTHER;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do not give default impl, let it be abstract

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is changed to an abstract method, then all subclasses must implement this method, which is unnecessary

package org.apache.doris.analysis;

public enum StmtType {
DDL,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add comment

  1. clarifying the criteria for classifying a statement. for example we have DDL and CREATE, so when to use DDL directly, when to add a new type of DDL?
  2. sort and classify these types for easy read and maintain

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1.The reason for not simplifying the DDL type is that we can see more detailed SQL types. The reason for adding the DDL type is that if there are new DDL operations in the future, if there is no corresponding method implemented, it will be directly a DDL type.
2.done

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 19, 2024
@qzsee
Copy link
Contributor Author

qzsee commented Jul 19, 2024

run buildall

1 similar comment
@qzsee
Copy link
Contributor Author

qzsee commented Jul 19, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39780 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 47d3fae6c156445545d4717e2f9988a367d4b4c9, data reload: false

------ Round 1 ----------------------------------
q1	17617	4368	4258	4258
q2	2015	195	189	189
q3	10446	1244	1107	1107
q4	10195	775	771	771
q5	7551	2674	2674	2674
q6	222	141	139	139
q7	948	604	598	598
q8	9219	2077	2025	2025
q9	8868	6576	6511	6511
q10	8683	3846	3814	3814
q11	467	236	239	236
q12	444	229	227	227
q13	17759	3003	3045	3003
q14	276	239	239	239
q15	518	495	484	484
q16	512	383	382	382
q17	963	594	748	594
q18	8144	7401	7433	7401
q19	8402	1414	1369	1369
q20	685	322	320	320
q21	4952	3158	3202	3158
q22	348	292	281	281
Total cold run time: 119234 ms
Total hot run time: 39780 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4443	4226	4264	4226
q2	376	279	258	258
q3	3079	2836	2944	2836
q4	1973	1704	1678	1678
q5	5597	5564	5455	5455
q6	231	132	132	132
q7	2260	1877	1859	1859
q8	3257	3439	3433	3433
q9	8790	8920	8796	8796
q10	4123	3733	3887	3733
q11	588	493	482	482
q12	816	681	607	607
q13	17144	3129	3170	3129
q14	315	305	281	281
q15	533	466	503	466
q16	491	432	462	432
q17	1828	1516	1492	1492
q18	8178	7898	7827	7827
q19	4555	1598	1623	1598
q20	2148	1860	1855	1855
q21	5098	4982	4783	4783
q22	605	508	519	508
Total cold run time: 76428 ms
Total hot run time: 55866 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173484 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 47d3fae6c156445545d4717e2f9988a367d4b4c9, data reload: false

query1	923	370	360	360
query2	6444	1888	1768	1768
query3	6656	206	209	206
query4	28318	17682	17166	17166
query5	3568	485	502	485
query6	265	196	156	156
query7	4574	283	280	280
query8	232	196	189	189
query9	8727	2413	2352	2352
query10	418	289	272	272
query11	10696	10133	10081	10081
query12	118	86	81	81
query13	1636	359	349	349
query14	10302	7759	7828	7759
query15	223	167	169	167
query16	7608	498	487	487
query17	1568	532	520	520
query18	1805	304	271	271
query19	211	146	150	146
query20	87	80	84	80
query21	201	131	126	126
query22	4432	4040	3918	3918
query23	34182	33656	33709	33656
query24	10675	2888	2872	2872
query25	599	402	404	402
query26	708	171	153	153
query27	2273	273	279	273
query28	5838	2045	2033	2033
query29	891	666	616	616
query30	250	150	151	150
query31	975	750	794	750
query32	96	53	58	53
query33	662	341	337	337
query34	880	495	490	490
query35	894	768	764	764
query36	1140	983	967	967
query37	150	87	87	87
query38	3000	2857	2866	2857
query39	917	873	865	865
query40	221	136	127	127
query41	48	47	48	47
query42	107	106	107	106
query43	520	472	478	472
query44	1057	715	728	715
query45	202	166	162	162
query46	1080	731	751	731
query47	1865	1767	1772	1767
query48	362	294	292	292
query49	846	455	408	408
query50	769	380	383	380
query51	6925	6805	6868	6805
query52	100	90	92	90
query53	362	293	288	288
query54	886	442	443	442
query55	77	72	72	72
query56	291	269	268	268
query57	1124	1035	1055	1035
query58	248	263	242	242
query59	2805	2649	2552	2552
query60	304	275	278	275
query61	98	97	97	97
query62	801	664	651	651
query63	315	284	290	284
query64	9157	2206	1662	1662
query65	3196	3151	3116	3116
query66	753	328	327	327
query67	15321	15173	15014	15014
query68	4483	530	531	530
query69	564	421	368	368
query70	1123	1132	1175	1132
query71	389	292	280	280
query72	7099	5240	5583	5240
query73	736	320	321	320
query74	6173	5700	5642	5642
query75	3362	2669	2690	2669
query76	2143	938	881	881
query77	458	305	305	305
query78	10182	8908	9335	8908
query79	3046	518	516	516
query80	1808	476	467	467
query81	596	277	226	226
query82	768	132	137	132
query83	328	166	164	164
query84	273	88	83	83
query85	1544	337	308	308
query86	460	327	324	324
query87	3313	3107	3097	3097
query88	4536	2359	2360	2359
query89	507	388	391	388
query90	1758	195	195	195
query91	130	101	105	101
query92	56	49	50	49
query93	5021	504	498	498
query94	806	278	275	275
query95	401	313	323	313
query96	602	269	270	269
query97	3190	3041	3034	3034
query98	226	203	196	196
query99	1554	1274	1244	1244
Total cold run time: 280617 ms
Total hot run time: 173484 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.44 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 47d3fae6c156445545d4717e2f9988a367d4b4c9, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.04	0.04
query3	0.23	0.04	0.05
query4	1.67	0.08	0.08
query5	0.50	0.49	0.48
query6	1.14	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.48	0.49
query10	0.53	0.54	0.54
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.58
query14	0.76	0.77	0.77
query15	0.85	0.80	0.82
query16	0.36	0.37	0.37
query17	1.03	1.02	1.02
query18	0.22	0.22	0.23
query19	1.93	1.73	1.81
query20	0.02	0.01	0.00
query21	15.39	0.74	0.66
query22	3.98	7.38	1.72
query23	18.33	1.34	1.27
query24	2.15	0.23	0.22
query25	0.16	0.09	0.08
query26	0.30	0.21	0.21
query27	0.45	0.23	0.22
query28	13.26	1.02	1.00
query29	12.64	3.26	3.29
query30	0.26	0.06	0.05
query31	2.88	0.39	0.39
query32	3.28	0.46	0.46
query33	2.90	2.86	2.91
query34	17.07	4.43	4.36
query35	4.41	4.44	4.43
query36	0.64	0.49	0.48
query37	0.19	0.15	0.15
query38	0.16	0.15	0.15
query39	0.04	0.03	0.03
query40	0.15	0.12	0.12
query41	0.09	0.05	0.05
query42	0.05	0.05	0.06
query43	0.05	0.04	0.04
Total cold run time: 109.69 s
Total hot run time: 30.44 s

@qzsee
Copy link
Contributor Author

qzsee commented Jul 19, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40063 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e8b10ee89d3d0efff05bcf321412fe2dd31889b7, data reload: false

------ Round 1 ----------------------------------
q1	18067	4748	4394	4394
q2	2245	197	187	187
q3	10454	1235	1010	1010
q4	10191	831	830	830
q5	7552	2727	2705	2705
q6	220	141	140	140
q7	958	614	612	612
q8	9220	2098	2088	2088
q9	8675	6573	6559	6559
q10	8618	3879	3785	3785
q11	497	245	242	242
q12	391	235	226	226
q13	17783	3023	3068	3023
q14	270	239	238	238
q15	516	514	494	494
q16	487	401	381	381
q17	976	634	723	634
q18	8179	7399	7452	7399
q19	6996	1403	1398	1398
q20	697	308	329	308
q21	4854	3130	3903	3130
q22	353	280	282	280
Total cold run time: 118199 ms
Total hot run time: 40063 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4361	4246	4291	4246
q2	374	283	273	273
q3	2978	2795	2756	2756
q4	1910	1649	1653	1649
q5	5290	5312	5319	5312
q6	220	135	134	134
q7	2170	1771	1701	1701
q8	3178	3368	3315	3315
q9	8415	8346	8383	8346
q10	3933	3680	3720	3680
q11	572	496	506	496
q12	758	605	600	600
q13	17086	3014	3026	3014
q14	319	279	273	273
q15	533	486	476	476
q16	469	432	422	422
q17	1761	1504	1494	1494
q18	7560	7626	7505	7505
q19	4815	1634	1473	1473
q20	1982	1782	1795	1782
q21	4812	4691	4705	4691
q22	568	524	492	492
Total cold run time: 74064 ms
Total hot run time: 54130 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173089 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e8b10ee89d3d0efff05bcf321412fe2dd31889b7, data reload: false

query1	912	386	374	374
query2	6439	1887	1802	1802
query3	6653	208	214	208
query4	28081	17535	17453	17453
query5	4210	481	484	481
query6	287	192	171	171
query7	4610	288	288	288
query8	254	197	193	193
query9	8456	2454	2422	2422
query10	450	281	271	271
query11	10427	9900	9920	9900
query12	130	85	88	85
query13	1646	380	376	376
query14	9550	7650	8217	7650
query15	242	170	167	167
query16	7380	474	464	464
query17	1596	568	546	546
query18	1456	286	283	283
query19	201	151	155	151
query20	100	83	83	83
query21	212	129	129	129
query22	4264	4046	3957	3957
query23	33807	33088	33062	33062
query24	12001	2896	2874	2874
query25	651	363	389	363
query26	1754	150	157	150
query27	2624	264	267	264
query28	6915	2011	1996	1996
query29	1010	625	618	618
query30	285	148	147	147
query31	955	770	761	761
query32	88	56	55	55
query33	780	357	352	352
query34	889	486	480	480
query35	866	739	729	729
query36	1124	940	949	940
query37	155	79	78	78
query38	2876	2760	2709	2709
query39	863	803	847	803
query40	276	119	121	119
query41	46	44	44	44
query42	122	100	99	99
query43	483	475	441	441
query44	1292	736	734	734
query45	193	162	165	162
query46	1099	719	728	719
query47	1861	1750	1771	1750
query48	368	284	282	282
query49	1189	423	439	423
query50	774	404	394	394
query51	6872	6772	6835	6772
query52	107	97	94	94
query53	364	290	280	280
query54	1113	453	446	446
query55	81	75	73	73
query56	295	271	288	271
query57	1153	1045	1056	1045
query58	259	257	244	244
query59	2806	2634	2599	2599
query60	298	278	287	278
query61	100	98	101	98
query62	827	653	686	653
query63	322	292	284	284
query64	10521	2268	1681	1681
query65	3182	3110	3093	3093
query66	1299	358	334	334
query67	15475	15018	14900	14900
query68	4670	541	538	538
query69	466	350	349	349
query70	1162	1153	1096	1096
query71	380	317	272	272
query72	6927	5711	5268	5268
query73	730	325	327	325
query74	6084	5740	5699	5699
query75	3397	2764	2694	2694
query76	2835	984	951	951
query77	474	326	297	297
query78	9770	9594	9188	9188
query79	3117	537	531	531
query80	1646	484	480	480
query81	579	216	222	216
query82	813	139	133	133
query83	273	175	170	170
query84	281	86	85	85
query85	2141	323	315	315
query86	333	335	329	329
query87	3274	3150	3118	3118
query88	4550	2357	2378	2357
query89	500	388	399	388
query90	1912	200	195	195
query91	133	104	101	101
query92	70	49	53	49
query93	4594	524	504	504
query94	939	295	257	257
query95	408	319	320	319
query96	610	272	273	272
query97	3282	3035	3010	3010
query98	212	193	294	193
query99	1711	1289	1243	1243
Total cold run time: 286249 ms
Total hot run time: 173089 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.8 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e8b10ee89d3d0efff05bcf321412fe2dd31889b7, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.04	0.05
query4	1.68	0.07	0.07
query5	0.49	0.47	0.47
query6	1.12	0.74	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.54	0.48	0.50
query10	0.55	0.55	0.53
query11	0.15	0.11	0.11
query12	0.14	0.12	0.13
query13	0.60	0.58	0.59
query14	0.76	0.79	0.81
query15	0.85	0.81	0.82
query16	0.40	0.37	0.36
query17	0.98	1.00	0.96
query18	0.23	0.21	0.23
query19	1.77	1.81	1.71
query20	0.01	0.02	0.02
query21	15.51	0.78	0.67
query22	5.06	6.32	2.09
query23	18.32	1.36	1.27
query24	2.04	0.23	0.23
query25	0.16	0.09	0.09
query26	0.30	0.21	0.21
query27	0.46	0.23	0.23
query28	13.30	1.02	1.03
query29	12.67	3.24	3.28
query30	0.25	0.06	0.06
query31	2.87	0.38	0.40
query32	3.26	0.46	0.47
query33	2.86	2.96	2.90
query34	16.98	4.40	4.45
query35	4.44	4.47	4.42
query36	0.64	0.47	0.51
query37	0.18	0.15	0.17
query38	0.15	0.14	0.15
query39	0.05	0.04	0.03
query40	0.15	0.13	0.11
query41	0.10	0.06	0.05
query42	0.07	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 110.54 s
Total hot run time: 30.8 s

morrySnow
morrySnow previously approved these changes Jul 22, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 22, 2024
@qzsee
Copy link
Contributor Author

qzsee commented Jul 22, 2024

run P0

@morningman
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39793 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c5b38ac782ea0da9f709d6bd483c47dce5a2f817, data reload: false

------ Round 1 ----------------------------------
q1	17612	4353	4261	4261
q2	2017	188	187	187
q3	10444	1147	1081	1081
q4	10182	893	902	893
q5	7522	2674	2703	2674
q6	221	137	140	137
q7	946	603	593	593
q8	9215	2047	2066	2047
q9	8942	6559	6489	6489
q10	8849	3785	3729	3729
q11	462	241	240	240
q12	406	226	232	226
q13	17766	2979	2975	2975
q14	287	226	237	226
q15	519	475	481	475
q16	479	373	377	373
q17	951	664	691	664
q18	7994	7467	7294	7294
q19	6441	1454	1460	1454
q20	707	331	330	330
q21	4823	3175	3165	3165
q22	345	280	280	280
Total cold run time: 117130 ms
Total hot run time: 39793 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4383	4223	4237	4223
q2	375	261	278	261
q3	3041	2846	2914	2846
q4	1997	1699	1686	1686
q5	5698	5528	5486	5486
q6	218	129	131	129
q7	2185	1829	1871	1829
q8	3237	3377	3395	3377
q9	8789	8660	8878	8660
q10	4110	3884	3786	3786
q11	582	504	486	486
q12	819	658	689	658
q13	16207	3220	3199	3199
q14	327	292	306	292
q15	522	484	492	484
q16	495	443	430	430
q17	1812	1520	1519	1519
q18	8081	7897	7795	7795
q19	1716	1421	1486	1421
q20	2172	1932	1848	1848
q21	5848	4966	4869	4869
q22	568	520	505	505
Total cold run time: 73182 ms
Total hot run time: 55789 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174722 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c5b38ac782ea0da9f709d6bd483c47dce5a2f817, data reload: false

query1	914	373	367	367
query2	6441	1860	1819	1819
query3	6639	210	221	210
query4	28469	17607	17332	17332
query5	3695	459	483	459
query6	267	190	162	162
query7	4574	298	289	289
query8	245	202	192	192
query9	8595	2455	2414	2414
query10	447	288	284	284
query11	11583	10078	10190	10078
query12	122	84	104	84
query13	1646	366	363	363
query14	10300	7626	7716	7626
query15	215	164	169	164
query16	7770	489	470	470
query17	1572	541	540	540
query18	1935	285	268	268
query19	189	145	150	145
query20	88	81	85	81
query21	200	130	130	130
query22	4306	4098	4015	4015
query23	34260	33641	33745	33641
query24	11058	2957	2876	2876
query25	644	402	379	379
query26	1189	153	155	153
query27	2672	285	276	276
query28	7182	2044	2070	2044
query29	911	664	634	634
query30	256	160	157	157
query31	962	736	779	736
query32	102	52	59	52
query33	763	337	339	337
query34	930	501	489	489
query35	894	775	765	765
query36	1135	994	981	981
query37	160	81	80	80
query38	3023	2827	2814	2814
query39	900	871	841	841
query40	206	117	117	117
query41	45	45	45	45
query42	114	101	99	99
query43	514	446	459	446
query44	1203	713	715	713
query45	192	165	161	161
query46	1083	707	707	707
query47	1827	1817	1770	1770
query48	365	288	295	288
query49	854	403	415	403
query50	777	391	400	391
query51	7139	6707	6635	6635
query52	110	89	91	89
query53	355	291	287	287
query54	890	453	445	445
query55	71	75	74	74
query56	294	260	278	260
query57	1164	1063	1095	1063
query58	288	250	262	250
query59	2990	2754	2560	2560
query60	294	272	278	272
query61	110	94	97	94
query62	808	656	634	634
query63	319	285	287	285
query64	9450	2207	2705	2207
query65	3189	3128	3102	3102
query66	760	326	368	326
query67	15402	15268	15041	15041
query68	4538	524	538	524
query69	579	443	370	370
query70	1224	1113	1110	1110
query71	387	283	273	273
query72	7593	5581	5817	5581
query73	753	330	324	324
query74	6268	5736	5703	5703
query75	3364	2681	2735	2681
query76	2423	994	930	930
query77	439	297	297	297
query78	9923	9170	9712	9170
query79	2916	507	517	507
query80	2257	470	466	466
query81	601	216	220	216
query82	1352	147	137	137
query83	297	168	171	168
query84	256	94	87	87
query85	1473	329	314	314
query86	463	330	305	305
query87	3296	3081	3080	3080
query88	3942	2452	2455	2452
query89	471	382	380	380
query90	1765	193	192	192
query91	127	100	104	100
query92	59	49	47	47
query93	2509	496	498	496
query94	1116	283	273	273
query95	405	323	324	323
query96	605	269	271	269
query97	3157	3030	2994	2994
query98	299	210	189	189
query99	1619	1264	1253	1253
Total cold run time: 284542 ms
Total hot run time: 174722 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.64 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c5b38ac782ea0da9f709d6bd483c47dce5a2f817, data reload: false

query1	0.05	0.03	0.03
query2	0.08	0.03	0.04
query3	0.22	0.05	0.06
query4	1.68	0.10	0.09
query5	0.49	0.48	0.49
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.04	0.04	0.04
query9	0.54	0.48	0.49
query10	0.54	0.55	0.54
query11	0.16	0.11	0.11
query12	0.16	0.12	0.12
query13	0.60	0.58	0.57
query14	0.76	0.79	0.77
query15	0.85	0.82	0.81
query16	0.36	0.36	0.37
query17	1.04	1.05	0.97
query18	0.22	0.22	0.22
query19	1.92	1.82	1.80
query20	0.01	0.00	0.01
query21	15.42	0.75	0.65
query22	4.94	6.62	1.80
query23	18.29	1.43	1.28
query24	2.19	0.22	0.22
query25	0.16	0.07	0.08
query26	0.30	0.20	0.20
query27	0.45	0.23	0.23
query28	13.24	1.01	1.00
query29	12.61	3.37	3.38
query30	0.25	0.06	0.05
query31	2.89	0.38	0.39
query32	3.25	0.48	0.47
query33	2.94	2.92	2.93
query34	17.17	4.37	4.36
query35	4.41	4.39	4.38
query36	0.65	0.45	0.49
query37	0.18	0.16	0.16
query38	0.15	0.15	0.15
query39	0.05	0.03	0.04
query40	0.14	0.12	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.75 s
Total hot run time: 30.64 s

@qzsee
Copy link
Contributor Author

qzsee commented Jul 24, 2024

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 24, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 39550 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4c941e8213fe61d1b58d1b962b64f39d42522e15, data reload: false

------ Round 1 ----------------------------------
q1	17627	4281	4357	4281
q2	2020	191	193	191
q3	10463	1202	1042	1042
q4	10190	849	727	727
q5	7540	2692	2702	2692
q6	221	140	143	140
q7	945	608	587	587
q8	9213	2048	2067	2048
q9	8634	6516	6571	6516
q10	8882	3783	3778	3778
q11	458	236	231	231
q12	433	239	229	229
q13	17770	3014	2997	2997
q14	269	231	250	231
q15	529	481	484	481
q16	506	386	379	379
q17	961	624	739	624
q18	8058	7356	7399	7356
q19	7710	1354	1325	1325
q20	677	323	326	323
q21	5061	3085	3285	3085
q22	349	287	288	287
Total cold run time: 118516 ms
Total hot run time: 39550 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4336	4249	4170	4170
q2	366	264	261	261
q3	2963	2864	2913	2864
q4	1952	1687	1712	1687
q5	5648	5516	5482	5482
q6	217	134	133	133
q7	2218	1933	1814	1814
q8	3274	3385	3426	3385
q9	8702	8745	8914	8745
q10	3999	3876	3778	3778
q11	580	503	505	503
q12	859	624	626	624
q13	16845	3185	3186	3185
q14	323	289	296	289
q15	513	499	485	485
q16	487	447	449	447
q17	1824	1536	1465	1465
q18	8107	7966	7892	7892
q19	1980	1504	1588	1504
q20	2583	1899	1876	1876
q21	9698	4981	4655	4655
q22	583	480	516	480
Total cold run time: 78057 ms
Total hot run time: 55724 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172662 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4c941e8213fe61d1b58d1b962b64f39d42522e15, data reload: false

query1	910	373	368	368
query2	6452	1851	1799	1799
query3	6633	211	220	211
query4	25063	17524	17253	17253
query5	3675	461	480	461
query6	274	171	181	171
query7	4597	287	300	287
query8	249	200	193	193
query9	8567	2424	2401	2401
query10	434	283	278	278
query11	11532	10055	10167	10055
query12	115	82	81	81
query13	1644	388	364	364
query14	10220	7646	6955	6955
query15	222	179	162	162
query16	7627	464	495	464
query17	1311	521	518	518
query18	1652	280	279	279
query19	194	145	147	145
query20	90	81	82	81
query21	229	127	127	127
query22	4372	4077	4039	4039
query23	34139	33788	33581	33581
query24	10833	2901	2887	2887
query25	589	395	384	384
query26	722	148	150	148
query27	2232	271	281	271
query28	6071	2085	2087	2085
query29	884	642	645	642
query30	266	155	154	154
query31	1004	775	749	749
query32	93	53	59	53
query33	656	330	333	330
query34	902	479	509	479
query35	873	751	746	746
query36	1138	983	950	950
query37	137	84	79	79
query38	3002	2869	2821	2821
query39	919	865	843	843
query40	220	122	121	121
query41	46	44	47	44
query42	116	107	103	103
query43	513	434	447	434
query44	1064	729	727	727
query45	190	162	162	162
query46	1085	704	747	704
query47	1874	1736	1764	1736
query48	361	299	288	288
query49	842	401	409	401
query50	773	387	393	387
query51	6855	6657	6631	6631
query52	97	113	96	96
query53	352	285	282	282
query54	877	449	449	449
query55	73	76	73	73
query56	292	263	276	263
query57	1145	1043	1046	1043
query58	235	246	277	246
query59	2942	2448	2553	2448
query60	309	274	278	274
query61	96	94	92	92
query62	828	619	621	619
query63	320	285	281	281
query64	9101	2222	1760	1760
query65	3158	3112	3105	3105
query66	764	335	344	335
query67	15491	15046	14946	14946
query68	4469	562	559	559
query69	483	406	351	351
query70	1178	1181	1153	1153
query71	365	278	274	274
query72	6959	5365	5844	5365
query73	736	331	323	323
query74	6191	5692	5721	5692
query75	3374	2715	2654	2654
query76	2181	917	923	917
query77	467	316	315	315
query78	11030	9150	8900	8900
query79	2932	531	517	517
query80	1374	467	467	467
query81	609	220	222	220
query82	698	135	131	131
query83	281	167	165	165
query84	278	83	85	83
query85	1072	318	344	318
query86	476	312	308	308
query87	3292	3109	3144	3109
query88	3896	2395	2363	2363
query89	480	377	368	368
query90	1811	188	195	188
query91	131	101	99	99
query92	60	49	53	49
query93	4529	519	510	510
query94	1104	273	277	273
query95	403	320	308	308
query96	606	271	276	271
query97	3186	3029	3023	3023
query98	219	197	207	197
query99	1575	1303	1248	1248
Total cold run time: 276986 ms
Total hot run time: 172662 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4c941e8213fe61d1b58d1b962b64f39d42522e15, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.08	0.07
query5	0.50	0.47	0.48
query6	1.14	0.73	0.72
query7	0.02	0.02	0.01
query8	0.06	0.04	0.05
query9	0.55	0.49	0.49
query10	0.53	0.55	0.54
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.60	0.58	0.58
query14	0.74	0.78	0.78
query15	0.85	0.80	0.80
query16	0.37	0.36	0.37
query17	0.99	1.03	1.03
query18	0.23	0.22	0.21
query19	1.87	1.76	1.72
query20	0.02	0.01	0.01
query21	15.38	0.77	0.66
query22	4.20	7.14	2.18
query23	18.23	1.42	1.24
query24	2.14	0.23	0.22
query25	0.16	0.09	0.08
query26	0.30	0.21	0.22
query27	0.45	0.22	0.23
query28	13.29	1.03	1.01
query29	12.60	3.30	3.28
query30	0.26	0.06	0.05
query31	2.95	0.40	0.39
query32	3.22	0.47	0.47
query33	2.88	2.92	2.91
query34	16.99	4.43	4.42
query35	4.46	4.45	4.42
query36	0.65	0.48	0.47
query37	0.19	0.16	0.15
query38	0.16	0.14	0.14
query39	0.05	0.03	0.03
query40	0.15	0.13	0.13
query41	0.10	0.04	0.06
query42	0.06	0.05	0.06
query43	0.05	0.04	0.04
Total cold run time: 109.72 s
Total hot run time: 31 s

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 25, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@morningman morningman merged commit 22dc6c9 into apache:master Jul 26, 2024
29 of 32 checks passed
dataroaring pushed a commit that referenced this pull request Jul 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.1-merged kind/behavior-changed reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants