Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](function)adjust aggregate function's nullable property #37330

Merged
merged 10 commits into from
Jul 15, 2024

Conversation

zhangstar333
Copy link
Contributor

@zhangstar333 zhangstar333 commented Jul 5, 2024

Proposed changes

in this PR have change some age function nullable property.
aggregate functions can be divided into 3 types by their nullable properties:

  • AlwaysNullable
  • AlwaysNotNullable
  • NullableAggregateFunction

this PR code of FE part from #37215, the author is @starocean999

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zhangstar333
Copy link
Contributor Author

run buildall

Copy link
Contributor

github-actions bot commented Jul 5, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39888 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8323753c20b73070c5c0e4867b44e6aaa15ec8c0, data reload: false

------ Round 1 ----------------------------------
q1	17610	4522	4348	4348
q2	2028	193	191	191
q3	10462	1152	1108	1108
q4	10198	740	969	740
q5	7509	2685	2620	2620
q6	218	138	138	138
q7	974	598	607	598
q8	9219	2102	2126	2102
q9	8727	6498	6474	6474
q10	8992	3737	3695	3695
q11	458	243	243	243
q12	452	237	235	235
q13	18837	2975	2988	2975
q14	269	235	226	226
q15	542	501	479	479
q16	536	382	380	380
q17	981	629	731	629
q18	8097	7546	7381	7381
q19	3723	1392	1403	1392
q20	650	329	322	322
q21	4950	3840	3272	3272
q22	398	344	340	340
Total cold run time: 115830 ms
Total hot run time: 39888 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4496	4254	4293	4254
q2	355	286	264	264
q3	2953	2811	2876	2811
q4	2019	1793	1692	1692
q5	5666	5525	5479	5479
q6	237	131	128	128
q7	2225	1859	1814	1814
q8	3247	3451	3445	3445
q9	8795	8708	8883	8708
q10	4074	3923	3732	3732
q11	572	501	505	501
q12	811	635	664	635
q13	16235	3181	3153	3153
q14	312	276	298	276
q15	539	504	495	495
q16	502	431	442	431
q17	1860	1543	1489	1489
q18	8132	7982	7860	7860
q19	1844	1570	1653	1570
q20	2206	1871	1859	1859
q21	5124	4963	5018	4963
q22	601	559	565	559
Total cold run time: 72805 ms
Total hot run time: 56118 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173170 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8323753c20b73070c5c0e4867b44e6aaa15ec8c0, data reload: false

query1	921	388	362	362
query2	6453	2330	2344	2330
query3	6646	212	224	212
query4	28634	17231	17173	17173
query5	3699	490	486	486
query6	282	192	170	170
query7	4590	283	291	283
query8	316	284	289	284
query9	8407	2354	2322	2322
query10	587	315	276	276
query11	10678	10041	10060	10041
query12	130	81	87	81
query13	1635	374	361	361
query14	10332	7646	7679	7646
query15	247	187	182	182
query16	7211	309	313	309
query17	1777	538	521	521
query18	1815	291	286	286
query19	201	152	159	152
query20	114	82	79	79
query21	202	122	127	122
query22	4329	4053	4035	4035
query23	33727	33571	33371	33371
query24	11025	2778	2881	2778
query25	627	372	395	372
query26	794	157	158	157
query27	2321	324	314	314
query28	6416	2071	2067	2067
query29	908	645	677	645
query30	257	158	152	152
query31	976	767	771	767
query32	102	58	57	57
query33	763	329	298	298
query34	981	499	503	499
query35	729	632	596	596
query36	1116	976	957	957
query37	142	77	86	77
query38	2942	2870	2803	2803
query39	912	847	831	831
query40	210	123	121	121
query41	55	51	54	51
query42	117	97	101	97
query43	600	545	554	545
query44	1156	725	735	725
query45	195	162	157	157
query46	1063	713	730	713
query47	1842	1766	1783	1766
query48	375	300	309	300
query49	858	424	452	424
query50	773	392	381	381
query51	6883	6685	6695	6685
query52	105	107	92	92
query53	353	288	285	285
query54	888	453	452	452
query55	75	77	72	72
query56	291	263	268	263
query57	1145	1010	1072	1010
query58	263	239	250	239
query59	3610	3196	3315	3196
query60	316	284	277	277
query61	100	97	102	97
query62	615	449	448	448
query63	321	299	298	298
query64	8701	2269	1731	1731
query65	3159	3092	3107	3092
query66	787	319	323	319
query67	15540	14955	14835	14835
query68	4573	516	524	516
query69	602	483	349	349
query70	1191	1086	1150	1086
query71	384	282	278	278
query72	8252	5556	5439	5439
query73	762	331	326	326
query74	6115	5771	5623	5623
query75	3525	2665	2691	2665
query76	2829	975	902	902
query77	679	314	308	308
query78	9523	8769	9039	8769
query79	2046	511	503	503
query80	2163	523	466	466
query81	589	216	221	216
query82	762	106	104	104
query83	282	164	162	162
query84	268	84	88	84
query85	1335	321	303	303
query86	447	327	320	320
query87	3264	3108	3102	3102
query88	4110	2451	2456	2451
query89	479	398	385	385
query90	1813	191	190	190
query91	134	104	104	104
query92	59	48	48	48
query93	2152	501	500	500
query94	1137	208	212	208
query95	400	313	313	313
query96	612	273	269	269
query97	3208	3021	3021	3021
query98	232	200	207	200
query99	1107	859	835	835
Total cold run time: 279374 ms
Total hot run time: 173170 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.86 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8323753c20b73070c5c0e4867b44e6aaa15ec8c0, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.05
query4	1.69	0.08	0.08
query5	0.48	0.48	0.49
query6	1.13	0.72	0.71
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.55	0.50	0.49
query10	0.53	0.54	0.55
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.59	0.58	0.59
query14	0.77	0.79	0.76
query15	0.83	0.80	0.82
query16	0.36	0.37	0.37
query17	0.96	1.03	1.01
query18	0.23	0.24	0.22
query19	1.86	1.80	1.69
query20	0.02	0.01	0.01
query21	15.39	0.75	0.66
query22	4.31	7.07	2.35
query23	18.28	1.33	1.23
query24	2.13	0.23	0.25
query25	0.15	0.09	0.08
query26	0.30	0.20	0.20
query27	0.45	0.23	0.22
query28	13.20	1.00	1.00
query29	12.58	3.35	3.29
query30	0.25	0.06	0.06
query31	2.88	0.37	0.39
query32	3.28	0.48	0.46
query33	2.88	2.98	2.88
query34	17.09	4.33	4.30
query35	4.34	4.41	4.38
query36	0.66	0.50	0.46
query37	0.19	0.15	0.16
query38	0.15	0.14	0.14
query39	0.04	0.03	0.04
query40	0.15	0.13	0.12
query41	0.09	0.04	0.04
query42	0.05	0.05	0.05
query43	0.05	0.03	0.04
Total cold run time: 109.6 s
Total hot run time: 30.86 s

@zhangstar333
Copy link
Contributor Author

run buildall

Copy link
Contributor

github-actions bot commented Jul 5, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39935 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 863166ed3d45248191352e6aa4f2571834a955b2, data reload: false

------ Round 1 ----------------------------------
q1	17619	4374	4306	4306
q2	2033	197	192	192
q3	10437	1157	1151	1151
q4	10198	817	814	814
q5	7497	2689	2638	2638
q6	230	135	136	135
q7	954	599	608	599
q8	9221	2083	2087	2083
q9	9030	6536	6506	6506
q10	9020	3739	3712	3712
q11	466	240	236	236
q12	464	233	229	229
q13	18342	2964	2982	2964
q14	271	231	226	226
q15	526	488	493	488
q16	529	384	368	368
q17	980	641	667	641
q18	8045	7478	7291	7291
q19	5490	1425	1538	1425
q20	710	326	342	326
q21	4952	3858	3264	3264
q22	393	341	347	341
Total cold run time: 117407 ms
Total hot run time: 39935 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4396	4237	4256	4237
q2	367	272	272	272
q3	2995	2919	2834	2834
q4	1992	1708	1718	1708
q5	5665	5515	5465	5465
q6	226	130	130	130
q7	2171	1889	1861	1861
q8	3299	3427	3482	3427
q9	8763	8770	8782	8770
q10	4078	3839	3728	3728
q11	590	517	499	499
q12	821	637	636	636
q13	15895	3164	3211	3164
q14	305	270	258	258
q15	550	494	490	490
q16	504	444	424	424
q17	1863	1542	1502	1502
q18	8252	8085	7823	7823
q19	1815	1736	1510	1510
q20	2137	1863	1867	1863
q21	5120	4823	4718	4718
q22	635	582	580	580
Total cold run time: 72439 ms
Total hot run time: 55899 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173261 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 863166ed3d45248191352e6aa4f2571834a955b2, data reload: false

query1	928	368	372	368
query2	6444	2370	2313	2313
query3	6646	210	222	210
query4	28263	17300	17253	17253
query5	3639	486	485	485
query6	262	195	182	182
query7	4579	297	288	288
query8	331	298	302	298
query9	8655	2427	2434	2427
query10	568	324	275	275
query11	11493	10075	10101	10075
query12	114	83	89	83
query13	1630	368	366	366
query14	10009	6908	7624	6908
query15	246	183	185	183
query16	7657	299	310	299
query17	1558	556	527	527
query18	1933	292	265	265
query19	191	147	145	145
query20	89	82	82	82
query21	209	125	121	121
query22	4445	4074	4029	4029
query23	34009	33564	33601	33564
query24	10991	2833	2870	2833
query25	589	411	368	368
query26	708	157	156	156
query27	2247	317	332	317
query28	6492	2139	2169	2139
query29	890	615	630	615
query30	259	153	157	153
query31	953	756	749	749
query32	92	55	54	54
query33	729	295	304	295
query34	889	491	490	490
query35	749	632	626	626
query36	1132	994	981	981
query37	139	78	82	78
query38	2937	2877	2835	2835
query39	922	833	805	805
query40	205	130	123	123
query41	60	50	50	50
query42	119	101	101	101
query43	604	553	598	553
query44	1174	740	739	739
query45	194	165	162	162
query46	1080	714	738	714
query47	1912	1770	1779	1770
query48	373	295	297	295
query49	824	426	404	404
query50	769	381	378	378
query51	6879	6850	6736	6736
query52	107	97	98	97
query53	359	288	288	288
query54	923	437	441	437
query55	76	73	72	72
query56	293	257	257	257
query57	1144	1051	1060	1051
query58	260	242	263	242
query59	3499	3157	3399	3157
query60	293	274	277	274
query61	95	92	95	92
query62	595	454	454	454
query63	317	283	291	283
query64	9095	2155	1663	1663
query65	3152	3083	3079	3079
query66	787	320	326	320
query67	15462	15052	15036	15036
query68	6107	534	537	534
query69	730	416	349	349
query70	1217	1129	1167	1129
query71	449	275	281	275
query72	7403	5489	5652	5489
query73	782	323	322	322
query74	6077	5654	5660	5654
query75	3727	2684	2667	2667
query76	3235	962	935	935
query77	626	303	299	299
query78	11642	9354	8896	8896
query79	6059	514	513	513
query80	1277	472	463	463
query81	588	218	224	218
query82	878	110	106	106
query83	248	171	170	170
query84	280	92	91	91
query85	701	316	296	296
query86	469	311	312	311
query87	3354	3132	3081	3081
query88	5071	2351	2445	2351
query89	492	377	373	373
query90	1809	184	187	184
query91	132	103	105	103
query92	62	49	50	49
query93	4828	509	509	509
query94	1093	208	206	206
query95	404	311	306	306
query96	594	268	267	267
query97	3263	3022	3061	3022
query98	226	195	199	195
query99	1142	831	850	831
Total cold run time: 290231 ms
Total hot run time: 173261 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.49 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 863166ed3d45248191352e6aa4f2571834a955b2, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.23	0.06	0.04
query4	1.67	0.06	0.07
query5	0.48	0.49	0.50
query6	1.14	0.73	0.74
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.55	0.55	0.55
query11	0.17	0.11	0.11
query12	0.14	0.12	0.12
query13	0.58	0.59	0.58
query14	0.75	0.79	0.80
query15	0.84	0.81	0.82
query16	0.34	0.37	0.35
query17	1.02	0.98	1.00
query18	0.23	0.26	0.24
query19	1.82	1.71	1.70
query20	0.01	0.01	0.02
query21	15.39	0.77	0.66
query22	3.67	8.20	1.87
query23	18.27	1.42	1.25
query24	2.17	0.22	0.21
query25	0.15	0.08	0.08
query26	0.28	0.21	0.21
query27	0.45	0.22	0.23
query28	13.25	1.01	1.00
query29	12.62	3.34	3.30
query30	0.26	0.06	0.05
query31	2.89	0.38	0.38
query32	3.28	0.47	0.46
query33	2.88	2.87	2.91
query34	17.24	4.32	4.39
query35	4.40	4.42	4.44
query36	0.66	0.47	0.47
query37	0.18	0.16	0.15
query38	0.16	0.14	0.14
query39	0.04	0.03	0.03
query40	0.16	0.12	0.12
query41	0.09	0.05	0.04
query42	0.06	0.04	0.04
query43	0.04	0.03	0.04
Total cold run time: 109.3 s
Total hot run time: 30.49 s

@zhangstar333
Copy link
Contributor Author

run buildall

Copy link
Contributor

github-actions bot commented Jul 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40189 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 203c0881a4c4b3cf3214979c238853d2120344ae, data reload: false

------ Round 1 ----------------------------------
q1	17606	5113	4313	4313
q2	2013	193	198	193
q3	10435	1258	1089	1089
q4	10205	900	779	779
q5	7493	2662	2656	2656
q6	220	138	137	137
q7	956	611	615	611
q8	9233	2089	2091	2089
q9	9091	6485	6513	6485
q10	8970	3719	3717	3717
q11	449	236	234	234
q12	447	232	233	232
q13	17781	3022	2977	2977
q14	272	232	227	227
q15	523	507	492	492
q16	497	383	392	383
q17	963	643	698	643
q18	8139	7578	7525	7525
q19	6224	1522	1500	1500
q20	709	321	319	319
q21	4884	3858	3245	3245
q22	399	343	352	343
Total cold run time: 117509 ms
Total hot run time: 40189 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4391	4292	4247	4247
q2	353	268	267	267
q3	3025	2913	2931	2913
q4	2034	1688	1736	1688
q5	5603	5472	5439	5439
q6	220	130	130	130
q7	2285	1912	1849	1849
q8	3294	3466	3445	3445
q9	8767	8828	8768	8768
q10	4158	3723	3826	3723
q11	606	498	504	498
q12	835	689	633	633
q13	16994	3146	3150	3146
q14	310	274	267	267
q15	534	484	475	475
q16	482	449	433	433
q17	1823	1555	1495	1495
q18	8163	8098	8030	8030
q19	1830	1705	1610	1610
q20	2161	1822	1848	1822
q21	6841	4756	4766	4756
q22	642	552	525	525
Total cold run time: 75351 ms
Total hot run time: 56159 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 171730 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 203c0881a4c4b3cf3214979c238853d2120344ae, data reload: false

query1	908	369	376	369
query2	6625	2621	2503	2503
query3	6632	206	213	206
query4	27470	17333	17273	17273
query5	3767	478	484	478
query6	273	173	180	173
query7	4582	296	291	291
query8	319	293	299	293
query9	8547	2374	2352	2352
query10	564	323	277	277
query11	11833	10040	10086	10040
query12	117	84	85	84
query13	1673	385	374	374
query14	10246	7683	7895	7683
query15	241	194	185	185
query16	7701	319	319	319
query17	1797	565	529	529
query18	1973	283	289	283
query19	205	152	160	152
query20	89	85	86	85
query21	209	133	134	133
query22	4570	4157	4135	4135
query23	33939	33680	33636	33636
query24	11049	2871	2820	2820
query25	621	438	406	406
query26	875	161	160	160
query27	2349	328	355	328
query28	6499	2162	2134	2134
query29	910	673	641	641
query30	249	159	148	148
query31	963	771	798	771
query32	101	53	56	53
query33	764	309	314	309
query34	1036	500	498	498
query35	750	642	648	642
query36	1138	990	987	987
query37	134	81	87	81
query38	3070	2840	2822	2822
query39	866	815	816	815
query40	209	125	131	125
query41	57	54	57	54
query42	115	99	100	99
query43	597	596	587	587
query44	1144	727	751	727
query45	196	164	160	160
query46	1080	712	696	696
query47	1876	1770	1799	1770
query48	383	303	310	303
query49	936	397	413	397
query50	764	377	396	377
query51	6807	6883	6759	6759
query52	111	92	87	87
query53	360	288	286	286
query54	937	471	440	440
query55	73	69	72	69
query56	292	268	260	260
query57	1124	1068	1063	1063
query58	244	225	239	225
query59	3585	3275	3173	3173
query60	304	273	289	273
query61	95	96	94	94
query62	604	462	436	436
query63	315	287	284	284
query64	9567	2198	1607	1607
query65	3155	3076	3080	3076
query66	705	319	332	319
query67	15652	15001	14919	14919
query68	7505	539	552	539
query69	730	441	349	349
query70	1196	1127	1151	1127
query71	519	287	276	276
query72	9086	5270	2779	2779
query73	801	320	320	320
query74	6210	5657	5660	5657
query75	5157	2654	2722	2654
query76	5055	1007	1003	1003
query77	804	302	282	282
query78	9732	8853	9011	8853
query79	6835	509	513	509
query80	1073	475	481	475
query81	571	222	225	222
query82	812	105	101	101
query83	380	173	170	170
query84	274	88	88	88
query85	1371	299	304	299
query86	455	309	306	306
query87	3308	3091	3089	3089
query88	4900	2475	2520	2475
query89	489	383	372	372
query90	1971	185	182	182
query91	131	102	101	101
query92	61	49	49	49
query93	5491	511	519	511
query94	1223	207	209	207
query95	397	311	313	311
query96	622	269	274	269
query97	3176	2990	3009	2990
query98	211	198	197	197
query99	1110	835	842	835
Total cold run time: 298597 ms
Total hot run time: 171730 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.45 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 203c0881a4c4b3cf3214979c238853d2120344ae, data reload: false

query1	0.05	0.03	0.03
query2	0.08	0.03	0.03
query3	0.22	0.05	0.06
query4	1.66	0.07	0.10
query5	0.49	0.48	0.50
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.04	0.04	0.04
query9	0.55	0.49	0.49
query10	0.55	0.54	0.55
query11	0.15	0.11	0.12
query12	0.15	0.13	0.13
query13	0.60	0.58	0.57
query14	0.79	0.77	0.80
query15	0.85	0.82	0.83
query16	0.36	0.36	0.37
query17	1.02	1.05	0.97
query18	0.23	0.21	0.25
query19	1.74	1.72	1.71
query20	0.01	0.01	0.02
query21	15.39	0.77	0.66
query22	4.15	7.18	1.75
query23	18.29	1.44	1.32
query24	2.08	0.23	0.22
query25	0.16	0.08	0.08
query26	0.29	0.22	0.21
query27	0.46	0.23	0.23
query28	13.34	1.01	0.99
query29	12.60	3.31	3.27
query30	0.26	0.06	0.05
query31	2.87	0.37	0.37
query32	3.31	0.47	0.46
query33	2.88	2.88	2.90
query34	17.23	4.35	4.34
query35	4.43	4.45	4.47
query36	0.65	0.46	0.49
query37	0.17	0.15	0.15
query38	0.15	0.14	0.14
query39	0.04	0.04	0.03
query40	0.15	0.13	0.12
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.79 s
Total hot run time: 30.45 s

Copy link
Contributor

github-actions bot commented Jul 8, 2024

clang-tidy review says "All clean, LGTM! 👍"

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40306 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a2ab3616ee0a6947291c080660697a78465056c5, data reload: false

------ Round 1 ----------------------------------
q1	17600	5078	4297	4297
q2	2034	189	192	189
q3	10435	1279	1153	1153
q4	10191	787	776	776
q5	7483	2678	2625	2625
q6	223	139	140	139
q7	951	606	617	606
q8	9228	2090	2139	2090
q9	8909	6533	6509	6509
q10	8974	3746	3720	3720
q11	457	242	243	242
q12	492	232	234	232
q13	17766	2986	2965	2965
q14	275	226	225	225
q15	537	486	487	486
q16	509	391	382	382
q17	984	681	696	681
q18	8003	7562	7519	7519
q19	4123	1511	1499	1499
q20	671	326	328	326
q21	5000	3296	3987	3296
q22	402	349	349	349
Total cold run time: 115247 ms
Total hot run time: 40306 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4338	4236	4205	4205
q2	370	260	277	260
q3	2998	2828	2903	2828
q4	2045	1762	1725	1725
q5	5703	5519	5464	5464
q6	228	132	133	132
q7	2180	1915	1881	1881
q8	3262	3404	3447	3404
q9	8708	8662	8839	8662
q10	4127	3848	3735	3735
q11	594	515	514	514
q12	853	609	650	609
q13	17133	3166	3135	3135
q14	311	264	284	264
q15	545	493	477	477
q16	481	424	432	424
q17	1812	1564	1519	1519
q18	8199	7905	7853	7853
q19	1783	1813	1631	1631
q20	2095	1875	1901	1875
q21	5151	4946	4829	4829
q22	614	581	591	581
Total cold run time: 73530 ms
Total hot run time: 56007 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174645 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a2ab3616ee0a6947291c080660697a78465056c5, data reload: false

query1	909	373	366	366
query2	6436	2572	2387	2387
query3	6631	202	211	202
query4	27116	17413	17213	17213
query5	3676	495	477	477
query6	265	173	152	152
query7	4572	305	294	294
query8	322	321	301	301
query9	8508	2420	2405	2405
query10	572	305	283	283
query11	12760	9947	10032	9947
query12	117	85	81	81
query13	1668	396	382	382
query14	10229	7633	7742	7633
query15	240	194	193	193
query16	7789	310	315	310
query17	1811	575	539	539
query18	1856	288	291	288
query19	197	155	157	155
query20	89	82	83	82
query21	209	138	126	126
query22	4314	4065	3962	3962
query23	34140	33653	33673	33653
query24	10904	2816	2850	2816
query25	616	421	407	407
query26	696	151	152	151
query27	2212	283	285	283
query28	6117	2230	2201	2201
query29	903	681	651	651
query30	266	155	156	155
query31	968	787	762	762
query32	95	61	59	59
query33	689	310	320	310
query34	918	490	506	490
query35	705	626	618	618
query36	1134	955	991	955
query37	143	83	89	83
query38	2936	2769	2752	2752
query39	883	839	822	822
query40	202	126	122	122
query41	56	55	54	54
query42	122	95	105	95
query43	583	548	563	548
query44	1094	746	751	746
query45	196	165	167	165
query46	1077	739	728	728
query47	1842	1753	1760	1753
query48	379	310	315	310
query49	941	406	431	406
query50	769	402	393	393
query51	6962	6832	6904	6832
query52	108	94	89	89
query53	364	283	294	283
query54	890	446	448	446
query55	74	73	72	72
query56	292	266	271	266
query57	1121	1035	1026	1026
query58	266	263	240	240
query59	3544	3040	3197	3040
query60	309	292	278	278
query61	95	92	93	92
query62	796	640	622	622
query63	323	289	285	285
query64	9125	2206	1636	1636
query65	3164	3115	3080	3080
query66	741	326	332	326
query67	15698	15020	14969	14969
query68	6204	558	560	558
query69	750	450	385	385
query70	1188	1048	1100	1048
query71	533	290	277	277
query72	9042	5475	5696	5475
query73	830	326	323	323
query74	6106	5693	5735	5693
query75	5002	2628	2672	2628
query76	4735	1064	921	921
query77	781	299	296	296
query78	9653	9024	8987	8987
query79	8417	520	515	515
query80	1074	483	535	483
query81	579	219	218	218
query82	853	136	137	136
query83	321	173	164	164
query84	274	86	97	86
query85	1333	325	301	301
query86	443	313	317	313
query87	3280	3045	3136	3045
query88	4619	2458	2474	2458
query89	561	397	394	394
query90	1962	191	184	184
query91	136	111	105	105
query92	59	50	49	49
query93	6402	526	525	525
query94	1291	207	213	207
query95	446	307	316	307
query96	626	273	274	273
query97	3229	2990	3034	2990
query98	217	202	211	202
query99	1543	1251	1257	1251
Total cold run time: 298233 ms
Total hot run time: 174645 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a2ab3616ee0a6947291c080660697a78465056c5, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.04	0.04
query3	0.22	0.04	0.04
query4	1.68	0.07	0.07
query5	0.51	0.50	0.50
query6	1.13	0.72	0.73
query7	0.03	0.01	0.01
query8	0.06	0.04	0.04
query9	0.55	0.49	0.50
query10	0.54	0.53	0.56
query11	0.16	0.11	0.11
query12	0.15	0.13	0.12
query13	0.59	0.59	0.58
query14	0.76	0.77	0.79
query15	0.86	0.82	0.83
query16	0.36	0.35	0.37
query17	1.01	0.99	0.97
query18	0.23	0.22	0.22
query19	1.79	1.70	1.82
query20	0.01	0.01	0.01
query21	15.38	0.74	0.66
query22	4.53	7.00	1.69
query23	18.28	1.41	1.26
query24	2.30	0.24	0.23
query25	0.15	0.09	0.08
query26	0.30	0.21	0.21
query27	0.45	0.23	0.22
query28	13.26	1.03	0.99
query29	12.57	3.26	3.29
query30	0.25	0.06	0.05
query31	2.89	0.38	0.39
query32	3.26	0.47	0.46
query33	2.92	2.95	2.92
query34	17.09	4.34	4.35
query35	4.46	4.40	4.44
query36	0.64	0.47	0.48
query37	0.18	0.16	0.15
query38	0.15	0.15	0.15
query39	0.04	0.03	0.03
query40	0.15	0.13	0.12
query41	0.09	0.05	0.06
query42	0.06	0.04	0.05
query43	0.04	0.04	0.03
Total cold run time: 110.19 s
Total hot run time: 30.31 s

@zhangstar333
Copy link
Contributor Author

run buildall

Copy link
Contributor

github-actions bot commented Jul 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39921 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit acadf88a0563bb4d0c3a10380f52f77a618bfb7e, data reload: false

------ Round 1 ----------------------------------
q1	17622	4330	4262	4262
q2	2016	195	192	192
q3	10454	1170	1023	1023
q4	10179	816	807	807
q5	7588	2670	2633	2633
q6	226	144	150	144
q7	992	612	614	612
q8	9522	2110	2065	2065
q9	8638	6463	6460	6460
q10	8923	3687	3702	3687
q11	476	248	237	237
q12	403	237	225	225
q13	17780	2993	3006	2993
q14	281	233	236	233
q15	539	503	495	495
q16	521	371	379	371
q17	958	642	695	642
q18	8138	7514	7428	7428
q19	3060	1499	1515	1499
q20	688	329	353	329
q21	4925	3242	3985	3242
q22	401	344	342	342
Total cold run time: 114330 ms
Total hot run time: 39921 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4326	4263	4285	4263
q2	361	276	273	273
q3	2928	2722	2749	2722
q4	1870	1594	1593	1593
q5	5260	5284	5310	5284
q6	222	131	134	131
q7	2075	1755	1719	1719
q8	3223	3378	3325	3325
q9	8368	8356	8346	8346
q10	3896	3670	3650	3650
q11	587	495	517	495
q12	788	637	622	622
q13	16367	3015	2996	2996
q14	300	258	283	258
q15	519	465	483	465
q16	492	426	417	417
q17	1788	1477	1470	1470
q18	7729	7682	7208	7208
q19	2514	1579	1582	1579
q20	1985	1818	1793	1793
q21	4829	4853	4712	4712
q22	622	572	570	570
Total cold run time: 71049 ms
Total hot run time: 53891 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173601 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit acadf88a0563bb4d0c3a10380f52f77a618bfb7e, data reload: false

query1	925	373	364	364
query2	6456	2373	2381	2373
query3	6651	211	226	211
query4	24338	17773	17320	17320
query5	4188	500	493	493
query6	288	199	168	168
query7	4596	300	289	289
query8	312	295	294	294
query9	8512	2403	2390	2390
query10	618	295	268	268
query11	11609	10121	10159	10121
query12	133	83	80	80
query13	1650	370	366	366
query14	10322	7047	7688	7047
query15	248	183	183	183
query16	7686	306	313	306
query17	1591	536	541	536
query18	1739	292	288	288
query19	203	159	156	156
query20	92	83	82	82
query21	237	133	126	126
query22	4251	4215	3982	3982
query23	33896	33142	33201	33142
query24	12185	2949	2809	2809
query25	686	395	391	391
query26	1851	155	150	150
query27	2958	267	275	267
query28	7711	2052	2051	2051
query29	1169	626	621	621
query30	304	149	145	145
query31	950	739	746	739
query32	94	53	56	53
query33	779	291	290	290
query34	1020	474	487	474
query35	699	590	587	587
query36	1069	973	964	964
query37	276	79	81	79
query38	2907	2761	2749	2749
query39	868	820	824	820
query40	284	127	123	123
query41	55	57	54	54
query42	118	99	103	99
query43	576	546	559	546
query44	1239	738	723	723
query45	189	166	161	161
query46	1098	723	744	723
query47	1838	1789	1778	1778
query48	359	290	299	290
query49	1194	425	409	409
query50	787	404	401	401
query51	6903	6782	6823	6782
query52	107	96	96	96
query53	366	300	296	296
query54	986	453	440	440
query55	75	76	75	75
query56	283	269	277	269
query57	1168	1044	1031	1031
query58	249	258	263	258
query59	3635	3217	3230	3217
query60	312	274	280	274
query61	99	95	95	95
query62	855	658	658	658
query63	321	294	304	294
query64	10457	2171	1670	1670
query65	3216	3150	3094	3094
query66	1367	362	339	339
query67	15609	15156	14864	14864
query68	5362	538	540	538
query69	646	490	354	354
query70	1149	1184	1170	1170
query71	454	280	287	280
query72	7058	5493	5859	5493
query73	779	323	322	322
query74	6126	5708	5718	5708
query75	3752	2702	2658	2658
query76	3607	887	945	887
query77	673	312	326	312
query78	9631	9666	8897	8897
query79	2090	516	496	496
query80	1731	483	472	472
query81	595	219	220	219
query82	1115	139	141	139
query83	291	176	170	170
query84	268	90	98	90
query85	1680	369	315	315
query86	488	335	297	297
query87	3274	3137	3105	3105
query88	3945	2347	2340	2340
query89	492	393	395	393
query90	1952	192	194	192
query91	139	108	105	105
query92	68	48	55	48
query93	2112	517	506	506
query94	1317	219	216	216
query95	408	317	322	317
query96	607	281	275	275
query97	3202	3060	3011	3011
query98	221	206	195	195
query99	1712	1277	1248	1248
Total cold run time: 286756 ms
Total hot run time: 173601 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit acadf88a0563bb4d0c3a10380f52f77a618bfb7e, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.05
query4	1.67	0.10	0.08
query5	0.50	0.49	0.48
query6	1.13	0.72	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.50	0.50
query10	0.55	0.56	0.55
query11	0.16	0.12	0.11
query12	0.14	0.12	0.12
query13	0.58	0.59	0.58
query14	0.80	0.78	0.79
query15	0.85	0.83	0.82
query16	0.37	0.36	0.36
query17	0.97	0.97	1.00
query18	0.24	0.22	0.22
query19	1.85	1.71	1.75
query20	0.01	0.01	0.01
query21	15.40	0.75	0.65
query22	4.52	6.57	2.14
query23	18.65	1.35	1.40
query24	2.13	0.23	0.22
query25	0.14	0.09	0.09
query26	0.30	0.21	0.20
query27	0.45	0.23	0.23
query28	13.25	1.01	1.00
query29	12.64	3.36	3.33
query30	0.26	0.06	0.06
query31	2.89	0.39	0.39
query32	3.27	0.49	0.47
query33	2.90	2.87	2.94
query34	17.05	4.34	4.40
query35	4.47	4.42	4.42
query36	0.66	0.50	0.49
query37	0.19	0.15	0.15
query38	0.15	0.15	0.14
query39	0.05	0.03	0.04
query40	0.15	0.13	0.11
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.5 s
Total hot run time: 31 s

@zhangstar333
Copy link
Contributor Author

run buildall

Copy link
Contributor

github-actions bot commented Jul 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39922 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8755ea6a9cce224285195a87fea0103a7f33fa99, data reload: false

------ Round 1 ----------------------------------
q1	17608	4568	4307	4307
q2	2016	186	191	186
q3	10473	1206	1167	1167
q4	10170	874	787	787
q5	7498	2667	2637	2637
q6	222	136	138	136
q7	959	605	603	603
q8	9231	2093	2092	2092
q9	9046	6487	6485	6485
q10	8889	3733	3688	3688
q11	448	237	234	234
q12	433	233	236	233
q13	17756	2978	2975	2975
q14	256	233	222	222
q15	538	479	463	463
q16	500	372	373	372
q17	968	739	663	663
q18	8143	7447	7378	7378
q19	7185	1525	1507	1507
q20	660	316	343	316
q21	4867	3127	3810	3127
q22	391	345	344	344
Total cold run time: 118257 ms
Total hot run time: 39922 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4420	4297	4309	4297
q2	368	255	273	255
q3	3006	2891	2945	2891
q4	1977	1736	1709	1709
q5	5498	5515	5425	5425
q6	224	150	140	140
q7	2289	1894	1879	1879
q8	3269	3470	3421	3421
q9	8648	8798	8700	8700
q10	4121	3766	3815	3766
q11	598	504	500	500
q12	828	652	653	652
q13	16203	3178	3189	3178
q14	284	279	274	274
q15	523	488	503	488
q16	470	437	420	420
q17	1816	1536	1504	1504
q18	8317	7872	7909	7872
q19	1789	1669	1641	1641
q20	2875	1869	1824	1824
q21	5221	4805	4879	4805
q22	618	542	586	542
Total cold run time: 73362 ms
Total hot run time: 56183 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174565 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8755ea6a9cce224285195a87fea0103a7f33fa99, data reload: false

query1	924	377	366	366
query2	6400	2400	2383	2383
query3	6651	207	215	207
query4	28187	17393	17611	17393
query5	3652	474	482	474
query6	263	178	156	156
query7	4584	286	290	286
query8	328	292	294	292
query9	8706	2386	2366	2366
query10	437	306	273	273
query11	11924	10116	10078	10078
query12	115	91	82	82
query13	1652	394	389	389
query14	9450	7851	7532	7532
query15	239	186	183	183
query16	7609	313	312	312
query17	1799	586	539	539
query18	1914	295	285	285
query19	209	158	157	157
query20	94	84	81	81
query21	207	126	172	126
query22	4349	4135	4177	4135
query23	34139	33718	33621	33621
query24	11022	2811	2870	2811
query25	618	397	429	397
query26	1099	150	153	150
query27	2891	284	293	284
query28	8048	2137	2112	2112
query29	886	656	646	646
query30	253	149	152	149
query31	1012	760	765	760
query32	92	53	60	53
query33	758	303	290	290
query34	889	498	517	498
query35	683	585	564	564
query36	1140	965	987	965
query37	151	81	81	81
query38	2970	2895	2808	2808
query39	860	795	802	795
query40	210	123	118	118
query41	54	49	51	49
query42	122	101	102	101
query43	585	522	547	522
query44	1231	732	753	732
query45	190	161	161	161
query46	1084	714	707	707
query47	1857	1773	1796	1773
query48	373	300	298	298
query49	867	409	415	409
query50	786	402	398	398
query51	6900	6779	6797	6779
query52	109	92	94	92
query53	365	297	300	297
query54	853	455	447	447
query55	75	72	73	72
query56	285	270	279	270
query57	1144	1053	1063	1053
query58	250	255	255	255
query59	3443	3233	3171	3171
query60	324	283	285	283
query61	132	94	95	94
query62	823	648	640	640
query63	318	291	286	286
query64	9517	2196	1621	1621
query65	3166	3068	3111	3068
query66	743	323	323	323
query67	15465	15125	14977	14977
query68	4436	543	552	543
query69	552	418	341	341
query70	1193	1146	1163	1146
query71	431	298	281	281
query72	7643	5354	5651	5354
query73	762	322	326	322
query74	6130	5618	5693	5618
query75	3406	2697	2681	2681
query76	2782	903	995	903
query77	477	303	307	303
query78	11136	8831	8760	8760
query79	3440	528	526	526
query80	1911	488	470	470
query81	603	214	220	214
query82	971	148	132	132
query83	314	171	167	167
query84	274	91	86	86
query85	1994	312	360	312
query86	463	323	301	301
query87	3335	3104	3093	3093
query88	4599	2462	2485	2462
query89	496	385	389	385
query90	1848	189	188	188
query91	132	107	104	104
query92	63	50	48	48
query93	4880	522	506	506
query94	920	211	213	211
query95	399	319	314	314
query96	619	272	271	271
query97	3199	3036	2981	2981
query98	220	197	194	194
query99	1673	1273	1297	1273
Total cold run time: 289776 ms
Total hot run time: 174565 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.01 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8755ea6a9cce224285195a87fea0103a7f33fa99, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.03
query3	0.23	0.05	0.05
query4	1.68	0.07	0.08
query5	0.53	0.49	0.49
query6	1.14	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.05	0.05
query9	0.55	0.52	0.49
query10	0.53	0.54	0.53
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.60	0.59	0.58
query14	0.76	0.77	0.79
query15	0.86	0.81	0.80
query16	0.37	0.36	0.37
query17	1.01	0.95	1.03
query18	0.23	0.22	0.22
query19	1.82	1.72	1.67
query20	0.02	0.01	0.01
query21	15.39	0.75	0.65
query22	4.17	6.70	2.48
query23	18.32	1.39	1.22
query24	2.05	0.21	0.23
query25	0.15	0.09	0.09
query26	0.29	0.22	0.21
query27	0.45	0.23	0.23
query28	13.32	1.01	1.00
query29	12.59	3.31	3.27
query30	0.25	0.06	0.05
query31	2.88	0.40	0.38
query32	3.27	0.48	0.47
query33	2.89	2.96	2.91
query34	16.84	4.38	4.34
query35	4.47	4.39	4.45
query36	0.65	0.46	0.49
query37	0.19	0.16	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.03
query40	0.16	0.12	0.12
query41	0.09	0.05	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.05
Total cold run time: 109.52 s
Total hot run time: 31.01 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 11, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@zhangstar333 zhangstar333 changed the title [test](Agg nullable) [fix](function)adjust aggregate function's nullable property Jul 11, 2024
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhangstar333 zhangstar333 merged commit 0816969 into apache:master Jul 15, 2024
28 of 32 checks passed
seawinde pushed a commit to seawinde/doris that referenced this pull request Jul 17, 2024
…37330)

## Proposed changes

in this PR have change some age function nullable property.
aggregate functions can be divided into 3 types by their nullable
properties:
- AlwaysNullable
- AlwaysNotNullable
- NullableAggregateFunction

this PR code of FE part from apache#37215,
the author is @starocean999
---------
Co-authored-by: starocean999 <[email protected]>
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
## Proposed changes

in this PR have change some age function nullable property.
aggregate functions can be divided into 3 types by their nullable
properties:
- AlwaysNullable
- AlwaysNotNullable
- NullableAggregateFunction

this PR code of FE part from #37215,
the author is @starocean999
---------
Co-authored-by: starocean999 <[email protected]>
yiguolei pushed a commit that referenced this pull request Aug 15, 2024
## Proposed changes

Issue Number: close #xxx

before `corr(nullable_x, nullable_y)` will core dump. not fixed.
no need to patch in master because the refactor
#37330 already changed the
implementation context
yiguolei pushed a commit that referenced this pull request Sep 11, 2024
nullable property of histogram on master is changed by
#37330
pick it to branch-2.1
related change on 2.1: #38608,
#38608 relies on AlwaysNotNullable property.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants