[pipelineX](localshuffle) Adjust local exchange strategy for distinct streaming agg

[Gabriel39]

Proposed changes

For preagg, data can be passthrough through local exchange which has lower overhead

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

[doris-robot]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

[Gabriel39]

run buildall

[github-actions]

clang-tidy review says "All clean, LGTM! 👍"

[doris-robot]

TeamCity be ut coverage result:
Function Coverage: 35.26% (8733/24769)
Line Coverage: 27.05% (71516/264338)
Region Coverage: 26.29% (37104/141125)
Branch Coverage: 23.18% (18972/81836)
Coverage Report: http://coverage.selectdb-in.cc/coverage/030c29e073dfee58e024bce51bd0455f68089369_030c29e073dfee58e024bce51bd0455f68089369/report/index.html

[doris-robot]

TPC-H: Total hot run time: 38126 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 030c29e073dfee58e024bce51bd0455f68089369, data reload: false

------ Round 1 ----------------------------------
q1	17607	4424	4138	4138
q2	2110	162	153	153
q3	10625	1145	1217	1145
q4	10537	836	842	836
q5	7600	3055	3004	3004
q6	199	121	122	121
q7	1044	601	586	586
q8	9344	2042	1996	1996
q9	7227	6610	6625	6610
q10	8330	3416	3581	3416
q11	423	231	217	217
q12	370	209	195	195
q13	17782	2826	2855	2826
q14	239	210	206	206
q15	507	465	453	453
q16	466	375	375	375
q17	959	595	562	562
q18	7351	6459	6441	6441
q19	1568	1435	1486	1435
q20	579	273	255	255
q21	3678	2985	2866	2866
q22	325	290	297	290
Total cold run time: 108870 ms
Total hot run time: 38126 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4097	4077	4065	4065
q2	330	239	233	233
q3	2996	2878	2845	2845
q4	1839	1560	1585	1560
q5	5341	5372	5326	5326
q6	195	116	116	116
q7	2294	1825	1850	1825
q8	3179	3313	3274	3274
q9	8714	8681	8654	8654
q10	3826	3788	3763	3763
q11	536	467	453	453
q12	737	534	560	534
q13	16916	2836	2818	2818
q14	277	257	249	249
q15	496	445	454	445
q16	462	422	404	404
q17	1744	1521	1479	1479
q18	7494	7202	7136	7136
q19	1603	1546	1529	1529
q20	1891	1758	1737	1737
q21	4964	4589	4796	4589
q22	536	459	443	443
Total cold run time: 70467 ms
Total hot run time: 53477 ms

[doris-robot]

TPC-DS: Total hot run time: 181374 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 030c29e073dfee58e024bce51bd0455f68089369, data reload: false

query1	938	372	355	355
query2	7371	1980	1895	1895
query3	6704	208	215	208
query4	32243	21317	21226	21226
query5	4296	399	406	399
query6	272	172	170	170
query7	4617	286	293	286
query8	234	180	177	177
query9	9298	2277	2273	2273
query10	553	271	267	267
query11	17330	14336	14453	14336
query12	133	92	86	86
query13	1629	409	414	409
query14	10880	8223	7113	7113
query15	281	199	189	189
query16	8238	264	257	257
query17	2038	590	556	556
query18	2111	293	282	282
query19	344	157	160	157
query20	91	91	91	91
query21	199	129	128	128
query22	5116	4854	4846	4846
query23	33398	32757	32796	32757
query24	12012	2940	2865	2865
query25	664	380	388	380
query26	1767	152	152	152
query27	2971	353	352	352
query28	7417	1870	1860	1860
query29	1048	663	620	620
query30	307	160	148	148
query31	975	765	746	746
query32	93	59	59	59
query33	775	263	252	252
query34	1105	480	490	480
query35	839	600	606	600
query36	1004	866	879	866
query37	280	68	66	66
query38	3559	3450	3430	3430
query39	1469	1425	1417	1417
query40	286	117	113	113
query41	49	47	50	47
query42	108	95	99	95
query43	496	448	439	439
query44	1140	729	727	727
query45	305	284	272	272
query46	1109	722	699	699
query47	1928	1914	1854	1854
query48	445	357	349	349
query49	1233	335	334	334
query50	766	377	376	376
query51	6763	6694	6637	6637
query52	106	87	94	87
query53	344	272	278	272
query54	337	245	241	241
query55	89	83	83	83
query56	258	243	243	243
query57	1260	1165	1169	1165
query58	238	204	218	204
query59	2664	2566	2486	2486
query60	277	255	251	251
query61	113	113	114	113
query62	689	455	449	449
query63	304	276	278	276
query64	6762	4050	3918	3918
query65	3075	3016	3012	3012
query66	1423	364	357	357
query67	15379	14783	14828	14783
query68	8596	518	527	518
query69	624	382	383	382
query70	1198	1169	1130	1130
query71	516	280	280	280
query72	6294	2722	2528	2528
query73	738	313	331	313
query74	7495	6693	6693	6693
query75	3571	2212	2229	2212
query76	5233	931	944	931
query77	647	263	251	251
query78	10996	10150	10274	10150
query79	10179	519	513	513
query80	1945	395	385	385
query81	506	212	211	211
query82	748	83	84	83
query83	222	138	144	138
query84	285	79	80	79
query85	1467	315	312	312
query86	413	317	294	294
query87	3684	3510	3497	3497
query88	4884	2276	2272	2272
query89	501	370	367	367
query90	2013	176	173	173
query91	166	135	137	135
query92	62	49	47	47
query93	7254	491	481	481
query94	1211	175	176	175
query95	426	322	328	322
query96	607	277	271	271
query97	2638	2501	2479	2479
query98	226	207	222	207
query99	1257	946	907	907
Total cold run time: 319147 ms
Total hot run time: 181374 ms

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 030c29e073dfee58e024bce51bd0455f68089369 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       20.9 seconds inserted 10000000 Rows, about 478K ops/s

[github-actions]

PR approved by anyone and no changes requested.

[github-actions]

PR approved by at least one committer and no changes requested.