Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](Nereids) normalize aggregate should not push down lambda's param #37109

Merged
merged 1 commit into from
Jul 2, 2024
Merged

[fix](Nereids) normalize aggregate should not push down lambda's param #37109

merged 1 commit into from
Jul 2, 2024

Conversation

morrySnow
Copy link
Contributor

@morrySnow morrySnow commented Jul 1, 2024 •
edited
Loading

ArrayItemSlot should not be inputSlots

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39979 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 458d3e59b9a6f8e50836e7585ee58b6151987be8, data reload: false

------ Round 1 ----------------------------------
q1	17680	4597	4350	4350
q2	3083	211	205	205
q3	11599	1290	1075	1075
q4	10739	820	805	805
q5	7942	2714	2668	2668
q6	222	142	143	142
q7	975	592	603	592
q8	9213	2071	2073	2071
q9	8841	6537	6501	6501
q10	8994	3706	3776	3706
q11	471	237	233	233
q12	416	236	229	229
q13	17761	2965	2987	2965
q14	280	223	223	223
q15	524	473	488	473
q16	510	377	371	371
q17	966	778	705	705
q18	8123	7597	7421	7421
q19	5053	1472	1470	1470
q20	682	320	318	318
q21	4924	3130	3995	3130
q22	387	336	326	326
Total cold run time: 119385 ms
Total hot run time: 39979 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4408	4184	4195	4184
q2	362	265	261	261
q3	2957	2758	2713	2713
q4	1816	1589	1629	1589
q5	5271	5250	5290	5250
q6	216	132	135	132
q7	2138	1735	1709	1709
q8	3195	3306	3304	3304
q9	8328	8298	8399	8298
q10	3915	3645	3686	3645
q11	573	477	488	477
q12	773	642	638	638
q13	17151	2984	2979	2979
q14	290	265	251	251
q15	510	489	487	487
q16	474	407	413	407
q17	1778	1476	1478	1476
q18	7721	7685	7433	7433
q19	1666	1543	1525	1525
q20	1991	1786	1806	1786
q21	5099	4754	4782	4754
q22	623	553	561	553
Total cold run time: 71255 ms
Total hot run time: 53851 ms

@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40687 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bbc2e8eb8f2496f90a318bf024c0442c45c30667, data reload: false

------ Round 1 ----------------------------------
q1	17755	4638	4265	4265
q2	2029	188	201	188
q3	10532	1224	1151	1151
q4	10217	748	721	721
q5	7477	2695	2645	2645
q6	218	137	136	136
q7	953	597	608	597
q8	9227	2085	2078	2078
q9	8811	6461	6490	6461
q10	8912	3764	3701	3701
q11	460	233	236	233
q12	575	240	236	236
q13	17868	2993	2985	2985
q14	292	231	225	225
q15	511	492	496	492
q16	506	383	382	382
q17	978	710	774	710
q18	8056	7498	7502	7498
q19	4178	1516	1387	1387
q20	650	322	327	322
q21	5213	3929	4033	3929
q22	421	349	345	345
Total cold run time: 115839 ms
Total hot run time: 40687 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4359	4243	4283	4243
q2	367	267	266	266
q3	2975	2887	2860	2860
q4	2039	1743	1757	1743
q5	5586	5516	5494	5494
q6	230	131	133	131
q7	2215	1869	1862	1862
q8	3314	3415	3448	3415
q9	8675	8678	8816	8678
q10	4090	3953	3716	3716
q11	606	505	499	499
q12	863	685	681	681
q13	16039	3152	3152	3152
q14	300	268	277	268
q15	542	496	509	496
q16	476	430	421	421
q17	1817	1539	1502	1502
q18	8121	8009	7851	7851
q19	1819	1585	1631	1585
q20	2146	1926	1899	1899
q21	5070	5141	4885	4885
q22	620	573	593	573
Total cold run time: 72269 ms
Total hot run time: 56220 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174270 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bbc2e8eb8f2496f90a318bf024c0442c45c30667, data reload: false

query1	909	379	376	376
query2	6456	2581	2353	2353
query3	6641	216	212	212
query4	18802	17377	17152	17152
query5	3706	473	478	473
query6	258	182	161	161
query7	4583	298	304	298
query8	312	292	298	292
query9	8557	2383	2374	2374
query10	578	337	284	284
query11	10632	9974	10085	9974
query12	115	81	86	81
query13	1650	375	367	367
query14	10158	7597	7626	7597
query15	275	185	188	185
query16	7956	283	280	280
query17	1870	564	550	550
query18	2021	287	279	279
query19	197	156	172	156
query20	98	84	92	84
query21	213	145	129	129
query22	4264	3999	4077	3999
query23	33715	33618	33693	33618
query24	10829	2862	2899	2862
query25	615	429	402	402
query26	708	160	157	157
query27	2230	330	332	330
query28	5972	2137	2152	2137
query29	912	673	658	658
query30	254	160	156	156
query31	1034	777	775	775
query32	100	56	61	56
query33	686	321	316	316
query34	865	473	482	473
query35	787	634	633	633
query36	1100	961	1006	961
query37	146	87	80	80
query38	2968	2824	2855	2824
query39	927	799	831	799
query40	211	215	123	123
query41	48	47	48	47
query42	111	106	105	105
query43	646	556	547	547
query44	1076	726	740	726
query45	188	171	164	164
query46	1071	743	729	729
query47	1863	1799	1742	1742
query48	383	302	293	293
query49	846	448	409	409
query50	767	380	395	380
query51	6849	6670	6780	6670
query52	105	91	93	91
query53	355	290	285	285
query54	874	441	441	441
query55	74	72	76	72
query56	282	267	250	250
query57	1131	1029	1063	1029
query58	256	236	241	236
query59	3538	3520	3393	3393
query60	304	276	284	276
query61	95	91	88	88
query62	595	442	447	442
query63	328	290	290	290
query64	8408	2218	1731	1731
query65	3180	3078	3095	3078
query66	744	365	329	329
query67	15494	14856	15005	14856
query68	8926	543	536	536
query69	710	412	323	323
query70	1366	1137	1029	1029
query71	525	279	279	279
query72	8954	5277	5700	5277
query73	2278	330	330	330
query74	5929	5445	5447	5445
query75	5653	2685	2666	2666
query76	5581	887	975	887
query77	754	298	298	298
query78	10438	9970	9936	9936
query79	7506	519	514	514
query80	1160	490	467	467
query81	563	221	228	221
query82	260	105	105	105
query83	320	167	165	165
query84	267	85	87	85
query85	974	270	266	266
query86	376	325	319	319
query87	3324	3073	3088	3073
query88	4304	2383	2373	2373
query89	496	376	379	376
query90	2010	185	187	185
query91	126	97	101	97
query92	61	48	50	48
query93	5439	506	492	492
query94	1633	190	179	179
query95	409	309	306	306
query96	605	276	265	265
query97	3200	3006	2997	2997
query98	248	193	190	190
query99	1064	830	826	826
Total cold run time: 289380 ms
Total hot run time: 174270 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.4 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bbc2e8eb8f2496f90a318bf024c0442c45c30667, data reload: false

query1	0.04	0.03	0.03
query2	0.09	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.08	0.10
query5	0.51	0.47	0.49
query6	1.14	0.73	0.71
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.55	0.49	0.49
query10	0.55	0.54	0.54
query11	0.14	0.11	0.12
query12	0.15	0.12	0.13
query13	0.62	0.59	0.58
query14	0.76	0.78	0.77
query15	0.84	0.82	0.80
query16	0.35	0.37	0.39
query17	1.04	1.02	1.00
query18	0.25	0.24	0.26
query19	1.82	1.69	1.72
query20	0.01	0.01	0.01
query21	15.56	0.79	0.66
query22	4.02	7.93	1.56
query23	18.32	1.37	1.38
query24	2.12	0.23	0.22
query25	0.15	0.08	0.08
query26	0.27	0.18	0.17
query27	0.08	0.08	0.08
query28	13.25	1.02	1.00
query29	12.60	3.35	3.39
query30	0.26	0.06	0.06
query31	2.90	0.38	0.39
query32	3.25	0.48	0.46
query33	2.91	2.93	2.92
query34	17.10	4.44	4.40
query35	4.45	4.46	4.56
query36	0.66	0.47	0.48
query37	0.19	0.16	0.16
query38	0.16	0.16	0.15
query39	0.04	0.03	0.04
query40	0.17	0.15	0.14
query41	0.08	0.06	0.05
query42	0.05	0.05	0.05
query43	0.05	0.03	0.04
Total cold run time: 109.47 s
Total hot run time: 30.4 s

@morrySnow
Copy link
Contributor Author

run p0

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 2, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 2, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 2, 2024

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit cce5d5b into apache:master Jul 2, 2024
27 of 30 checks passed
@morrySnow morrySnow deleted the fix_agg_lambda branch July 2, 2024 11:37
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jul 4, 2024
@morrySnow
[fix](Nereids) normalize aggregate should not push down lambda's param (
eb2387c 
apache#37109)

pick from master apache#37109

ArrayItemSlot should not be inputSlot
@morrySnow morrySnow mentioned this pull request Jul 4, 2024
Merged
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jul 4, 2024
@morrySnow
[fix](Nereids) normalize aggregate should not push down lambda's param (
7ee7f32 
apache#37109)

pick from master apache#37109

ArrayItemSlot should not be inputSlot
morrySnow added a commit that referenced this pull request Jul 5, 2024
@morrySnow
[fix](Nereids) normalize aggregate should not push down lambda's param (
256221a 
#37109) (#37285)

pick from master #37109

ArrayItemSlot should not be inputSlot
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
@morrySnow @dataroaring
[fix](Nereids) normalize aggregate should not push down lambda's param (
fd6a934 
#37109)

ArrayItemSlot should not be inputSlot
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@keanji-x keanji-x keanji-x approved these changes

@924060929 924060929 924060929 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.1-merged p0_b reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@morrySnow @doris-robot @924060929 @keanji-x @dataroaring