Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](nerieds) avoid redundant enumeration same LogicalProject in memo #38317

Merged
merged 1 commit into from
Jul 29, 2024
Merged

[fix](nerieds) avoid redundant enumeration same LogicalProject in memo #38317

merged 1 commit into from
Jul 29, 2024

Conversation

morrySnow
Copy link
Contributor

No description provided.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39443 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5749ce545c70a8d77208323a971f7d690cd34ca8, data reload: false

------ Round 1 ----------------------------------
q1	17633	4744	4260	4260
q2	2003	191	187	187
q3	10455	1204	1043	1043
q4	10177	735	765	735
q5	7551	2721	2659	2659
q6	226	137	142	137
q7	954	597	601	597
q8	9212	2071	2091	2071
q9	8748	6539	6545	6539
q10	8895	3772	3780	3772
q11	451	236	239	236
q12	522	219	225	219
q13	17955	3035	2966	2966
q14	270	233	235	233
q15	523	472	471	471
q16	508	391	382	382
q17	945	656	572	572
q18	8124	7342	7302	7302
q19	7033	1378	1305	1305
q20	697	336	321	321
q21	5028	3155	3208	3155
q22	346	282	281	281
Total cold run time: 118256 ms
Total hot run time: 39443 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4395	4279	4193	4193
q2	375	267	276	267
q3	2987	2967	2971	2967
q4	2042	1708	1799	1708
q5	5722	5611	5513	5513
q6	222	131	141	131
q7	2164	1856	1810	1810
q8	3237	3428	3403	3403
q9	8689	8781	8816	8781
q10	4091	3940	3783	3783
q11	590	498	494	494
q12	823	659	628	628
q13	15902	3179	3211	3179
q14	319	291	295	291
q15	529	484	495	484
q16	479	439	458	439
q17	1993	1512	1509	1509
q18	8009	7887	7797	7797
q19	1709	1590	1563	1563
q20	2392	1904	1865	1865
q21	5235	4963	4873	4873
q22	614	495	500	495
Total cold run time: 72518 ms
Total hot run time: 56173 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 171186 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5749ce545c70a8d77208323a971f7d690cd34ca8, data reload: false

query1	918	379	373	373
query2	6464	1907	1798	1798
query3	6636	205	218	205
query4	24165	17654	17458	17458
query5	3695	481	506	481
query6	285	183	166	166
query7	4582	291	291	291
query8	243	204	200	200
query9	8674	2426	2441	2426
query10	452	263	285	263
query11	10651	10229	10001	10001
query12	115	84	84	84
query13	1635	368	366	366
query14	10299	7731	7387	7387
query15	228	166	160	160
query16	7517	476	485	476
query17	1208	548	533	533
query18	1823	291	276	276
query19	190	145	148	145
query20	92	82	83	82
query21	204	141	136	136
query22	4440	4027	3984	3984
query23	34083	33848	33818	33818
query24	10893	2926	2842	2842
query25	616	379	402	379
query26	712	156	152	152
query27	2203	276	283	276
query28	5998	2092	2101	2092
query29	896	653	639	639
query30	247	161	162	161
query31	976	753	735	735
query32	92	50	55	50
query33	650	343	340	340
query34	904	511	511	511
query35	889	792	747	747
query36	1161	989	963	963
query37	142	91	86	86
query38	2942	2898	2801	2801
query39	934	862	878	862
query40	215	135	128	128
query41	47	47	47	47
query42	114	105	105	105
query43	504	482	469	469
query44	1126	733	724	724
query45	204	176	173	173
query46	1104	732	764	732
query47	1847	1747	1757	1747
query48	372	299	289	289
query49	821	396	401	396
query50	778	393	396	393
query51	6833	6656	6731	6656
query52	104	99	97	97
query53	358	295	295	295
query54	866	445	448	445
query55	74	79	71	71
query56	298	264	265	264
query57	1149	1030	1066	1030
query58	259	252	274	252
query59	2814	2635	2763	2635
query60	306	278	282	278
query61	97	127	90	90
query62	794	655	632	632
query63	326	297	296	296
query64	9132	2208	1979	1979
query65	3438	3140	3121	3121
query66	743	331	336	331
query67	16448	15201	14814	14814
query68	4521	550	558	550
query69	603	457	364	364
query70	1191	1164	1167	1164
query71	416	288	285	285
query72	7447	2732	2524	2524
query73	749	323	326	323
query74	6177	5659	5705	5659
query75	3401	2653	2698	2653
query76	2255	919	880	880
query77	587	306	308	306
query78	12715	9312	8919	8919
query79	3143	545	541	541
query80	1716	497	479	479
query81	604	219	227	219
query82	533	132	131	131
query83	324	166	175	166
query84	264	97	84	84
query85	710	308	301	301
query86	481	338	324	324
query87	3292	3128	3130	3128
query88	4353	2361	2382	2361
query89	499	391	400	391
query90	1807	202	197	197
query91	128	99	109	99
query92	60	54	54	54
query93	3479	522	517	517
query94	1260	293	289	289
query95	415	319	320	319
query96	615	279	277	277
query97	3238	3037	3023	3023
query98	221	204	195	195
query99	1598	1273	1230	1230
Total cold run time: 278827 ms
Total hot run time: 171186 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.92 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5749ce545c70a8d77208323a971f7d690cd34ca8, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.50	0.50	0.47
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.06	0.05	0.04
query9	0.56	0.49	0.48
query10	0.54	0.54	0.54
query11	0.15	0.12	0.12
query12	0.15	0.12	0.12
query13	0.61	0.60	0.58
query14	0.76	0.78	0.78
query15	0.87	0.82	0.81
query16	0.37	0.38	0.35
query17	0.98	0.97	1.07
query18	0.22	0.22	0.21
query19	1.86	1.75	1.70
query20	0.02	0.01	0.01
query21	15.40	0.74	0.64
query22	4.08	7.00	2.23
query23	18.28	1.33	1.38
query24	2.22	0.23	0.22
query25	0.16	0.08	0.09
query26	0.29	0.22	0.21
query27	0.45	0.24	0.23
query28	13.13	1.02	1.01
query29	12.59	3.30	3.28
query30	0.25	0.08	0.06
query31	2.86	0.39	0.39
query32	3.27	0.49	0.47
query33	2.90	2.90	2.95
query34	16.90	4.35	4.34
query35	4.46	4.42	4.37
query36	0.65	0.47	0.47
query37	0.20	0.15	0.16
query38	0.15	0.14	0.14
query39	0.04	0.04	0.03
query40	0.15	0.12	0.11
query41	0.09	0.06	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.47 s
Total hot run time: 30.92 s

@morrySnow
Copy link
Contributor Author

run buildall

@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39914 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 49b2ee907c0be2ea2246ecbc233c6dc534cfbc5d, data reload: false

------ Round 1 ----------------------------------
q1	17626	4340	4228	4228
q2	2002	199	192	192
q3	10455	1147	1029	1029
q4	10137	765	776	765
q5	7544	2698	2694	2694
q6	221	137	136	136
q7	950	584	584	584
q8	9231	1911	1888	1888
q9	8839	6570	6539	6539
q10	8841	3815	3801	3801
q11	465	250	250	250
q12	484	230	218	218
q13	18859	2921	2974	2921
q14	281	243	250	243
q15	519	475	495	475
q16	498	401	381	381
q17	971	616	701	616
q18	7997	7431	7353	7353
q19	6188	1040	1012	1012
q20	686	344	325	325
q21	5300	3978	4646	3978
q22	366	304	286	286
Total cold run time: 118460 ms
Total hot run time: 39914 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4455	4251	4278	4251
q2	391	278	257	257
q3	3052	2982	3007	2982
q4	1972	1781	1743	1743
q5	5554	5490	5447	5447
q6	217	132	138	132
q7	2287	1872	1828	1828
q8	3270	3451	3388	3388
q9	8779	8857	8787	8787
q10	4109	3693	3904	3693
q11	618	494	506	494
q12	842	639	626	626
q13	16114	3171	3165	3165
q14	324	309	271	271
q15	537	516	484	484
q16	500	433	426	426
q17	1789	1496	1520	1496
q18	8155	7937	7803	7803
q19	2419	1556	1596	1556
q20	2268	1854	1865	1854
q21	5455	5037	5196	5037
q22	604	487	476	476
Total cold run time: 73711 ms
Total hot run time: 56196 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 170295 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 49b2ee907c0be2ea2246ecbc233c6dc534cfbc5d, data reload: false

query1	930	384	364	364
query2	6363	1868	1822	1822
query3	6654	210	221	210
query4	20593	17484	17405	17405
query5	3568	502	470	470
query6	258	172	165	165
query7	4583	290	293	290
query8	233	188	187	187
query9	8668	2422	2395	2395
query10	406	267	258	258
query11	11229	10097	10081	10081
query12	126	87	88	87
query13	1637	371	347	347
query14	10138	7500	7437	7437
query15	212	163	163	163
query16	7700	487	468	468
query17	1296	550	532	532
query18	1959	281	278	278
query19	191	144	143	143
query20	92	86	101	86
query21	211	101	97	97
query22	4167	4044	4041	4041
query23	33788	34032	34023	34023
query24	10897	2943	2883	2883
query25	585	398	375	375
query26	706	151	152	151
query27	2264	283	273	273
query28	6106	2068	2079	2068
query29	678	420	436	420
query30	260	151	152	151
query31	993	761	765	761
query32	102	55	60	55
query33	650	309	299	299
query34	866	481	489	481
query35	906	749	740	740
query36	1070	946	975	946
query37	151	92	83	83
query38	2943	2870	2826	2826
query39	885	875	844	844
query40	213	120	115	115
query41	47	44	44	44
query42	115	102	105	102
query43	501	477	459	459
query44	1057	739	748	739
query45	210	179	176	176
query46	1094	739	743	739
query47	1871	1787	1784	1784
query48	356	293	290	290
query49	828	410	426	410
query50	792	396	408	396
query51	6743	6718	6645	6645
query52	109	97	87	87
query53	255	180	181	180
query54	889	457	444	444
query55	77	77	76	76
query56	307	257	255	255
query57	1109	1052	1057	1052
query58	260	284	256	256
query59	2800	2672	2492	2492
query60	300	268	262	262
query61	99	92	98	92
query62	786	640	658	640
query63	209	175	177	175
query64	9169	2288	1759	1759
query65	3146	3117	3084	3084
query66	743	393	337	337
query67	15207	14874	14963	14874
query68	5361	550	558	550
query69	664	407	329	329
query70	1242	1151	1195	1151
query71	442	273	285	273
query72	7802	2750	2578	2578
query73	781	326	324	324
query74	6021	5739	5624	5624
query75	3510	2602	2638	2602
query76	3138	1068	903	903
query77	632	297	316	297
query78	9550	9128	8851	8851
query79	2838	519	521	519
query80	1935	477	482	477
query81	603	219	218	218
query82	1449	133	138	133
query83	267	168	173	168
query84	266	85	81	81
query85	1318	317	310	310
query86	459	323	309	309
query87	3240	3119	3064	3064
query88	4426	2411	2398	2398
query89	416	288	290	288
query90	1760	274	193	193
query91	125	103	101	101
query92	63	49	48	48
query93	3681	547	539	539
query94	833	301	286	286
query95	367	266	258	258
query96	619	278	270	270
query97	3162	3039	3045	3039
query98	226	200	199	199
query99	1531	1270	1246	1246
Total cold run time: 273083 ms
Total hot run time: 170295 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.29 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 49b2ee907c0be2ea2246ecbc233c6dc534cfbc5d, data reload: false

query1	0.04	0.04	0.03
query2	0.09	0.03	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.50	0.49	0.50
query6	1.14	0.72	0.72
query7	0.02	0.01	0.01
query8	0.04	0.04	0.04
query9	0.54	0.49	0.48
query10	0.55	0.53	0.54
query11	0.15	0.12	0.11
query12	0.16	0.12	0.12
query13	0.59	0.58	0.58
query14	0.77	0.79	0.84
query15	0.86	0.82	0.81
query16	0.37	0.37	0.35
query17	1.03	1.03	1.04
query18	0.24	0.21	0.21
query19	1.82	1.67	1.72
query20	0.01	0.01	0.02
query21	15.40	0.77	0.66
query22	4.15	8.22	1.77
query23	18.30	1.39	1.17
query24	2.07	0.25	0.22
query25	0.15	0.08	0.08
query26	0.30	0.21	0.22
query27	0.46	0.23	0.24
query28	13.27	1.02	0.99
query29	12.67	3.40	3.35
query30	0.27	0.06	0.05
query31	2.88	0.40	0.37
query32	3.29	0.48	0.47
query33	2.87	2.85	2.85
query34	17.03	4.33	4.41
query35	4.48	4.40	4.46
query36	0.65	0.47	0.45
query37	0.17	0.16	0.16
query38	0.16	0.14	0.15
query39	0.04	0.04	0.03
query40	0.14	0.12	0.12
query41	0.09	0.05	0.04
query42	0.06	0.05	0.04
query43	0.04	0.04	0.03
Total cold run time: 109.76 s
Total hot run time: 30.29 s

@morrySnow
Copy link
Contributor Author

run buildall

@morrySnow morrySnow changed the title [DNM](nerieds) avoid exploration unexpected number of group expressions [fix](nerieds) avoid exploration unexpected number of group expressions Jul 26, 2024
@morrySnow morrySnow changed the title [fix](nerieds) avoid exploration unexpected number of group expressions [fix](nerieds) avoid redundant enumeration same LogicalProject in memo Jul 26, 2024
@morrySnow morrySnow force-pushed the exploration_space branch 2 times, most recently from 960d886 to 2c32bd4 Compare July 26, 2024 04:27
@morrySnow
Copy link
Contributor Author

run buildall

@morrySnow morrySnow marked this pull request as ready for review July 26, 2024 07:03
xzj7019
xzj7019 reviewed Jul 26, 2024
View reviewed changes
...e/src/main/java/org/apache/doris/nereids/rules/exploration/join/InnerJoinLAsscomProject.java
@@ -98,7 +98,7 @@ public Rule build() {
newTopHashConjuncts.forEach(expr -> topUsedExprIds.addAll(expr.getInputSlotExprIds()));
newTopOtherConjuncts.forEach(expr -> topUsedExprIds.addAll(expr.getInputSlotExprIds()));
Plan left = CBOUtils.newProject(topUsedExprIds, newBottomJoin);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not apply this for both sides? Means left side's new project is a must?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need a project between join and join

@doris-robot
Copy link

TPC-H: Total hot run time: 40687 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eea4e2ed4008ec3005cf8ad7c2f90d5bea62eb7f, data reload: false

------ Round 1 ----------------------------------
q1	17623	4340	4267	4267
q2	2015	200	196	196
q3	10446	1157	1068	1068
q4	10149	755	685	685
q5	7550	2705	2657	2657
q6	221	138	140	138
q7	958	599	592	592
q8	9215	1925	1912	1912
q9	8863	6555	6600	6555
q10	8863	3808	3793	3793
q11	460	261	244	244
q12	509	235	224	224
q13	17768	2990	2992	2990
q14	281	242	248	242
q15	521	482	511	482
q16	504	391	385	385
q17	965	674	703	674
q18	7967	7447	7364	7364
q19	3741	1000	989	989
q20	662	338	336	336
q21	5383	4595	4650	4595
q22	369	299	301	299
Total cold run time: 115033 ms
Total hot run time: 40687 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4524	4305	4281	4281
q2	379	277	270	270
q3	3020	2860	2882	2860
q4	2047	1735	1647	1647
q5	5652	5570	5519	5519
q6	221	135	135	135
q7	2204	1921	1829	1829
q8	3285	3451	3437	3437
q9	8798	8815	8880	8815
q10	4106	3892	3754	3754
q11	585	492	502	492
q12	837	695	664	664
q13	16925	3264	3206	3206
q14	317	301	305	301
q15	522	500	494	494
q16	497	427	439	427
q17	1864	1569	1531	1531
q18	8057	7928	7763	7763
q19	1789	1670	1526	1526
q20	2372	1921	1855	1855
q21	9370	5254	5197	5197
q22	562	528	507	507
Total cold run time: 77933 ms
Total hot run time: 56510 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 170460 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eea4e2ed4008ec3005cf8ad7c2f90d5bea62eb7f, data reload: false

query1	922	386	372	372
query2	6431	1990	1902	1902
query3	6642	209	225	209
query4	25583	17592	17394	17394
query5	3688	508	490	490
query6	267	176	176	176
query7	4580	284	294	284
query8	257	200	195	195
query9	8627	2466	2452	2452
query10	427	269	263	263
query11	11819	10082	10163	10082
query12	127	88	84	84
query13	1619	371	362	362
query14	9679	7453	7833	7453
query15	220	162	169	162
query16	7676	478	444	444
query17	1127	544	538	538
query18	1970	305	286	286
query19	192	145	139	139
query20	93	82	91	82
query21	206	99	100	99
query22	4340	3932	3936	3932
query23	34124	33963	33656	33656
query24	11415	3098	2982	2982
query25	633	399	396	396
query26	1161	155	156	155
query27	2347	273	282	273
query28	6969	2076	2067	2067
query29	877	430	448	430
query30	260	164	158	158
query31	990	781	752	752
query32	101	60	57	57
query33	759	334	312	312
query34	965	475	525	475
query35	887	727	752	727
query36	1130	966	946	946
query37	146	88	85	85
query38	2946	2891	2824	2824
query39	921	868	866	866
query40	201	119	128	119
query41	49	45	49	45
query42	107	103	100	100
query43	523	483	477	477
query44	1234	712	710	710
query45	214	180	179	179
query46	1100	764	740	740
query47	1829	1745	1785	1745
query48	372	306	296	296
query49	834	418	420	418
query50	798	400	400	400
query51	6767	6647	6687	6647
query52	111	87	94	87
query53	258	180	177	177
query54	894	455	449	449
query55	80	73	73	73
query56	310	255	248	248
query57	1152	1037	1061	1037
query58	270	280	255	255
query59	2988	2751	2838	2751
query60	294	260	266	260
query61	99	97	100	97
query62	799	619	659	619
query63	205	182	178	178
query64	9409	2246	1736	1736
query65	3214	3128	3115	3115
query66	742	330	334	330
query67	15497	14838	14757	14757
query68	4567	536	533	533
query69	555	401	326	326
query70	1189	1095	1065	1065
query71	404	264	272	264
query72	7450	2863	2525	2525
query73	767	328	325	325
query74	6054	5743	5691	5691
query75	3357	2667	2674	2667
query76	2139	1001	1026	1001
query77	611	303	313	303
query78	9569	9077	8993	8993
query79	2625	516	517	516
query80	1977	480	483	480
query81	586	218	224	218
query82	766	139	139	139
query83	284	171	171	171
query84	253	80	77	77
query85	1516	417	304	304
query86	460	283	322	283
query87	3258	3062	3130	3062
query88	4047	2464	2483	2464
query89	393	284	278	278
query90	1687	198	196	196
query91	125	97	98	97
query92	63	48	47	47
query93	1847	527	526	526
query94	796	284	297	284
query95	354	270	272	270
query96	596	281	280	280
query97	3245	3059	3038	3038
query98	225	200	190	190
query99	1540	1276	1241	1241
Total cold run time: 276778 ms
Total hot run time: 170460 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.7 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eea4e2ed4008ec3005cf8ad7c2f90d5bea62eb7f, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.24	0.06	0.05
query4	1.64	0.10	0.11
query5	0.48	0.50	0.49
query6	1.14	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.48	0.49
query10	0.55	0.55	0.55
query11	0.15	0.12	0.11
query12	0.16	0.12	0.12
query13	0.60	0.59	0.58
query14	0.79	0.77	0.78
query15	0.86	0.82	0.82
query16	0.37	0.37	0.36
query17	1.05	1.00	1.00
query18	0.23	0.22	0.22
query19	1.91	1.77	1.80
query20	0.01	0.02	0.01
query21	15.40	0.76	0.64
query22	4.11	6.70	2.67
query23	18.24	1.47	1.26
query24	2.18	0.24	0.23
query25	0.16	0.09	0.09
query26	0.30	0.21	0.21
query27	0.46	0.25	0.23
query28	13.24	1.02	1.00
query29	12.60	3.41	3.41
query30	0.25	0.06	0.06
query31	2.86	0.39	0.38
query32	3.27	0.48	0.47
query33	2.91	2.93	2.94
query34	17.20	4.34	4.38
query35	4.43	4.43	4.43
query36	0.67	0.46	0.46
query37	0.19	0.16	0.15
query38	0.17	0.15	0.15
query39	0.04	0.04	0.03
query40	0.16	0.13	0.13
query41	0.10	0.05	0.05
query42	0.07	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.98 s
Total hot run time: 31.7 s

@morrySnow
Copy link
Contributor Author

run cloud_p0

@morrySnow morrySnow closed this Jul 26, 2024
@morrySnow morrySnow reopened this Jul 26, 2024
@morrySnow
Copy link
Contributor Author

run cloud_p0

xzj7019
xzj7019 reviewed Jul 29, 2024
View reviewed changes
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/cascades/ApplyRuleJob.java
@@ -74,6 +74,11 @@ public final void execute() throws AnalysisException {
GroupExpressionMatching groupExpressionMatching
= new GroupExpressionMatching(rule.getPattern(), groupExpression);
for (Plan plan : groupExpressionMatching) {
if (rule.isExploration()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the checking is moved here, can the logic in getJoinRules be removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, in OptimizeGroupExpressionJob we could avoid generate ApplyRuleJob at all. so remove it will generate many useless job to slow down planning

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 29, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@morrySnow morrySnow merged commit 1b17f99 into apache:master Jul 29, 2024
48 of 50 checks passed
@morrySnow morrySnow deleted the exploration_space branch July 29, 2024 03:48
dataroaring pushed a commit that referenced this pull request Jul 29, 2024
@morrySnow @dataroaring
[fix](nerieds) avoid redundant enumeration same LogicalProject in memo (
a63832c 
#38317)

1. use set to compare project
2. use map to store enforcer
3. avoid genarate useless project under bottom join when do join reorder
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Nov 12, 2024
@morrySnow @gavinchou
[fix](nerieds) avoid redundant enumeration same LogicalProject in memo (
cdaabef 
apache#38317)

1. use set to compare project
2. use map to store enforcer
3. avoid genarate useless project under bottom join when do join reorder
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xzj7019 xzj7019 xzj7019 approved these changes

@englefly englefly englefly approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.1-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@morrySnow @doris-robot @englefly @xzj7019 @dataroaring