Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](planner) Empty table source should not begin a transaction to avoid infinite transaction #38991

Conversation

TangSiyang2001
Copy link
Collaborator

Proposed changes

Issue Number: close #38956

As title.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added the doing label Aug 7, 2024
@TangSiyang2001 TangSiyang2001 changed the title [fix](planner) Empty table source do not begin a transaction [fix](planner) Empty table source should not begin a transaction Aug 7, 2024
@TangSiyang2001 TangSiyang2001 changed the title [fix](planner) Empty table source should not begin a transaction [fix](planner) Empty table source should not begin a transaction to avoid infinite transaction Aug 7, 2024
dataroaring
dataroaring previously approved these changes Aug 7, 2024
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

github-actions bot commented Aug 7, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 7, 2024
Copy link
Contributor

github-actions bot commented Aug 7, 2024

PR approved by anyone and no changes requested.

liaoxin01
liaoxin01 previously approved these changes Aug 7, 2024
Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@doris-robot
Copy link

TPC-H: Total hot run time: 41708 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 426dcdb3a98c5e5061892d5bdf584b9ef5ab6f36, data reload: false

------ Round 1 ----------------------------------
q1	17605	4209	4087	4087
q2	2020	203	192	192
q3	10470	1327	1356	1327
q4	10169	852	964	852
q5	7618	2963	2978	2963
q6	220	138	136	136
q7	1060	623	615	615
q8	9433	1935	1954	1935
q9	8545	6616	6566	6566
q10	8722	3832	3834	3832
q11	433	246	266	246
q12	416	223	222	222
q13	17769	2934	2914	2914
q14	272	240	250	240
q15	517	478	489	478
q16	530	411	384	384
q17	981	938	913	913
q18	8082	7368	7247	7247
q19	1387	1214	1228	1214
q20	562	327	334	327
q21	5291	4816	4742	4742
q22	347	285	276	276
Total cold run time: 112449 ms
Total hot run time: 41708 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4062	4031	4014	4014
q2	332	225	219	219
q3	2992	3038	3132	3038
q4	1977	2064	1982	1982
q5	5553	5484	5460	5460
q6	216	129	127	127
q7	2123	1734	1835	1734
q8	3297	3376	3346	3346
q9	8682	8639	8772	8639
q10	3949	4024	3914	3914
q11	545	444	463	444
q12	766	610	607	607
q13	16279	3072	3087	3072
q14	297	273	272	272
q15	524	500	506	500
q16	476	421	409	409
q17	1759	1764	1714	1714
q18	8090	7796	7642	7642
q19	1725	1745	1722	1722
q20	2066	1867	1837	1837
q21	5808	5491	5515	5491
q22	498	464	475	464
Total cold run time: 72016 ms
Total hot run time: 56647 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169425 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 426dcdb3a98c5e5061892d5bdf584b9ef5ab6f36, data reload: false

query1	909	380	377	377
query2	6475	1667	1679	1667
query3	6665	211	219	211
query4	20436	17583	17412	17412
query5	3670	546	514	514
query6	285	162	164	162
query7	4594	286	283	283
query8	245	205	192	192
query9	8496	2350	2325	2325
query10	404	264	276	264
query11	10755	10188	10302	10188
query12	118	86	88	86
query13	1622	373	366	366
query14	9897	6906	6356	6356
query15	199	162	168	162
query16	6839	441	459	441
query17	952	571	554	554
query18	1915	288	293	288
query19	198	148	153	148
query20	96	91	88	88
query21	207	102	101	101
query22	4430	4299	4147	4147
query23	33692	33561	33437	33437
query24	10368	3183	3114	3114
query25	756	424	459	424
query26	1800	155	159	155
query27	3103	293	288	288
query28	7393	2006	1985	1985
query29	1311	472	445	445
query30	242	164	161	161
query31	982	795	775	775
query32	104	58	59	58
query33	704	321	346	321
query34	924	488	493	488
query35	861	754	727	727
query36	1004	874	895	874
query37	284	79	82	79
query38	2876	2786	2771	2771
query39	873	832	821	821
query40	253	116	116	116
query41	48	47	45	45
query42	124	104	103	103
query43	453	427	418	418
query44	1196	738	735	735
query45	211	178	177	177
query46	1080	792	793	792
query47	1842	1727	1734	1727
query48	367	296	297	296
query49	941	435	434	434
query50	898	426	428	426
query51	6866	6755	6694	6694
query52	103	94	91	91
query53	257	185	183	183
query54	672	477	474	474
query55	77	79	73	73
query56	284	269	269	269
query57	1179	1166	1044	1044
query58	258	276	276	276
query59	2427	2297	2330	2297
query60	290	264	269	264
query61	113	91	91	91
query62	893	663	663	663
query63	214	182	181	181
query64	5633	1907	1907	1907
query65	3181	3225	3093	3093
query66	1311	327	331	327
query67	15259	14899	14707	14707
query68	7760	569	583	569
query69	739	372	316	316
query70	1140	1089	1070	1070
query71	591	284	281	281
query72	8136	2677	2511	2511
query73	971	328	328	328
query74	6116	5704	5691	5691
query75	4779	2695	2722	2695
query76	5994	1278	1289	1278
query77	926	309	311	309
query78	9541	8897	8982	8897
query79	4958	533	526	526
query80	1943	511	521	511
query81	560	226	229	226
query82	793	135	134	134
query83	274	172	171	171
query84	266	76	79	76
query85	1527	332	310	310
query86	395	292	276	276
query87	3265	3077	3107	3077
query88	3910	2405	2396	2396
query89	459	289	285	285
query90	2118	190	194	190
query91	132	99	99	99
query92	65	48	48	48
query93	4731	602	610	602
query94	994	295	285	285
query95	375	263	269	263
query96	619	277	284	277
query97	3237	3096	3129	3096
query98	218	203	196	196
query99	1630	1292	1278	1278
Total cold run time: 282276 ms
Total hot run time: 169425 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.97 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 426dcdb3a98c5e5061892d5bdf584b9ef5ab6f36, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.06	0.06
query5	0.48	0.48	0.47
query6	1.14	0.71	0.71
query7	0.02	0.01	0.01
query8	0.05	0.05	0.04
query9	0.57	0.52	0.52
query10	0.56	0.57	0.56
query11	0.15	0.11	0.11
query12	0.16	0.12	0.12
query13	0.62	0.62	0.60
query14	0.77	0.81	0.79
query15	0.90	0.85	0.86
query16	0.35	0.35	0.37
query17	0.95	0.98	1.02
query18	0.23	0.22	0.22
query19	1.81	1.76	1.76
query20	0.01	0.01	0.01
query21	15.39	0.76	0.66
query22	3.89	7.92	1.27
query23	17.98	1.28	1.29
query24	2.25	0.22	0.21
query25	0.19	0.08	0.08
query26	0.31	0.22	0.21
query27	0.45	0.24	0.23
query28	13.17	1.00	0.96
query29	12.53	3.30	3.31
query30	0.25	0.06	0.05
query31	2.87	0.40	0.41
query32	3.28	0.48	0.48
query33	2.97	2.94	2.95
query34	15.41	4.24	4.25
query35	4.30	4.29	4.30
query36	0.69	0.48	0.48
query37	0.18	0.17	0.16
query38	0.16	0.15	0.15
query39	0.04	0.03	0.04
query40	0.16	0.13	0.14
query41	0.09	0.05	0.05
query42	0.05	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 107.44 s
Total hot run time: 29.97 s

@TangSiyang2001 TangSiyang2001 force-pushed the fix-empty-source-insert-infinite-txn branch from 426dcdb to e74de59 Compare August 7, 2024 04:57
@github-actions github-actions bot added doing and removed approved Indicates a PR has been approved by one committer. labels Aug 7, 2024
@TangSiyang2001 TangSiyang2001 force-pushed the fix-empty-source-insert-infinite-txn branch from e74de59 to 530c4a2 Compare August 7, 2024 05:44
@TangSiyang2001 TangSiyang2001 force-pushed the fix-empty-source-insert-infinite-txn branch from 530c4a2 to 85d8715 Compare August 7, 2024 06:27
@TangSiyang2001
Copy link
Collaborator Author

run buildall

1 similar comment
@TangSiyang2001
Copy link
Collaborator Author

run buildall

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 7, 2024
Copy link
Contributor

github-actions bot commented Aug 7, 2024

PR approved by at least one committer and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 42030 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 85d87158c426f8e6d7ef3ab82bb69c2c8c440da7, data reload: false

------ Round 1 ----------------------------------
q1	17635	4134	4070	4070
q2	2024	199	207	199
q3	10443	1338	1364	1338
q4	10160	913	997	913
q5	7617	3070	2990	2990
q6	226	139	139	139
q7	1065	624	619	619
q8	9461	1975	1968	1968
q9	8496	6583	6652	6583
q10	8809	3855	3864	3855
q11	438	255	257	255
q12	421	229	230	229
q13	17762	2962	2940	2940
q14	274	246	245	245
q15	523	493	497	493
q16	528	392	382	382
q17	982	945	939	939
q18	8030	7301	7274	7274
q19	1442	1224	1225	1224
q20	587	333	340	333
q21	5337	4764	4893	4764
q22	350	278	286	278
Total cold run time: 112610 ms
Total hot run time: 42030 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4086	4031	3993	3993
q2	330	218	220	218
q3	3021	3047	3180	3047
q4	2030	2119	1968	1968
q5	5666	5515	5452	5452
q6	220	134	131	131
q7	2157	1841	1803	1803
q8	3311	3389	3353	3353
q9	8641	8707	8710	8707
q10	3968	3992	3973	3973
q11	562	469	480	469
q12	787	665	604	604
q13	15053	3124	3144	3124
q14	306	275	260	260
q15	542	497	482	482
q16	473	431	418	418
q17	1772	1724	1761	1724
q18	8366	7810	7805	7805
q19	1966	1736	1742	1736
q20	2061	1852	1838	1838
q21	5781	5426	5406	5406
q22	517	444	478	444
Total cold run time: 71616 ms
Total hot run time: 56955 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169113 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 85d87158c426f8e6d7ef3ab82bb69c2c8c440da7, data reload: false

query1	904	376	371	371
query2	6506	1721	1734	1721
query3	6659	211	217	211
query4	20274	17512	17253	17253
query5	3632	543	521	521
query6	299	175	167	167
query7	4607	295	295	295
query8	250	195	196	195
query9	8500	2308	2295	2295
query10	445	283	263	263
query11	10529	9997	10126	9997
query12	126	86	84	84
query13	1632	377	360	360
query14	8470	6301	7515	6301
query15	216	168	165	165
query16	6954	495	465	465
query17	962	606	562	562
query18	1922	290	292	290
query19	198	157	148	148
query20	93	87	90	87
query21	204	107	105	105
query22	4141	4177	4047	4047
query23	33741	33809	33616	33616
query24	9997	3108	3108	3108
query25	712	427	442	427
query26	1612	154	159	154
query27	3019	282	286	282
query28	7699	2015	1977	1977
query29	1191	460	481	460
query30	244	156	153	153
query31	969	775	786	775
query32	107	58	57	57
query33	693	341	339	339
query34	943	512	511	511
query35	906	789	763	763
query36	1060	946	891	891
query37	240	87	164	87
query38	2967	2846	2778	2778
query39	870	811	829	811
query40	250	111	112	111
query41	46	44	44	44
query42	129	107	102	102
query43	443	418	413	413
query44	1175	728	744	728
query45	211	175	178	175
query46	1079	807	775	775
query47	1873	1756	1755	1755
query48	365	291	290	290
query49	885	427	424	424
query50	905	438	434	434
query51	6711	6625	6627	6625
query52	109	86	103	86
query53	251	178	187	178
query54	630	445	448	445
query55	77	71	70	70
query56	269	263	260	260
query57	1142	1042	1041	1041
query58	256	273	266	266
query59	2508	2303	2374	2303
query60	294	260	261	260
query61	97	90	94	90
query62	885	659	646	646
query63	238	181	178	178
query64	5662	1906	1879	1879
query65	3160	3086	3111	3086
query66	1320	331	330	330
query67	15311	15066	14884	14884
query68	4500	571	579	571
query69	736	384	307	307
query70	1125	1086	1111	1086
query71	493	282	288	282
query72	8111	2683	2505	2505
query73	789	325	325	325
query74	6073	5665	5703	5665
query75	4184	2735	2736	2735
query76	3349	1174	1219	1174
query77	687	307	316	307
query78	9414	8949	8896	8896
query79	2826	528	529	528
query80	2181	496	502	496
query81	583	235	228	228
query82	1126	135	132	132
query83	304	167	171	167
query84	270	80	78	78
query85	1198	322	360	322
query86	438	297	302	297
query87	3282	3058	3121	3058
query88	3904	2395	2399	2395
query89	399	299	286	286
query90	1886	199	189	189
query91	127	100	102	100
query92	64	50	50	50
query93	2376	621	612	612
query94	947	300	289	289
query95	382	273	266	266
query96	617	283	279	279
query97	3217	3065	3113	3065
query98	223	218	195	195
query99	1601	1285	1258	1258
Total cold run time: 268186 ms
Total hot run time: 169113 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.9 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 85d87158c426f8e6d7ef3ab82bb69c2c8c440da7, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.22	0.05	0.04
query4	1.69	0.07	0.07
query5	0.48	0.48	0.49
query6	1.14	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.58	0.51	0.53
query10	0.56	0.55	0.57
query11	0.14	0.12	0.11
query12	0.15	0.12	0.13
query13	0.63	0.60	0.60
query14	0.78	0.79	0.80
query15	0.91	0.86	0.85
query16	0.37	0.36	0.36
query17	0.96	1.01	1.01
query18	0.21	0.20	0.21
query19	1.83	1.76	1.72
query20	0.01	0.01	0.01
query21	15.43	0.74	0.64
query22	4.17	7.40	1.16
query23	17.80	1.43	1.31
query24	2.27	0.22	0.22
query25	0.18	0.08	0.08
query26	0.32	0.21	0.21
query27	0.45	0.22	0.22
query28	13.20	0.99	0.97
query29	12.53	3.32	3.29
query30	0.24	0.05	0.06
query31	2.88	0.41	0.40
query32	3.24	0.48	0.47
query33	2.99	2.99	3.00
query34	15.41	4.27	4.26
query35	4.32	4.29	4.30
query36	0.67	0.48	0.48
query37	0.18	0.16	0.15
query38	0.17	0.15	0.14
query39	0.04	0.04	0.04
query40	0.16	0.14	0.12
query41	0.11	0.05	0.06
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 107.71 s
Total hot run time: 29.9 s

@dataroaring dataroaring merged commit b7fa6c1 into apache:master Aug 7, 2024
28 of 30 checks passed
dataroaring pushed a commit that referenced this pull request Aug 7, 2024
…void infinite transaction (#38991)

## Proposed changes

Issue Number: close #38956 

As title.
TangSiyang2001 added a commit to TangSiyang2001/doris that referenced this pull request Aug 8, 2024
…void infinite transaction (apache#38991)

## Proposed changes

Issue Number: close apache#38956 

As title.
dataroaring pushed a commit that referenced this pull request Aug 8, 2024
…void infinite transaction (#38991) (#39108)

## Proposed changes

Issue Number: close #38956 

As title.
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Aug 14, 2024
…void infinite transaction (apache#38991)

## Proposed changes

Issue Number: close apache#38956 

As title.
@wm1581066 wm1581066 added the usercase Important user case type label label Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. area/nereids dev/2.1.6-merged dev/3.0.1-merged doing reviewed usercase Important user case type label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] Insert from empty source will begin an infinite transaction
6 participants