Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bugfix]thread pool resource leak #36990

Merged
merged 6 commits into from
Jul 3, 2024
Merged

Conversation

wuwenchi
Copy link
Contributor

@wuwenchi wuwenchi commented Jun 28, 2024

Proposed changes

  1. When the thread pool is not needed, you need to manually close the thread pool.
  2. Before rolling back, we need to abort the existing task.
  3. Add some exception information

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@wuwenchi
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40746 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e83a97a03e3b1b6ab98d1a03dc942ba879633c66, data reload: false

------ Round 1 ----------------------------------
q1	17754	4448	4283	4283
q2	2020	194	207	194
q3	10521	1247	1030	1030
q4	10221	783	853	783
q5	7472	2712	2668	2668
q6	224	135	136	135
q7	952	604	594	594
q8	9253	2088	2092	2088
q9	8921	6495	6521	6495
q10	8962	3701	3688	3688
q11	474	239	229	229
q12	477	236	226	226
q13	17772	3002	3047	3002
q14	269	223	220	220
q15	520	487	472	472
q16	520	368	372	368
q17	976	686	689	686
q18	8068	7562	7573	7562
q19	3533	1421	1476	1421
q20	658	327	332	327
q21	5093	3978	3936	3936
q22	402	342	339	339
Total cold run time: 115062 ms
Total hot run time: 40746 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4444	4223	4225	4223
q2	384	271	261	261
q3	3006	2864	2903	2864
q4	1991	1667	1628	1628
q5	5705	5529	5540	5529
q6	222	130	127	127
q7	2217	1872	1863	1863
q8	3305	3431	3479	3431
q9	8719	8694	8858	8694
q10	4134	3847	3682	3682
q11	586	520	493	493
q12	869	638	657	638
q13	16382	3181	3199	3181
q14	298	268	258	258
q15	530	509	481	481
q16	498	441	438	438
q17	1846	1535	1514	1514
q18	8015	8023	7790	7790
q19	1839	1482	1628	1482
q20	3057	1899	1861	1861
q21	5151	4983	4738	4738
q22	685	561	544	544
Total cold run time: 73883 ms
Total hot run time: 55720 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172943 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e83a97a03e3b1b6ab98d1a03dc942ba879633c66, data reload: false

query1	906	384	377	377
query2	6460	2433	2313	2313
query3	6642	207	211	207
query4	19180	17659	17000	17000
query5	3641	479	479	479
query6	274	161	166	161
query7	4582	301	308	301
query8	302	290	282	282
query9	8552	2450	2440	2440
query10	573	304	281	281
query11	10525	9931	10010	9931
query12	121	84	86	84
query13	1654	376	380	376
query14	9550	7830	6362	6362
query15	222	182	186	182
query16	7818	265	256	256
query17	1922	549	535	535
query18	1943	272	269	269
query19	200	148	154	148
query20	89	83	78	78
query21	213	136	124	124
query22	4311	4168	3925	3925
query23	33879	33551	33512	33512
query24	10745	2905	2878	2878
query25	578	390	362	362
query26	711	154	150	150
query27	2298	335	339	335
query28	5933	2194	2189	2189
query29	885	651	623	623
query30	255	160	162	160
query31	994	763	751	751
query32	94	53	56	53
query33	674	284	282	282
query34	877	478	484	478
query35	752	641	641	641
query36	1128	955	976	955
query37	145	78	71	71
query38	3003	2810	2837	2810
query39	909	843	830	830
query40	203	135	134	134
query41	53	53	54	53
query42	109	97	103	97
query43	606	583	561	561
query44	1073	737	741	737
query45	186	156	170	156
query46	1080	718	744	718
query47	1858	1739	1799	1739
query48	364	297	300	297
query49	850	417	403	403
query50	756	384	378	378
query51	6822	6718	6689	6689
query52	104	97	99	97
query53	366	304	288	288
query54	882	458	454	454
query55	74	76	75	75
query56	270	254	255	254
query57	1145	1051	1051	1051
query58	233	239	244	239
query59	3427	3209	3166	3166
query60	306	271	279	271
query61	95	92	89	89
query62	591	440	446	440
query63	325	295	296	295
query64	8567	2343	1915	1915
query65	3153	3141	3102	3102
query66	740	319	332	319
query67	15460	14915	15061	14915
query68	7823	574	566	566
query69	655	546	419	419
query70	1254	1129	1113	1113
query71	473	279	270	270
query72	7276	5598	5338	5338
query73	808	327	325	325
query74	5864	5459	5475	5459
query75	4208	2663	2689	2663
query76	4344	984	931	931
query77	657	304	305	304
query78	10480	9925	9775	9775
query79	2546	520	518	518
query80	2195	473	474	473
query81	589	218	218	218
query82	751	107	100	100
query83	287	169	172	169
query84	281	88	84	84
query85	1183	287	331	287
query86	417	327	300	300
query87	3271	3103	3083	3083
query88	4027	2357	2382	2357
query89	485	395	407	395
query90	1837	187	190	187
query91	132	95	100	95
query92	62	48	49	48
query93	3899	526	530	526
query94	1044	187	185	185
query95	408	316	316	316
query96	606	273	269	269
query97	3265	3019	3025	3019
query98	211	205	187	187
query99	1316	841	843	841
Total cold run time: 275892 ms
Total hot run time: 172943 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.76 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e83a97a03e3b1b6ab98d1a03dc942ba879633c66, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.06	0.06
query4	1.67	0.08	0.07
query5	0.50	0.47	0.49
query6	1.14	0.73	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.53	0.55	0.54
query11	0.16	0.12	0.11
query12	0.14	0.12	0.13
query13	0.59	0.58	0.60
query14	0.75	0.76	0.79
query15	0.85	0.81	0.82
query16	0.35	0.37	0.36
query17	1.04	0.95	1.00
query18	0.23	0.23	0.25
query19	1.79	1.76	1.73
query20	0.02	0.01	0.01
query21	15.46	0.73	0.66
query22	4.26	7.29	2.12
query23	18.42	1.35	1.24
query24	2.15	0.24	0.21
query25	0.16	0.09	0.08
query26	0.27	0.19	0.17
query27	0.08	0.08	0.08
query28	13.21	1.02	1.00
query29	12.62	3.28	3.27
query30	0.25	0.07	0.06
query31	2.85	0.39	0.38
query32	3.28	0.46	0.47
query33	2.89	2.89	2.89
query34	17.09	4.44	4.44
query35	4.47	4.51	4.52
query36	0.65	0.47	0.48
query37	0.19	0.15	0.15
query38	0.14	0.14	0.14
query39	0.04	0.03	0.04
query40	0.15	0.14	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.55 s
Total hot run time: 30.76 s

@wuwenchi
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39861 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d2d1428a1f9b1faca0e9738ac30d8b92f861a3ac, data reload: false

------ Round 1 ----------------------------------
q1	17593	4404	4300	4300
q2	2018	185	198	185
q3	10442	1208	1068	1068
q4	10205	816	868	816
q5	7505	2669	2657	2657
q6	219	137	134	134
q7	945	597	592	592
q8	9221	2095	2058	2058
q9	8945	6473	6500	6473
q10	9029	3779	3715	3715
q11	455	233	245	233
q12	420	238	231	231
q13	17795	3008	2994	2994
q14	263	224	219	219
q15	529	485	488	485
q16	513	377	372	372
q17	981	746	695	695
q18	8052	7370	7416	7370
q19	6303	1565	1467	1467
q20	677	327	331	327
q21	4931	3137	3290	3137
q22	389	336	333	333
Total cold run time: 117430 ms
Total hot run time: 39861 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4389	4315	4249	4249
q2	383	258	263	258
q3	2958	2884	2881	2881
q4	2009	1709	1703	1703
q5	5646	5527	5490	5490
q6	224	130	130	130
q7	2232	1892	1837	1837
q8	3270	3419	3467	3419
q9	8733	8691	8794	8691
q10	4138	3842	3657	3657
q11	606	495	497	495
q12	832	627	644	627
q13	17136	3154	3219	3154
q14	304	268	276	268
q15	529	483	465	465
q16	485	427	451	427
q17	1822	1540	1516	1516
q18	8050	8009	7843	7843
q19	1840	1557	1777	1557
q20	3054	1872	1849	1849
q21	5171	5005	4616	4616
q22	938	579	593	579
Total cold run time: 74749 ms
Total hot run time: 55711 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 175036 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d2d1428a1f9b1faca0e9738ac30d8b92f861a3ac, data reload: false

query1	919	408	399	399
query2	6447	2408	2366	2366
query3	6629	209	219	209
query4	19795	17463	17575	17463
query5	3585	495	485	485
query6	270	168	179	168
query7	4577	297	295	295
query8	331	267	299	267
query9	8492	2491	2455	2455
query10	565	302	283	283
query11	10456	10015	10100	10015
query12	114	89	83	83
query13	1654	374	376	374
query14	10319	7783	7781	7781
query15	239	198	186	186
query16	7727	277	275	275
query17	1881	549	516	516
query18	1961	281	276	276
query19	194	151	152	151
query20	88	80	80	80
query21	219	139	134	134
query22	4255	4070	3969	3969
query23	33874	33886	33330	33330
query24	10677	2895	3026	2895
query25	663	378	384	378
query26	711	163	157	157
query27	2409	324	325	324
query28	5930	2202	2186	2186
query29	890	630	627	627
query30	272	171	164	164
query31	993	747	762	747
query32	88	52	58	52
query33	689	289	286	286
query34	908	496	492	492
query35	800	651	642	642
query36	1100	997	998	997
query37	148	79	78	78
query38	2989	2818	2799	2799
query39	927	847	835	835
query40	213	126	125	125
query41	52	51	55	51
query42	107	107	107	107
query43	606	562	531	531
query44	1146	764	748	748
query45	193	173	158	158
query46	1118	715	730	715
query47	1883	1771	1764	1764
query48	368	308	303	303
query49	832	409	405	405
query50	766	382	385	382
query51	6917	6910	6777	6777
query52	100	96	93	93
query53	361	293	297	293
query54	905	446	438	438
query55	77	73	76	73
query56	311	263	260	260
query57	1115	1089	1058	1058
query58	246	226	232	226
query59	3276	3346	3009	3009
query60	304	285	286	285
query61	91	92	88	88
query62	620	465	452	452
query63	343	293	291	291
query64	8618	2279	1757	1757
query65	3258	3155	3152	3152
query66	742	320	328	320
query67	15370	14980	14923	14923
query68	5747	548	548	548
query69	624	488	404	404
query70	1219	1082	1184	1082
query71	478	273	268	268
query72	7179	5366	5427	5366
query73	775	331	325	325
query74	5938	5517	5517	5517
query75	3964	2716	2654	2654
query76	3632	969	1024	969
query77	619	298	293	293
query78	10681	9817	9910	9817
query79	1745	521	540	521
query80	1047	478	476	476
query81	594	234	230	230
query82	765	102	104	102
query83	190	170	166	166
query84	265	84	85	84
query85	1203	288	264	264
query86	440	337	314	314
query87	3279	3143	3108	3108
query88	3580	2371	2371	2371
query89	481	384	400	384
query90	1763	188	192	188
query91	126	97	98	97
query92	60	50	49	49
query93	1732	523	522	522
query94	982	186	192	186
query95	404	331	391	331
query96	580	270	269	269
query97	3248	3101	3115	3101
query98	222	208	206	206
query99	1186	833	821	821
Total cold run time: 269471 ms
Total hot run time: 175036 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.87 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d2d1428a1f9b1faca0e9738ac30d8b92f861a3ac, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.03	0.04
query3	0.23	0.05	0.06
query4	1.69	0.10	0.09
query5	0.49	0.50	0.49
query6	1.13	0.72	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.50	0.50
query10	0.56	0.54	0.54
query11	0.14	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.77	0.79	0.76
query15	0.83	0.81	0.80
query16	0.37	0.36	0.36
query17	0.97	1.00	1.00
query18	0.22	0.24	0.26
query19	1.87	1.78	1.80
query20	0.02	0.01	0.01
query21	15.44	0.77	0.65
query22	3.76	8.00	2.26
query23	18.29	1.34	1.20
query24	2.11	0.22	0.22
query25	0.15	0.08	0.09
query26	0.26	0.17	0.18
query27	0.09	0.08	0.08
query28	13.24	1.00	1.00
query29	12.61	3.32	3.27
query30	0.25	0.06	0.05
query31	2.87	0.38	0.38
query32	3.29	0.47	0.46
query33	2.91	2.88	2.92
query34	17.15	4.41	4.39
query35	4.48	4.44	4.46
query36	0.66	0.46	0.51
query37	0.18	0.15	0.16
query38	0.16	0.14	0.14
query39	0.04	0.04	0.04
query40	0.18	0.14	0.14
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.08 s
Total hot run time: 30.87 s

morningman
morningman previously approved these changes Jun 30, 2024
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 30, 2024
Copy link
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 1, 2024
@wuwenchi
Copy link
Contributor Author

wuwenchi commented Jul 1, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39581 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ee99afa554e37d06848111aa2feb052ff7ff8411, data reload: false

------ Round 1 ----------------------------------
q1	17865	4869	4387	4387
q2	2356	187	190	187
q3	10551	1189	1127	1127
q4	10199	790	893	790
q5	7484	2656	2617	2617
q6	222	139	142	139
q7	955	610	600	600
q8	9229	2065	2061	2061
q9	8916	6425	6454	6425
q10	8935	3708	3718	3708
q11	481	236	237	236
q12	410	234	231	231
q13	17768	2940	2982	2940
q14	259	233	224	224
q15	529	477	482	477
q16	485	373	365	365
q17	971	667	738	667
q18	8313	7445	7266	7266
q19	5941	1477	1399	1399
q20	665	320	342	320
q21	4822	3078	3897	3078
q22	405	337	342	337
Total cold run time: 117761 ms
Total hot run time: 39581 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4353	4210	4223	4210
q2	380	269	267	267
q3	2961	2767	2681	2681
q4	1888	1611	1548	1548
q5	5224	5233	5242	5233
q6	219	136	132	132
q7	2097	1727	1676	1676
q8	3192	3322	3299	3299
q9	8295	8310	8319	8310
q10	3881	3656	3633	3633
q11	592	486	491	486
q12	789	643	615	615
q13	17461	2998	2994	2994
q14	294	264	268	264
q15	529	477	481	477
q16	452	399	412	399
q17	1750	1477	1465	1465
q18	7552	7463	7362	7362
q19	1708	1574	1592	1574
q20	1988	1774	1773	1773
q21	4879	4641	4716	4641
q22	632	557	590	557
Total cold run time: 71116 ms
Total hot run time: 53596 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 171928 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ee99afa554e37d06848111aa2feb052ff7ff8411, data reload: false

query1	909	378	381	378
query2	6460	2476	2345	2345
query3	6662	205	209	205
query4	20833	17526	17139	17139
query5	4171	506	500	500
query6	266	186	165	165
query7	4593	309	294	294
query8	294	284	271	271
query9	8536	2425	2408	2408
query10	624	314	298	298
query11	10642	10022	9950	9950
query12	139	89	87	87
query13	1653	375	366	366
query14	10228	7664	6363	6363
query15	220	183	182	182
query16	7780	265	281	265
query17	1882	528	528	528
query18	1884	271	269	269
query19	196	153	157	153
query20	92	83	84	83
query21	215	129	130	129
query22	4261	3989	4029	3989
query23	33571	33304	32987	32987
query24	11807	2863	2825	2825
query25	689	364	369	364
query26	1784	156	170	156
query27	2968	313	319	313
query28	7340	2107	2075	2075
query29	1059	625	631	625
query30	286	149	152	149
query31	952	749	761	749
query32	99	52	53	52
query33	766	289	285	285
query34	957	484	469	469
query35	749	638	621	621
query36	1125	948	931	931
query37	160	78	76	76
query38	2821	2731	2730	2730
query39	883	821	792	792
query40	269	155	128	128
query41	55	52	54	52
query42	125	99	100	99
query43	593	558	552	552
query44	1243	766	745	745
query45	198	165	161	161
query46	1083	749	735	735
query47	1847	1743	1729	1729
query48	373	308	299	299
query49	1166	422	422	422
query50	773	385	389	385
query51	6821	6750	6807	6750
query52	106	101	96	96
query53	367	295	293	293
query54	1023	444	457	444
query55	78	75	77	75
query56	303	286	267	267
query57	1186	1020	1058	1020
query58	262	249	256	249
query59	3303	3299	3077	3077
query60	341	279	284	279
query61	97	94	95	94
query62	655	436	438	436
query63	326	300	296	296
query64	9903	2278	1814	1814
query65	3170	3118	3138	3118
query66	1387	356	356	356
query67	15397	14863	14931	14863
query68	5574	545	549	545
query69	622	424	329	329
query70	1182	1092	1072	1072
query71	433	283	278	278
query72	7465	5271	5649	5271
query73	786	325	326	325
query74	5836	5366	5431	5366
query75	3760	2653	2673	2653
query76	3548	952	973	952
query77	660	322	309	309
query78	10399	9837	9738	9738
query79	2697	532	520	520
query80	2016	534	476	476
query81	540	223	221	221
query82	763	109	113	109
query83	280	171	174	171
query84	267	92	92	92
query85	1447	281	272	272
query86	482	328	296	296
query87	3263	3078	3090	3078
query88	3712	2398	2396	2396
query89	478	384	380	380
query90	1839	190	190	190
query91	128	102	103	102
query92	66	50	51	50
query93	3141	512	514	512
query94	1159	191	186	186
query95	411	314	322	314
query96	628	283	283	283
query97	3174	3016	3024	3016
query98	215	200	195	195
query99	1205	831	837	831
Total cold run time: 281182 ms
Total hot run time: 171928 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.56 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ee99afa554e37d06848111aa2feb052ff7ff8411, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.08	0.07
query5	0.51	0.47	0.48
query6	1.13	0.73	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.54	0.48	0.49
query10	0.55	0.53	0.54
query11	0.15	0.12	0.12
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.79	0.80	0.78
query15	0.84	0.82	0.81
query16	0.37	0.36	0.36
query17	1.04	1.03	1.04
query18	0.21	0.25	0.26
query19	1.85	1.83	1.69
query20	0.01	0.01	0.01
query21	15.47	0.78	0.67
query22	4.49	7.48	1.69
query23	18.27	1.36	1.32
query24	2.06	0.22	0.22
query25	0.17	0.09	0.09
query26	0.28	0.18	0.17
query27	0.09	0.08	0.08
query28	13.30	1.02	1.01
query29	12.62	3.34	3.35
query30	0.26	0.06	0.06
query31	2.89	0.39	0.39
query32	3.24	0.46	0.46
query33	2.91	2.92	2.89
query34	17.48	4.45	4.45
query35	4.49	4.47	4.49
query36	0.65	0.46	0.47
query37	0.18	0.15	0.15
query38	0.15	0.14	0.14
query39	0.04	0.04	0.04
query40	0.17	0.14	0.14
query41	0.09	0.04	0.04
query42	0.06	0.04	0.05
query43	0.05	0.04	0.03
Total cold run time: 110.24 s
Total hot run time: 30.56 s

@wuwenchi
Copy link
Contributor Author

wuwenchi commented Jul 1, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39763 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 416ce31f76000d0d07c69d614ff64aefb444822c, data reload: false

------ Round 1 ----------------------------------
q1	17585	4406	4297	4297
q2	2025	192	193	192
q3	10458	1193	1115	1115
q4	10203	786	757	757
q5	7468	2627	2607	2607
q6	222	139	136	136
q7	945	596	610	596
q8	9231	2058	2037	2037
q9	8841	6476	6481	6476
q10	8996	3753	3858	3753
q11	461	250	247	247
q12	535	247	238	238
q13	17897	3006	2975	2975
q14	264	221	225	221
q15	520	465	476	465
q16	518	373	370	370
q17	950	661	665	661
q18	8008	7337	7393	7337
q19	5229	1542	1508	1508
q20	650	323	337	323
q21	4892	3114	3945	3114
q22	403	338	340	338
Total cold run time: 116301 ms
Total hot run time: 39763 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4383	4232	4247	4232
q2	366	268	254	254
q3	2999	2856	2842	2842
q4	1994	1724	1674	1674
q5	5671	5493	5499	5493
q6	225	133	129	129
q7	2211	1859	1915	1859
q8	3271	3391	3462	3391
q9	8719	8673	8884	8673
q10	4171	4063	3759	3759
q11	590	502	507	502
q12	846	680	678	678
q13	16394	3185	3179	3179
q14	314	283	280	280
q15	556	487	504	487
q16	506	432	432	432
q17	1824	1542	1522	1522
q18	8353	8026	7870	7870
q19	2349	1768	1666	1666
q20	2472	1955	1909	1909
q21	9508	5193	5096	5096
q22	669	549	589	549
Total cold run time: 78391 ms
Total hot run time: 56476 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174453 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 416ce31f76000d0d07c69d614ff64aefb444822c, data reload: false

query1	2143	400	374	374
query2	9997	2366	2219	2219
query3	6720	202	213	202
query4	19356	17492	17336	17336
query5	3699	476	468	468
query6	259	173	190	173
query7	4593	308	302	302
query8	310	264	284	264
query9	8457	2408	2356	2356
query10	556	280	288	280
query11	10400	10044	10008	10008
query12	120	85	84	84
query13	1637	348	364	348
query14	9191	7624	7461	7461
query15	259	186	189	186
query16	8053	263	260	260
query17	1932	553	513	513
query18	1963	271	265	265
query19	278	152	149	149
query20	90	85	80	80
query21	208	134	125	125
query22	4437	4165	4028	4028
query23	34204	33919	33566	33566
query24	10447	2812	2898	2812
query25	608	394	395	394
query26	719	152	149	149
query27	2223	327	327	327
query28	6003	2151	2130	2130
query29	889	657	625	625
query30	260	156	161	156
query31	964	742	773	742
query32	98	54	52	52
query33	674	278	280	278
query34	890	481	492	481
query35	733	628	635	628
query36	1110	978	969	969
query37	146	77	81	77
query38	2976	2901	2868	2868
query39	886	851	843	843
query40	216	138	129	129
query41	55	55	56	55
query42	120	106	104	104
query43	581	526	540	526
query44	1053	729	734	729
query45	197	167	166	166
query46	1079	744	711	711
query47	1869	1782	1809	1782
query48	386	302	300	300
query49	860	422	421	421
query50	768	390	389	389
query51	6847	6845	6724	6724
query52	105	92	97	92
query53	359	296	289	289
query54	889	456	446	446
query55	74	74	76	74
query56	296	298	281	281
query57	1113	1074	1061	1061
query58	256	240	250	240
query59	3330	3232	3109	3109
query60	310	289	295	289
query61	113	112	111	111
query62	602	463	459	459
query63	314	285	288	285
query64	8607	2346	1854	1854
query65	3197	3081	3092	3081
query66	762	335	332	332
query67	15817	14934	14986	14934
query68	8202	547	546	546
query69	715	438	353	353
query70	1169	1058	1074	1058
query71	509	283	279	279
query72	9125	5619	5627	5619
query73	1224	322	320	320
query74	5980	5543	5498	5498
query75	4599	2655	2647	2647
query76	4626	997	940	940
query77	749	298	299	298
query78	10447	9822	9772	9772
query79	8314	512	509	509
query80	1230	486	473	473
query81	582	224	223	223
query82	746	109	108	108
query83	290	168	174	168
query84	307	87	85	85
query85	1252	280	268	268
query86	391	321	296	296
query87	3342	3099	3103	3099
query88	4214	2364	2343	2343
query89	533	402	381	381
query90	1959	189	185	185
query91	129	97	97	97
query92	56	48	47	47
query93	6846	491	494	491
query94	1232	185	181	181
query95	408	311	311	311
query96	615	262	262	262
query97	3248	3048	3066	3048
query98	208	199	189	189
query99	1147	808	846	808
Total cold run time: 293057 ms
Total hot run time: 174453 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.9 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 416ce31f76000d0d07c69d614ff64aefb444822c, data reload: false

query1	0.04	0.04	0.03
query2	0.09	0.04	0.04
query3	0.22	0.04	0.04
query4	1.69	0.07	0.07
query5	0.49	0.48	0.47
query6	1.14	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.50	0.49
query10	0.54	0.53	0.54
query11	0.14	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.59
query14	0.76	0.78	0.79
query15	0.83	0.82	0.80
query16	0.37	0.37	0.35
query17	0.97	0.96	0.98
query18	0.22	0.25	0.24
query19	1.83	1.73	1.69
query20	0.01	0.01	0.01
query21	15.43	0.75	0.65
query22	4.24	7.06	2.30
query23	18.30	1.35	1.28
query24	2.09	0.23	0.22
query25	0.15	0.09	0.08
query26	0.27	0.18	0.18
query27	0.08	0.08	0.08
query28	13.21	1.02	1.00
query29	12.63	3.26	3.28
query30	0.26	0.06	0.06
query31	2.87	0.39	0.38
query32	3.27	0.47	0.48
query33	2.83	2.92	2.92
query34	17.16	4.49	4.39
query35	4.48	4.49	4.47
query36	0.66	0.46	0.46
query37	0.19	0.16	0.16
query38	0.15	0.14	0.14
query39	0.04	0.04	0.03
query40	0.17	0.14	0.15
query41	0.09	0.06	0.06
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.36 s
Total hot run time: 30.9 s

@wuwenchi wuwenchi changed the title [bugfix]Thread resource leak [bugfix]thread pool resource leak Jul 2, 2024
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 3, 2024
Copy link
Contributor

github-actions bot commented Jul 3, 2024

PR approved by at least one committer and no changes requested.

Copy link
Contributor

@kaka11chen kaka11chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman merged commit 0809e19 into apache:master Jul 3, 2024
26 of 30 checks passed
morningman pushed a commit that referenced this pull request Jul 4, 2024
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
## Proposed changes

1. When the thread pool is not needed, you need to manually close the
thread pool.
2. Before rolling back, we need to abort the existing task.
3. Add some exception information
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants