Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](pipeline) Fix blocked tasks if query is canceled before opening #38206

Merged
merged 1 commit into from
Jul 23, 2024
Merged

[fix](pipeline) Fix blocked tasks if query is canceled before opening #38206

merged 1 commit into from
Jul 23, 2024

Conversation

Gabriel39
Copy link
Contributor

@Gabriel39 Gabriel39 commented Jul 22, 2024 •
edited
Loading

Proposed changes

If query is canceled before opening, pipeline tasks will be always blocked until timeout

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Gabriel39
Copy link
Contributor Author

run buildall

@Gabriel39 Gabriel39 mentioned this pull request Jul 22, 2024
Merged
@Gabriel39 Gabriel39 changed the title [fix](pipeline) Fix blocked tasks if query is canceled before openning [fix](pipeline) Fix blocked tasks if query is canceled before opening Jul 22, 2024
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39961 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 103782d4f1cf8640c17b3c22af4f3d6a3f39f8a3, data reload: false

------ Round 1 ----------------------------------
q1	17621	4447	4286	4286
q2	2005	189	196	189
q3	10445	1151	1121	1121
q4	10187	836	761	761
q5	7512	2688	2624	2624
q6	222	134	134	134
q7	955	595	605	595
q8	9215	2057	2085	2057
q9	8939	6529	6569	6529
q10	8949	3805	3788	3788
q11	472	235	239	235
q12	441	228	222	222
q13	17761	2970	2961	2961
q14	281	240	242	240
q15	529	486	501	486
q16	499	388	388	388
q17	970	711	702	702
q18	8111	7482	7513	7482
q19	7540	1407	1343	1343
q20	799	336	331	331
q21	5033	3198	3203	3198
q22	355	289	292	289
Total cold run time: 118841 ms
Total hot run time: 39961 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4427	4278	4278	4278
q2	381	265	262	262
q3	3061	2941	2943	2941
q4	1996	1728	1731	1728
q5	5552	5540	5473	5473
q6	236	135	129	129
q7	2225	1872	1850	1850
q8	3278	3441	3411	3411
q9	8765	8807	8825	8807
q10	4125	3835	3813	3813
q11	592	492	502	492
q12	834	625	647	625
q13	17320	3158	3166	3158
q14	326	291	295	291
q15	523	499	479	479
q16	514	441	445	441
q17	1829	1505	1520	1505
q18	8095	7958	7740	7740
q19	1768	1565	1504	1504
q20	2164	1868	1848	1848
q21	9036	4889	4734	4734
q22	633	514	540	514
Total cold run time: 77680 ms
Total hot run time: 56023 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174871 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 103782d4f1cf8640c17b3c22af4f3d6a3f39f8a3, data reload: false

query1	920	382	368	368
query2	6441	1937	1884	1884
query3	6638	208	216	208
query4	28568	17492	17456	17456
query5	3657	472	457	457
query6	266	174	160	160
query7	4573	286	287	286
query8	242	198	205	198
query9	8437	2443	2449	2443
query10	434	295	276	276
query11	10540	10110	10159	10110
query12	117	88	86	86
query13	1645	367	371	367
query14	10284	7656	7565	7565
query15	232	169	167	167
query16	7668	453	471	453
query17	1449	572	548	548
query18	1784	292	286	286
query19	199	152	159	152
query20	100	86	85	85
query21	211	133	128	128
query22	4207	4010	3923	3923
query23	34365	33944	33852	33852
query24	10763	2934	3030	2934
query25	619	424	435	424
query26	738	158	160	158
query27	2177	270	281	270
query28	5782	2082	2128	2082
query29	889	665	617	617
query30	260	157	155	155
query31	976	741	760	741
query32	95	54	56	54
query33	663	338	326	326
query34	905	500	499	499
query35	851	744	781	744
query36	1117	996	1008	996
query37	139	82	86	82
query38	2943	2900	2833	2833
query39	917	861	866	861
query40	213	127	131	127
query41	62	47	45	45
query42	120	97	102	97
query43	495	487	467	467
query44	1058	713	712	712
query45	192	162	164	162
query46	1102	728	709	709
query47	1847	1777	1758	1758
query48	381	289	291	289
query49	830	405	411	405
query50	779	395	391	391
query51	6728	6742	6575	6575
query52	102	89	88	88
query53	362	301	290	290
query54	877	451	448	448
query55	73	74	73	73
query56	288	264	266	264
query57	1130	1046	1062	1046
query58	252	258	270	258
query59	2865	2755	2927	2755
query60	321	305	284	284
query61	99	101	100	100
query62	823	656	676	656
query63	332	296	296	296
query64	9200	2222	7489	2222
query65	3161	3162	3106	3106
query66	732	333	327	327
query67	15703	15006	14845	14845
query68	7182	531	534	531
query69	708	444	353	353
query70	1184	1145	1091	1091
query71	517	286	276	276
query72	8733	5750	5373	5373
query73	774	325	325	325
query74	6110	5663	5645	5645
query75	4488	2687	2695	2687
query76	4365	949	884	884
query77	819	315	310	310
query78	9608	8962	9828	8962
query79	4916	514	514	514
query80	1108	477	471	471
query81	578	223	233	223
query82	481	148	140	140
query83	311	175	174	174
query84	270	89	90	89
query85	1112	336	298	298
query86	407	314	317	314
query87	3334	3174	3128	3128
query88	4103	2451	2472	2451
query89	496	386	377	377
query90	2086	201	196	196
query91	132	101	102	101
query92	66	51	50	50
query93	4487	499	498	498
query94	1436	339	282	282
query95	411	331	315	315
query96	622	275	273	273
query97	3233	3041	3036	3036
query98	215	196	193	193
query99	1530	1283	1282	1282
Total cold run time: 288760 ms
Total hot run time: 174871 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.4 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 103782d4f1cf8640c17b3c22af4f3d6a3f39f8a3, data reload: false

query1	0.04	0.03	0.03
query2	0.09	0.04	0.04
query3	0.22	0.05	0.05
query4	1.66	0.11	0.10
query5	0.50	0.48	0.50
query6	1.14	0.74	0.73
query7	0.02	0.01	0.01
query8	0.06	0.05	0.05
query9	0.54	0.51	0.49
query10	0.55	0.53	0.54
query11	0.15	0.12	0.11
query12	0.15	0.12	0.13
query13	0.62	0.59	0.59
query14	0.76	0.78	0.78
query15	0.86	0.81	0.81
query16	0.35	0.37	0.36
query17	0.95	1.03	1.01
query18	0.23	0.23	0.23
query19	1.75	1.71	1.67
query20	0.01	0.01	0.01
query21	15.39	0.76	0.65
query22	4.55	6.63	2.42
query23	18.30	1.47	1.31
query24	2.11	0.24	0.21
query25	0.16	0.09	0.09
query26	0.30	0.21	0.21
query27	0.45	0.24	0.23
query28	13.28	1.03	1.00
query29	12.63	3.46	3.45
query30	0.24	0.06	0.05
query31	2.88	0.39	0.39
query32	3.28	0.48	0.47
query33	2.89	2.95	2.91
query34	16.99	4.38	4.34
query35	4.44	4.43	4.44
query36	0.66	0.47	0.47
query37	0.20	0.15	0.15
query38	0.16	0.15	0.15
query39	0.04	0.03	0.04
query40	0.15	0.12	0.13
query41	0.09	0.05	0.04
query42	0.05	0.05	0.05
query43	0.04	0.04	0.03
Total cold run time: 109.93 s
Total hot run time: 31.4 s

yiguolei pushed a commit that referenced this pull request Jul 22, 2024
@Gabriel39
[fix](pipeline) Fix blocked tasks if query is canceled before opening (…
aff3f29 
…#38200)

## Proposed changes

pick #38206

<!--Describe your changes.-->
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 23, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@Gabriel39 Gabriel39 merged commit cf1bc4a into apache:master Jul 23, 2024
29 of 32 checks passed
dataroaring pushed a commit that referenced this pull request Jul 24, 2024
@Gabriel39 @dataroaring
[fix](pipeline) Fix blocked tasks if query is canceled before opening (…
0b3ec7d 
…#38206)

If query is canceled before opening, pipeline tasks will be always
blocked until timeout
BiteTheDDDDt pushed a commit that referenced this pull request Aug 2, 2024
@Gabriel39 @Doris-Extras
[fix](pipeline) Fix blocked tasks if query is canceled before opening (…
aa087d3 
…#38200)

## Proposed changes

pick #38206

<!--Describe your changes.-->
morningman pushed a commit to morningman/doris that referenced this pull request Aug 18, 2024
@Gabriel39 @morningman
[fix](pipeline) Fix blocked tasks if query is canceled before opening (…
bec44a9 
…apache#38200)

## Proposed changes

pick apache#38206

<!--Describe your changes.-->
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
@yiguolei yiguolei mentioned this pull request Sep 5, 2024
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhangstar333 zhangstar333 zhangstar333 approved these changes

@BiteTheDDDDt BiteTheDDDDt BiteTheDDDDt approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.1-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@Gabriel39 @doris-robot @BiteTheDDDDt @zhangstar333 @wm1581066 @dataroaring