Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](move-memtable) close stream when cancel load stream stub #38912

Merged
merged 1 commit into from
Aug 6, 2024
Merged

[fix](move-memtable) close stream when cancel load stream stub #38912

merged 1 commit into from
Aug 6, 2024

Conversation

kaijchen
Copy link
Contributor

@kaijchen kaijchen commented Aug 6, 2024

Proposed changes

Fix load stream leak when sink v2 meet error.
Tested by check_before_quit.groovy.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@kaijchen
Copy link
Contributor Author

kaijchen commented Aug 6, 2024

run buildall

@github-actions github-actions bot added the doing label Aug 6, 2024
dataroaring
dataroaring approved these changes Aug 6, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 6, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 6, 2024

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 6, 2024

PR approved by anyone and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 6, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 41433 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 175442477126a9dc75eead3becff7a92493e76c1, data reload: false

------ Round 1 ----------------------------------
q1	17699	4055	4076	4055
q2	2016	201	202	201
q3	10439	1277	1272	1272
q4	10165	797	954	797
q5	7671	2917	2955	2917
q6	225	136	142	136
q7	1063	612	605	605
q8	9437	1849	1945	1849
q9	8491	6612	6592	6592
q10	8741	3820	3852	3820
q11	437	250	249	249
q12	431	236	239	236
q13	17764	2933	2929	2929
q14	276	240	247	240
q15	522	497	494	494
q16	536	402	387	387
q17	967	925	925	925
q18	7979	7243	7233	7233
q19	1684	1208	1216	1208
q20	570	325	331	325
q21	5276	4772	4685	4685
q22	350	278	279	278
Total cold run time: 112739 ms
Total hot run time: 41433 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4090	4028	4032	4028
q2	327	217	215	215
q3	2985	2995	3120	2995
q4	2039	2074	1986	1986
q5	5551	5408	5409	5408
q6	221	131	131	131
q7	2125	1770	1843	1770
q8	3279	3306	3327	3306
q9	8624	8678	8820	8678
q10	3932	4078	3886	3886
q11	553	450	459	450
q12	775	608	611	608
q13	16607	3147	3129	3129
q14	313	273	289	273
q15	523	476	500	476
q16	454	417	420	417
q17	1778	1777	1738	1738
q18	8282	7838	7714	7714
q19	1739	1775	1739	1739
q20	2052	1839	1829	1829
q21	5775	5250	5543	5250
q22	540	482	469	469
Total cold run time: 72564 ms
Total hot run time: 56495 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169379 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 175442477126a9dc75eead3becff7a92493e76c1, data reload: false

query1	920	369	366	366
query2	6412	1787	1752	1752
query3	6667	211	236	211
query4	20218	17344	17269	17269
query5	3622	515	523	515
query6	285	170	173	170
query7	4597	302	303	302
query8	264	199	201	199
query9	8510	2392	2374	2374
query10	422	280	266	266
query11	10599	9965	10120	9965
query12	121	90	92	90
query13	1624	382	377	377
query14	9967	6841	7434	6841
query15	210	164	175	164
query16	6954	441	481	441
query17	938	556	534	534
query18	1911	285	277	277
query19	196	143	152	143
query20	94	89	86	86
query21	199	101	97	97
query22	4221	4263	4018	4018
query23	33796	33420	33366	33366
query24	10153	3149	3080	3080
query25	678	408	423	408
query26	1818	152	153	152
query27	2894	283	290	283
query28	7252	2045	2018	2018
query29	1215	442	436	436
query30	238	160	153	153
query31	944	773	761	761
query32	98	56	57	56
query33	685	310	327	310
query34	924	503	514	503
query35	887	788	769	769
query36	1041	903	872	872
query37	199	80	81	80
query38	2977	2863	2916	2863
query39	900	804	800	800
query40	260	112	111	111
query41	45	46	44	44
query42	126	108	112	108
query43	485	431	412	412
query44	1208	756	769	756
query45	209	175	177	175
query46	1087	790	779	779
query47	1789	1703	1718	1703
query48	370	289	292	289
query49	916	421	430	421
query50	894	431	439	431
query51	6669	6683	6684	6683
query52	106	91	89	89
query53	256	184	188	184
query54	640	460	457	457
query55	75	77	73	73
query56	271	264	252	252
query57	1108	1070	1022	1022
query58	283	277	287	277
query59	2621	2408	2331	2331
query60	290	266	287	266
query61	94	92	93	92
query62	874	659	657	657
query63	215	188	181	181
query64	5702	1903	1878	1878
query65	3156	3086	3127	3086
query66	1258	326	327	326
query67	15320	14860	14880	14860
query68	4352	568	596	568
query69	432	300	294	294
query70	1077	1074	1114	1074
query71	441	279	280	279
query72	7113	2690	2486	2486
query73	761	328	325	325
query74	6010	5703	5603	5603
query75	3388	2732	2727	2727
query76	2705	1221	1294	1221
query77	420	311	309	309
query78	9359	8899	8952	8899
query79	1865	592	534	534
query80	1183	508	501	501
query81	566	227	228	227
query82	1051	144	130	130
query83	240	166	171	166
query84	270	76	78	76
query85	1320	300	289	289
query86	401	289	327	289
query87	3287	3125	3070	3070
query88	2992	2412	2403	2403
query89	400	298	295	295
query90	1754	194	187	187
query91	125	103	103	103
query92	56	50	50	50
query93	1534	620	621	620
query94	786	285	285	285
query95	387	323	263	263
query96	600	286	284	284
query97	3216	3057	3081	3057
query98	219	195	196	195
query99	1622	1290	1260	1260
Total cold run time: 261909 ms
Total hot run time: 169379 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.81 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 175442477126a9dc75eead3becff7a92493e76c1, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.22	0.04	0.04
query4	1.69	0.06	0.07
query5	0.49	0.48	0.48
query6	1.13	0.72	0.71
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.58	0.51	0.50
query10	0.55	0.56	0.57
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.62	0.62	0.60
query14	0.76	0.80	0.80
query15	0.89	0.87	0.85
query16	0.36	0.35	0.36
query17	0.99	0.99	0.95
query18	0.23	0.22	0.20
query19	1.84	1.70	1.71
query20	0.02	0.01	0.01
query21	15.44	0.76	0.66
query22	4.14	8.16	1.32
query23	17.91	1.34	1.27
query24	2.28	0.22	0.22
query25	0.19	0.07	0.08
query26	0.32	0.21	0.22
query27	0.45	0.23	0.23
query28	13.18	0.98	0.96
query29	12.52	3.23	3.27
query30	0.26	0.06	0.05
query31	2.86	0.40	0.42
query32	3.25	0.49	0.49
query33	2.93	2.95	2.99
query34	15.45	4.26	4.27
query35	4.32	4.27	4.28
query36	0.69	0.47	0.48
query37	0.19	0.16	0.16
query38	0.15	0.15	0.14
query39	0.04	0.03	0.03
query40	0.16	0.14	0.14
query41	0.10	0.04	0.04
query42	0.05	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 107.77 s
Total hot run time: 29.81 s

liaoxin01
liaoxin01 approved these changes Aug 6, 2024
View reviewed changes
Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@liaoxin01 liaoxin01 merged commit e8d342d into apache:master Aug 6, 2024
29 of 31 checks passed
dataroaring pushed a commit that referenced this pull request Aug 7, 2024
@kaijchen @dataroaring
[fix](move-memtable) close stream when cancel load stream stub (#38912)
c235846 
Fix load stream leak when sink v2 meet error.
Tested by `check_before_quit.groovy`.
kaijchen added a commit to kaijchen/doris that referenced this pull request Aug 7, 2024
@kaijchen
[fix](move-memtable) close stream when cancel load stream stub (apach…
607fc15 
…e#38912)

Fix load stream leak when sink v2 meet error.
Tested by `check_before_quit.groovy`.
@kaijchen kaijchen mentioned this pull request Aug 7, 2024
Merged
yiguolei pushed a commit that referenced this pull request Aug 7, 2024
@kaijchen
[fix](move-memtable) close stream when cancel load stream stub (#38912)…
0a3874f 
… (#39039)

backport #38912
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

@liaoxin01 liaoxin01 liaoxin01 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.1-merged doing reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@kaijchen @doris-robot @liaoxin01 @dataroaring @yiguolei