Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Optimize] Add session variable max_fetch_remote_schema_tablet_count to limit tablets size for remote schema fetch #37217

Merged
merged 4 commits into from
Jul 8, 2024
Merged

[Optimize] Add session variable max_fetch_remote_schema_tablet_count to limit tablets size for remote schema fetch #37217

merged 4 commits into from
Jul 8, 2024

Conversation

eldenmoon
Copy link
Member

Describing tables with many partitions and tablets can cause high CPU usage. To mitigate this, we estimate and pick sample tablets for schema fetch, reducing the overall cost.

@eldenmoon
[Optimize] Add session variable `max_fetch_remote_schema_tablet_count…
4709886 
…` to limit tablets size for remote schema fetch

Describing tables with many partitions and tablets can cause high CPU usage. To mitigate this, we estimate and pick sample tablets for schema fetch, reducing the overall cost.
@eldenmoon
add test
6be039c
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@eldenmoon
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39529 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6be039c22ee0cde2af8bbc38381a0634afed1dbe, data reload: false

------ Round 1 ----------------------------------
q1	17604	4247	4440	4247
q2	2013	193	192	192
q3	10471	1234	1131	1131
q4	10196	865	778	778
q5	7460	2634	2635	2634
q6	219	136	141	136
q7	952	598	616	598
q8	9236	2087	2066	2066
q9	9007	6479	6394	6394
q10	8944	3760	3698	3698
q11	448	243	237	237
q12	432	239	233	233
q13	17774	2959	2978	2959
q14	260	239	220	220
q15	528	483	481	481
q16	493	388	373	373
q17	973	650	733	650
q18	7970	7378	7398	7378
q19	5201	1517	1351	1351
q20	657	313	341	313
q21	4891	3114	3864	3114
q22	405	348	346	346
Total cold run time: 116134 ms
Total hot run time: 39529 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4380	4256	4288	4256
q2	367	263	269	263
q3	3017	2767	2879	2767
q4	2036	1776	1684	1684
q5	5651	5519	5472	5472
q6	224	130	130	130
q7	2179	1794	1881	1794
q8	3258	3404	3410	3404
q9	8744	8637	8814	8637
q10	4119	3935	3714	3714
q11	600	501	516	501
q12	831	659	649	649
q13	15894	3178	3183	3178
q14	298	285	269	269
q15	546	499	477	477
q16	482	418	429	418
q17	1798	1536	1492	1492
q18	8156	7935	7748	7748
q19	3008	1698	1501	1501
q20	2194	1882	1854	1854
q21	5212	4932	4783	4783
q22	610	535	556	535
Total cold run time: 73604 ms
Total hot run time: 55526 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173369 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6be039c22ee0cde2af8bbc38381a0634afed1dbe, data reload: false

query1	920	393	366	366
query2	6452	2313	2432	2313
query3	6627	200	209	200
query4	19803	17587	17368	17368
query5	3679	485	468	468
query6	252	173	157	157
query7	4589	300	298	298
query8	321	298	299	298
query9	8611	2399	2349	2349
query10	593	300	288	288
query11	10602	10041	9834	9834
query12	122	84	84	84
query13	1646	375	385	375
query14	10090	7837	7108	7108
query15	233	192	188	188
query16	7785	268	256	256
query17	1922	535	529	529
query18	1936	271	267	267
query19	202	146	149	146
query20	88	83	82	82
query21	213	152	145	145
query22	4443	4043	4130	4043
query23	34003	33660	33482	33482
query24	11137	2951	2832	2832
query25	616	384	389	384
query26	1125	160	167	160
query27	2272	324	328	324
query28	6247	2124	2123	2123
query29	914	669	630	630
query30	231	154	168	154
query31	982	747	755	747
query32	97	53	53	53
query33	756	288	283	283
query34	989	478	494	478
query35	772	631	627	627
query36	1180	997	960	960
query37	138	83	79	79
query38	2946	2804	2979	2804
query39	897	825	846	825
query40	207	128	129	128
query41	56	53	58	53
query42	117	107	105	105
query43	566	559	543	543
query44	1181	754	782	754
query45	194	167	171	167
query46	1086	751	729	729
query47	1836	1744	1729	1729
query48	366	315	300	300
query49	853	404	402	402
query50	761	385	386	385
query51	6887	6767	6742	6742
query52	110	91	95	91
query53	356	284	307	284
query54	887	454	445	445
query55	74	74	76	74
query56	288	257	256	256
query57	1128	1048	1062	1048
query58	244	240	246	240
query59	3451	3400	3142	3142
query60	315	269	280	269
query61	97	89	109	89
query62	616	444	437	437
query63	321	328	298	298
query64	8746	2235	1741	1741
query65	3162	3104	3146	3104
query66	739	320	343	320
query67	15158	14990	14799	14799
query68	4583	540	539	539
query69	521	386	326	326
query70	1200	1138	1166	1138
query71	409	283	276	276
query72	7012	5234	5434	5234
query73	749	327	326	326
query74	5883	5442	5499	5442
query75	3358	2675	2628	2628
query76	2697	952	958	952
query77	524	306	310	306
query78	10256	9901	9763	9763
query79	2402	514	521	514
query80	2514	470	483	470
query81	609	213	218	213
query82	886	103	104	103
query83	304	176	175	175
query84	269	93	86	86
query85	2043	286	275	275
query86	497	321	325	321
query87	3274	3015	3067	3015
query88	3802	2473	2453	2453
query89	486	394	399	394
query90	1811	185	189	185
query91	126	99	99	99
query92	61	47	47	47
query93	2455	514	519	514
query94	1143	186	189	186
query95	411	316	314	314
query96	593	268	267	267
query97	3185	3056	3016	3016
query98	218	210	191	191
query99	1348	858	820	820
Total cold run time: 271862 ms
Total hot run time: 173369 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.93 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6be039c22ee0cde2af8bbc38381a0634afed1dbe, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.03
query3	0.22	0.05	0.05
query4	1.67	0.08	0.09
query5	0.49	0.49	0.49
query6	1.13	0.72	0.73
query7	0.02	0.01	0.02
query8	0.05	0.05	0.04
query9	0.55	0.48	0.48
query10	0.55	0.56	0.53
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.58	0.58
query14	0.77	0.77	0.80
query15	0.85	0.82	0.81
query16	0.35	0.37	0.36
query17	0.97	1.03	0.98
query18	0.22	0.26	0.26
query19	1.79	1.67	1.67
query20	0.02	0.01	0.01
query21	15.44	0.74	0.65
query22	4.04	6.97	2.36
query23	18.28	1.40	1.25
query24	2.15	0.22	0.22
query25	0.15	0.09	0.10
query26	0.26	0.18	0.17
query27	0.08	0.08	0.08
query28	13.29	1.02	1.01
query29	12.59	3.37	3.34
query30	0.25	0.06	0.06
query31	2.86	0.38	0.39
query32	3.28	0.49	0.46
query33	2.89	2.95	2.92
query34	17.26	4.36	4.46
query35	4.49	4.50	4.42
query36	0.65	0.46	0.47
query37	0.20	0.16	0.15
query38	0.16	0.15	0.14
query39	0.04	0.04	0.03
query40	0.20	0.13	0.15
query41	0.09	0.04	0.04
query42	0.06	0.05	0.05
query43	0.04	0.04	0.03
Total cold run time: 109.41 s
Total hot run time: 30.93 s

@eldenmoon eldenmoon requested a review from csun5285 July 3, 2024 10:52
csun5285
csun5285 approved these changes Jul 3, 2024
View reviewed changes
Copy link
Contributor

@csun5285 csun5285 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 3, 2024

PR approved by anyone and no changes requested.

@eldenmoon
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39827 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 98bc206353eaa4b51698b4b8d490f4ae24779340, data reload: false

------ Round 1 ----------------------------------
q1	17641	4277	4271	4271
q2	2011	191	191	191
q3	10449	1171	1078	1078
q4	10191	835	742	742
q5	7482	2660	2655	2655
q6	225	138	137	137
q7	954	605	607	605
q8	9229	2090	2050	2050
q9	8941	6475	6445	6445
q10	8821	3698	3728	3698
q11	461	247	239	239
q12	432	239	230	230
q13	17777	3023	3021	3021
q14	268	233	218	218
q15	524	498	506	498
q16	528	400	388	388
q17	953	638	781	638
q18	7952	7403	7400	7400
q19	7950	1468	1519	1468
q20	662	328	325	325
q21	4827	3188	3252	3188
q22	403	350	342	342
Total cold run time: 118681 ms
Total hot run time: 39827 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4493	4220	4228	4220
q2	388	282	257	257
q3	2961	2892	2917	2892
q4	2001	1616	1727	1616
q5	5589	5471	5417	5417
q6	236	132	132	132
q7	2235	1862	1842	1842
q8	3319	3420	3410	3410
q9	8683	8690	8838	8690
q10	4164	3738	3753	3738
q11	592	506	510	506
q12	846	687	675	675
q13	16043	3164	3202	3164
q14	300	285	279	279
q15	532	509	479	479
q16	509	422	435	422
q17	1831	1573	1490	1490
q18	8155	7906	7652	7652
q19	1754	1620	1696	1620
q20	2170	1873	1857	1857
q21	5066	4957	4752	4752
q22	644	575	574	574
Total cold run time: 72511 ms
Total hot run time: 55684 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173779 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 98bc206353eaa4b51698b4b8d490f4ae24779340, data reload: false

query1	920	375	368	368
query2	6447	2386	2399	2386
query3	6635	203	214	203
query4	19096	17490	17387	17387
query5	3669	490	494	490
query6	257	174	168	168
query7	4581	294	293	293
query8	301	290	305	290
query9	8815	2490	2456	2456
query10	565	291	279	279
query11	10692	9922	9925	9922
query12	117	90	84	84
query13	1635	369	369	369
query14	9685	7659	7699	7659
query15	235	192	187	187
query16	7253	317	302	302
query17	1356	536	524	524
query18	1842	287	278	278
query19	201	168	164	164
query20	93	83	84	83
query21	222	137	129	129
query22	4212	4108	4152	4108
query23	34107	33554	33733	33554
query24	11442	2889	2872	2872
query25	635	434	426	426
query26	722	166	164	164
query27	2338	337	327	327
query28	6328	2220	2170	2170
query29	882	618	628	618
query30	245	160	155	155
query31	989	774	747	747
query32	96	54	60	54
query33	747	295	328	295
query34	1045	476	494	476
query35	759	655	631	631
query36	1152	1010	999	999
query37	134	76	79	76
query38	2990	2853	2783	2783
query39	869	838	841	838
query40	210	125	125	125
query41	53	51	50	50
query42	118	106	101	101
query43	605	553	575	553
query44	1286	739	751	739
query45	191	160	163	160
query46	1080	777	717	717
query47	1864	1756	1806	1756
query48	363	295	300	295
query49	842	431	429	429
query50	764	389	379	379
query51	6875	6821	6757	6757
query52	102	96	102	96
query53	355	290	296	290
query54	878	464	445	445
query55	75	74	73	73
query56	296	267	278	267
query57	1122	1023	1046	1023
query58	258	250	247	247
query59	3371	3170	3048	3048
query60	293	284	292	284
query61	94	92	92	92
query62	628	445	458	445
query63	319	293	287	287
query64	9004	2259	1755	1755
query65	3188	3075	3083	3075
query66	762	325	329	325
query67	15295	14962	14918	14918
query68	4487	528	547	528
query69	646	425	324	324
query70	1155	1167	1098	1098
query71	405	288	283	283
query72	7605	5476	5232	5232
query73	744	320	321	320
query74	6000	5485	5535	5485
query75	3474	2689	2690	2689
query76	2686	1015	925	925
query77	651	320	322	320
query78	9654	9071	8843	8843
query79	3121	518	513	513
query80	1965	470	470	470
query81	606	224	222	222
query82	783	115	115	115
query83	324	168	168	168
query84	274	90	90	90
query85	1571	280	268	268
query86	479	324	295	295
query87	3324	3118	3139	3118
query88	4411	2376	2360	2360
query89	487	374	373	373
query90	1790	187	189	187
query91	128	100	98	98
query92	60	48	48	48
query93	4977	526	511	511
query94	1191	206	210	206
query95	485	314	314	314
query96	617	263	263	263
query97	3182	3053	3033	3033
query98	208	205	202	202
query99	1238	861	833	833
Total cold run time: 273328 ms
Total hot run time: 173779 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.34 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 98bc206353eaa4b51698b4b8d490f4ae24779340, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.49	0.48	0.49
query6	1.13	0.72	0.73
query7	0.02	0.01	0.02
query8	0.05	0.05	0.04
query9	0.55	0.48	0.49
query10	0.55	0.54	0.54
query11	0.15	0.11	0.11
query12	0.15	0.13	0.13
query13	0.59	0.58	0.58
query14	0.75	0.78	0.78
query15	0.85	0.81	0.82
query16	0.37	0.37	0.36
query17	0.96	0.98	0.99
query18	0.23	0.24	0.24
query19	1.76	1.72	1.68
query20	0.02	0.01	0.01
query21	15.40	0.78	0.67
query22	4.69	7.52	1.70
query23	18.28	1.51	1.19
query24	2.22	0.22	0.23
query25	0.16	0.08	0.08
query26	0.27	0.19	0.18
query27	0.08	0.08	0.08
query28	13.19	1.03	1.01
query29	12.65	3.27	3.27
query30	0.26	0.06	0.06
query31	2.88	0.39	0.38
query32	3.27	0.49	0.46
query33	2.86	2.94	2.88
query34	17.24	4.44	4.49
query35	4.47	4.49	4.49
query36	0.65	0.47	0.47
query37	0.20	0.16	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.17	0.14	0.14
query41	0.10	0.05	0.05
query42	0.05	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.98 s
Total hot run time: 30.34 s

@xiaokang xiaokang added usercase Important user case type label dev/3.0.x labels Jul 8, 2024
@eldenmoon
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40098 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dc7f8fab5e1b7a3245edb58d5a2bdd327e719e47, data reload: false

------ Round 1 ----------------------------------
q1	18192	4525	4382	4382
q2	2562	209	190	190
q3	11448	1181	1152	1152
q4	10987	793	782	782
q5	7567	2699	2665	2665
q6	224	145	137	137
q7	967	601	595	595
q8	9214	2044	2078	2044
q9	8681	6510	6467	6467
q10	8894	3743	3753	3743
q11	442	237	237	237
q12	390	226	227	226
q13	18839	2986	3015	2986
q14	252	221	227	221
q15	530	489	491	489
q16	476	375	366	366
q17	951	651	699	651
q18	7989	7514	7424	7424
q19	1696	1537	1505	1505
q20	698	333	324	324
q21	4915	3172	3824	3172
q22	392	340	344	340
Total cold run time: 116306 ms
Total hot run time: 40098 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4306	4294	4217	4217
q2	375	273	249	249
q3	2995	2761	2700	2700
q4	1874	1591	1560	1560
q5	5249	5274	5245	5245
q6	218	131	130	130
q7	2081	1746	1772	1746
q8	3180	3319	3317	3317
q9	8330	8333	8376	8333
q10	3877	3648	3606	3606
q11	580	478	473	473
q12	756	629	590	590
q13	17748	2963	2999	2963
q14	300	285	261	261
q15	521	487	478	478
q16	465	418	429	418
q17	1775	1471	1474	1471
q18	7466	7431	7323	7323
q19	4008	1604	1613	1604
q20	2014	1770	1782	1770
q21	4922	4800	4735	4735
q22	632	512	551	512
Total cold run time: 73672 ms
Total hot run time: 53701 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172821 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dc7f8fab5e1b7a3245edb58d5a2bdd327e719e47, data reload: false

query1	900	368	353	353
query2	6481	2608	2401	2401
query3	6661	210	215	210
query4	27851	17350	17145	17145
query5	4199	481	456	456
query6	271	161	158	158
query7	4586	298	293	293
query8	328	308	293	293
query9	8507	2359	2346	2346
query10	606	304	286	286
query11	10680	9972	9997	9972
query12	133	89	83	83
query13	1654	379	377	377
query14	9741	7382	7479	7382
query15	232	187	186	186
query16	7672	318	308	308
query17	1576	543	546	543
query18	1938	295	277	277
query19	195	157	157	157
query20	89	84	84	84
query21	218	135	127	127
query22	4270	3989	3945	3945
query23	33760	33443	33414	33414
query24	12351	2937	2830	2830
query25	676	392	390	390
query26	1806	155	158	155
query27	2973	323	322	322
query28	7671	2056	2050	2050
query29	1126	647	617	617
query30	294	150	146	146
query31	957	737	748	737
query32	95	55	56	55
query33	782	304	305	304
query34	960	486	486	486
query35	725	631	639	631
query36	1111	933	915	915
query37	280	81	78	78
query38	2834	2704	2750	2704
query39	863	794	800	794
query40	288	127	130	127
query41	57	53	56	53
query42	112	101	101	101
query43	610	567	549	549
query44	1237	754	732	732
query45	200	160	163	160
query46	1069	716	722	716
query47	1891	1787	1798	1787
query48	387	302	308	302
query49	1168	429	444	429
query50	783	390	393	390
query51	6972	6868	6804	6804
query52	101	93	93	93
query53	369	293	292	292
query54	932	454	454	454
query55	75	73	73	73
query56	298	291	291	291
query57	1142	1021	1052	1021
query58	368	238	249	238
query59	3721	3232	3356	3232
query60	304	279	275	275
query61	97	96	96	96
query62	617	447	425	425
query63	326	283	279	279
query64	10507	2209	1611	1611
query65	3209	3092	3095	3092
query66	1250	343	324	324
query67	15517	15096	14957	14957
query68	9351	550	558	550
query69	721	434	386	386
query70	1156	1063	1102	1063
query71	521	283	281	281
query72	9206	5475	5360	5360
query73	1374	322	324	322
query74	5893	5459	5460	5459
query75	5218	2662	2718	2662
query76	5659	1053	899	899
query77	820	319	355	319
query78	10355	9051	8877	8877
query79	11028	516	515	515
query80	1125	476	473	473
query81	575	218	215	215
query82	728	105	106	105
query83	326	161	162	161
query84	277	89	88	88
query85	1441	304	327	304
query86	369	334	268	268
query87	3360	3105	3106	3105
query88	5136	2464	2468	2464
query89	545	382	371	371
query90	2180	186	187	186
query91	132	102	99	99
query92	60	49	47	47
query93	7252	501	504	501
query94	1322	209	209	209
query95	397	310	311	310
query96	611	272	272	272
query97	3192	3015	3004	3004
query98	211	199	194	194
query99	1193	839	847	839
Total cold run time: 313393 ms
Total hot run time: 172821 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.37 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dc7f8fab5e1b7a3245edb58d5a2bdd327e719e47, data reload: false

query1	0.04	0.03	0.04
query2	0.08	0.04	0.04
query3	0.23	0.04	0.05
query4	1.68	0.06	0.07
query5	0.53	0.48	0.48
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.55	0.48	0.49
query10	0.53	0.54	0.53
query11	0.16	0.11	0.11
query12	0.15	0.13	0.13
query13	0.58	0.58	0.58
query14	0.75	0.79	0.76
query15	0.86	0.81	0.82
query16	0.36	0.37	0.35
query17	0.97	1.02	1.03
query18	0.22	0.24	0.24
query19	1.79	1.84	1.66
query20	0.02	0.01	0.01
query21	15.45	0.76	0.66
query22	4.64	6.58	1.80
query23	18.28	1.33	1.24
query24	2.15	0.23	0.23
query25	0.14	0.09	0.09
query26	0.30	0.21	0.22
query27	0.46	0.23	0.22
query28	13.28	1.02	0.99
query29	12.61	3.28	3.29
query30	0.26	0.05	0.06
query31	2.86	0.37	0.37
query32	3.31	0.47	0.46
query33	2.88	2.88	2.87
query34	17.01	4.37	4.35
query35	4.39	4.37	4.39
query36	0.63	0.48	0.50
query37	0.18	0.16	0.16
query38	0.16	0.14	0.14
query39	0.05	0.03	0.04
query40	0.16	0.12	0.13
query41	0.10	0.05	0.05
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.1 s
Total hot run time: 30.37 s

xiaokang
xiaokang approved these changes Jul 8, 2024
View reviewed changes
Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 8, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 8, 2024

PR approved by at least one committer and no changes requested.

@eldenmoon eldenmoon merged commit 8a52371 into apache:master Jul 8, 2024
25 of 27 checks passed
@eldenmoon eldenmoon deleted the var-fetch-opt branch July 8, 2024 15:35
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Jul 9, 2024
@eldenmoon
[Optimize] Add session variable `max_fetch_remote_schema_tablet_count…
7c7676e 
…` to limit tablets size for remote schema fetch (apache#37217)

Describing tables with many partitions and tablets can cause high CPU
usage. To mitigate this, we estimate and pick sample tablets for schema
fetch, reducing the overall cost.
@eldenmoon eldenmoon mentioned this pull request Jul 9, 2024
Merged
eldenmoon added a commit that referenced this pull request Jul 11, 2024
@eldenmoon
[Optimize] Add session variable `max_fetch_remote_schema_tablet_count… (
e1cb568 
#37505)

pick from #37217
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Jul 16, 2024
@eldenmoon
[Optimize] Add session variable `max_fetch_remote_schema_tablet_count…
55c0693 
…` to limit tablets size for remote schema fetch (apache#37217)

Describing tables with many partitions and tablets can cause high CPU
usage. To mitigate this, we estimate and pick sample tablets for schema
fetch, reducing the overall cost.
@eldenmoon eldenmoon mentioned this pull request Jul 16, 2024
Merged
xiaokang pushed a commit that referenced this pull request Jul 17, 2024
@eldenmoon
[Pick](Variant) pick some fix (#37926)
2994232 
#37674
#37839
#37883
#37857
#37794
#37640
#37557
#37217
#37121
@yiguolei yiguolei mentioned this pull request Jul 19, 2024
Open
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@csun5285 csun5285 csun5285 approved these changes

@xiaokang xiaokang xiaokang approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.0-merged reviewed usercase Important user case type label
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@eldenmoon @doris-robot @xiaokang @csun5285 @wm1581066