Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](cloud-balance) use right file size instead of block size #38939

Merged
merged 3 commits into from
Aug 7, 2024
Merged

[fix](cloud-balance) use right file size instead of block size #38939

merged 3 commits into from
Aug 7, 2024

Conversation

dataroaring
Copy link
Contributor

introduced by #34191.

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@dataroaring
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 6, 2024

clang-tidy review says "All clean, LGTM! 👍"

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 6, 2024

PR approved by anyone and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 6, 2024

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 6, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 42093 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4b7e5cea63c17ea5668c76c2abc9dc2b87540721, data reload: false

------ Round 1 ----------------------------------
q1	17710	4393	4129	4129
q2	2036	201	196	196
q3	10615	1419	1427	1419
q4	10278	855	927	855
q5	7661	3009	2995	2995
q6	226	141	140	140
q7	1067	613	614	613
q8	9448	1926	2006	1926
q9	8530	6613	6690	6613
q10	8710	3865	3865	3865
q11	439	258	247	247
q12	421	227	225	225
q13	17760	2937	2957	2937
q14	276	245	252	245
q15	528	488	499	488
q16	522	388	383	383
q17	971	935	929	929
q18	8676	7364	7241	7241
q19	1424	1232	1228	1228
q20	556	324	344	324
q21	5502	4816	4889	4816
q22	355	286	279	279
Total cold run time: 113711 ms
Total hot run time: 42093 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4163	4086	4034	4034
q2	326	224	222	222
q3	3032	3127	3218	3127
q4	2083	1982	2008	1982
q5	5576	5525	5468	5468
q6	223	132	133	132
q7	2190	1849	1769	1769
q8	3348	3399	3382	3382
q9	8753	8782	8764	8764
q10	3998	3980	3982	3980
q11	560	469	463	463
q12	754	578	574	574
q13	13253	3136	3131	3131
q14	294	277	275	275
q15	535	499	482	482
q16	460	425	424	424
q17	1798	1727	1759	1727
q18	8248	7771	7781	7771
q19	1735	1760	1743	1743
q20	2082	1890	1838	1838
q21	5874	5590	5447	5447
q22	516	482	474	474
Total cold run time: 69801 ms
Total hot run time: 57209 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 168330 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4b7e5cea63c17ea5668c76c2abc9dc2b87540721, data reload: false

query1	919	378	378	378
query2	6478	1677	1745	1677
query3	6650	216	223	216
query4	20043	17447	17190	17190
query5	3636	505	509	505
query6	321	170	164	164
query7	4590	304	289	289
query8	247	191	189	189
query9	8528	2335	2318	2318
query10	414	269	262	262
query11	10546	10214	10046	10046
query12	126	88	89	88
query13	1628	364	368	364
query14	9770	6214	6006	6006
query15	210	165	162	162
query16	6961	449	452	449
query17	930	544	542	542
query18	1918	291	286	286
query19	199	153	145	145
query20	97	87	86	86
query21	212	101	98	98
query22	4373	4066	4143	4066
query23	33599	33906	33463	33463
query24	9499	3124	3063	3063
query25	682	431	454	431
query26	1225	165	158	158
query27	3063	287	289	287
query28	7603	2036	1983	1983
query29	1008	443	456	443
query30	236	156	159	156
query31	977	816	757	757
query32	102	55	56	55
query33	670	316	330	316
query34	924	490	509	490
query35	884	756	742	742
query36	1050	868	865	865
query37	237	82	81	81
query38	2870	2838	2746	2746
query39	862	827	820	820
query40	283	112	115	112
query41	45	45	42	42
query42	116	100	97	97
query43	491	422	441	422
query44	1155	723	728	723
query45	207	178	178	178
query46	1102	804	772	772
query47	1845	1759	1695	1695
query48	362	288	287	287
query49	946	423	419	419
query50	907	436	434	434
query51	6791	6605	6648	6605
query52	100	86	93	86
query53	255	186	177	177
query54	641	445	460	445
query55	76	71	72	71
query56	284	260	243	243
query57	1144	1043	1049	1043
query58	280	288	281	281
query59	2606	2399	2437	2399
query60	305	288	284	284
query61	96	91	96	91
query62	903	664	652	652
query63	213	178	182	178
query64	4731	1897	1902	1897
query65	3167	3113	3083	3083
query66	1019	329	328	328
query67	15476	14941	14771	14771
query68	4317	556	570	556
query69	451	321	302	302
query70	1113	980	1104	980
query71	366	281	283	281
query72	7197	2682	2486	2486
query73	767	331	324	324
query74	6088	5696	5705	5696
query75	3428	2704	2727	2704
query76	2095	1240	1303	1240
query77	462	306	310	306
query78	9514	8921	8930	8921
query79	1847	534	531	531
query80	1050	511	506	506
query81	547	222	231	222
query82	1033	141	126	126
query83	230	165	170	165
query84	262	77	76	76
query85	1149	323	295	295
query86	404	337	294	294
query87	3288	3114	3100	3100
query88	2922	2408	2439	2408
query89	395	298	287	287
query90	1681	192	196	192
query91	123	100	104	100
query92	59	50	49	49
query93	1465	631	637	631
query94	792	299	292	292
query95	377	270	259	259
query96	601	293	359	293
query97	3253	3053	3085	3053
query98	217	193	196	193
query99	1615	1283	1265	1265
Total cold run time: 258941 ms
Total hot run time: 168330 ms

@dataroaring
Copy link
Contributor Author

run buildall

@dataroaring dataroaring added usercase Important user case type label dev/3.0.x labels Aug 6, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 41912 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4d673208635540e2790fc74e002950977ec6baa8, data reload: false

------ Round 1 ----------------------------------
q1	17676	4285	4119	4119
q2	2052	219	201	201
q3	10546	1412	1380	1380
q4	10217	804	948	804
q5	7585	3101	3014	3014
q6	228	138	143	138
q7	1080	625	610	610
q8	9438	1860	1971	1860
q9	8435	6648	6631	6631
q10	8753	3887	3895	3887
q11	436	253	249	249
q12	409	228	228	228
q13	17763	2917	2972	2917
q14	276	245	242	242
q15	527	496	507	496
q16	542	386	395	386
q17	1003	914	932	914
q18	8087	7415	7222	7222
q19	2816	1189	1236	1189
q20	561	339	347	339
q21	5334	4838	4796	4796
q22	366	290	294	290
Total cold run time: 114130 ms
Total hot run time: 41912 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4132	4086	4059	4059
q2	335	240	223	223
q3	3010	3014	3015	3014
q4	1913	1882	1879	1879
q5	5252	5298	5258	5258
q6	218	132	130	130
q7	2093	1745	1680	1680
q8	3224	3315	3259	3259
q9	8357	8356	8295	8295
q10	3773	3876	3888	3876
q11	551	452	456	452
q12	723	569	559	559
q13	16806	2975	2942	2942
q14	302	263	257	257
q15	523	474	478	474
q16	446	414	406	406
q17	1736	1721	1704	1704
q18	7851	7398	7187	7187
q19	1696	1729	1707	1707
q20	1975	1784	1776	1776
q21	5513	5268	5338	5268
q22	521	450	464	450
Total cold run time: 70950 ms
Total hot run time: 54855 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 168152 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4d673208635540e2790fc74e002950977ec6baa8, data reload: false

query1	917	378	368	368
query2	6490	1686	1645	1645
query3	6711	213	226	213
query4	20122	17351	17543	17351
query5	4331	522	514	514
query6	282	175	167	167
query7	4609	314	298	298
query8	247	205	194	194
query9	8508	2349	2330	2330
query10	437	274	268	268
query11	10387	9887	9900	9887
query12	138	97	88	88
query13	1641	377	375	375
query14	8673	6749	7271	6749
query15	202	165	167	165
query16	7106	464	471	464
query17	978	582	571	571
query18	1928	292	285	285
query19	196	155	152	152
query20	100	87	86	86
query21	207	101	102	101
query22	4261	3907	3883	3883
query23	33938	33031	33051	33031
query24	10346	3107	3052	3052
query25	693	373	377	373
query26	1827	153	154	153
query27	3001	275	273	273
query28	7082	1968	1951	1951
query29	1364	432	412	412
query30	293	157	148	148
query31	926	764	738	738
query32	107	55	62	55
query33	711	310	315	310
query34	927	472	470	470
query35	846	730	730	730
query36	986	843	858	843
query37	318	91	78	78
query38	2858	2817	2776	2776
query39	876	812	790	790
query40	283	113	112	112
query41	47	44	47	44
query42	123	103	100	100
query43	466	411	420	411
query44	1186	717	724	717
query45	212	175	180	175
query46	1087	826	789	789
query47	1787	1722	1739	1722
query48	359	288	285	285
query49	1180	417	435	417
query50	900	430	430	430
query51	6761	6657	6746	6657
query52	103	86	96	86
query53	258	184	175	175
query54	628	452	454	452
query55	81	80	77	77
query56	283	284	265	265
query57	1169	1015	1028	1015
query58	272	282	268	268
query59	2660	2456	2382	2382
query60	309	285	278	278
query61	97	94	95	94
query62	914	649	669	649
query63	222	178	177	177
query64	5862	1941	1887	1887
query65	3165	3082	3079	3079
query66	1435	330	341	330
query67	15201	14836	14681	14681
query68	4346	577	576	576
query69	459	306	297	297
query70	1084	1059	1066	1059
query71	427	282	291	282
query72	7109	2659	2548	2548
query73	772	319	326	319
query74	5938	5705	5562	5562
query75	3372	2735	2748	2735
query76	2549	1217	1298	1217
query77	448	305	315	305
query78	9374	8934	8826	8826
query79	2089	529	531	529
query80	1470	504	509	504
query81	575	233	229	229
query82	1029	133	129	129
query83	259	169	170	169
query84	278	82	79	79
query85	1287	319	366	319
query86	391	306	272	272
query87	3278	3090	3074	3074
query88	2902	2403	2410	2403
query89	390	295	285	285
query90	1807	189	187	187
query91	123	99	102	99
query92	64	51	51	51
query93	1563	631	627	627
query94	918	289	297	289
query95	384	263	266	263
query96	600	281	278	278
query97	3228	3117	3069	3069
query98	226	204	205	204
query99	1644	1276	1283	1276
Total cold run time: 263003 ms
Total hot run time: 168152 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.19 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4d673208635540e2790fc74e002950977ec6baa8, data reload: false

query1	0.04	0.04	0.05
query2	0.07	0.04	0.04
query3	0.23	0.05	0.04
query4	1.69	0.07	0.07
query5	0.49	0.48	0.48
query6	1.16	0.72	0.71
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.58	0.51	0.51
query10	0.57	0.57	0.57
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.61	0.60	0.60
query14	0.77	0.78	0.79
query15	0.90	0.87	0.86
query16	0.36	0.35	0.35
query17	1.05	0.99	0.97
query18	0.22	0.22	0.21
query19	1.82	1.77	1.77
query20	0.01	0.01	0.01
query21	15.38	0.75	0.67
query22	4.01	7.31	1.57
query23	18.04	1.31	1.22
query24	2.14	0.23	0.22
query25	0.18	0.09	0.08
query26	0.32	0.23	0.21
query27	0.47	0.23	0.24
query28	13.16	1.00	0.97
query29	12.52	3.24	3.25
query30	0.25	0.06	0.05
query31	2.90	0.40	0.41
query32	3.21	0.49	0.48
query33	2.93	2.99	2.98
query34	15.47	4.24	4.25
query35	4.26	4.29	4.29
query36	0.68	0.47	0.49
query37	0.19	0.16	0.15
query38	0.16	0.14	0.16
query39	0.04	0.03	0.04
query40	0.15	0.14	0.13
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 107.59 s
Total hot run time: 30.19 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 7, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 7, 2024

PR approved by at least one committer and no changes requested.

@gavinchou gavinchou merged commit d33f580 into apache:master Aug 7, 2024
28 of 30 checks passed
dataroaring added a commit that referenced this pull request Aug 7, 2024
@dataroaring
[fix](cloud-balance) use right file size instead of block size (#38939)
51ca8ac 
introduced by #34191
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Aug 14, 2024
@dataroaring @wyxxxcat
[fix](cloud-balance) use right file size instead of block size (apach…
459170f 
…e#38939)

introduced by apache#34191
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ByteYue ByteYue ByteYue approved these changes

@gavinchou gavinchou gavinchou approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.1-merged doing meta-change reviewed usercase Important user case type label
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@dataroaring @doris-robot @gavinchou @ByteYue