Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore](rowset writer) print rowset rows number when meet too many segments (#39091) #39530

Merged
merged 1 commit into from
Aug 19, 2024

Conversation

sollhui
Copy link
Contributor

@sollhui sollhui commented Aug 18, 2024

pick (#39091)

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@sollhui
Copy link
Contributor Author

sollhui commented Aug 18, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50064 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6124c8b06393e762956bcbb9d5bde32c32a68c49, data reload: false

------ Round 1 ----------------------------------
q1	17913	4460	4385	4385
q2	2084	199	187	187
q3	10368	1900	1978	1900
q4	10333	1279	1359	1279
q5	8420	3871	3940	3871
q6	275	143	143	143
q7	2086	1657	1635	1635
q8	9547	2784	2762	2762
q9	13835	10134	10138	10134
q10	8684	3597	3512	3512
q11	448	280	279	279
q12	505	336	351	336
q13	18375	3978	4038	3978
q14	373	358	360	358
q15	561	504	512	504
q16	711	589	609	589
q17	1136	985	969	969
q18	7310	6792	6892	6792
q19	1744	1626	1539	1539
q20	548	348	345	345
q21	4398	4127	4138	4127
q22	542	440	448	440
Total cold run time: 120196 ms
Total hot run time: 50064 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4366	4324	4357	4324
q2	358	264	278	264
q3	4184	4151	4187	4151
q4	2762	2761	2751	2751
q5	7131	7197	7182	7182
q6	258	134	139	134
q7	3256	2872	2880	2872
q8	4365	4466	4479	4466
q9	16998	16849	16838	16838
q10	4352	4410	4343	4343
q11	760	676	728	676
q12	1042	873	880	873
q13	6970	3785	3754	3754
q14	475	450	458	450
q15	542	504	519	504
q16	779	698	707	698
q17	3819	3840	3788	3788
q18	8785	8800	8717	8717
q19	1783	1754	1733	1733
q20	2394	2138	2135	2135
q21	8439	8429	8507	8429
q22	1045	974	984	974
Total cold run time: 84863 ms
Total hot run time: 80056 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 208986 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6124c8b06393e762956bcbb9d5bde32c32a68c49, data reload: false

query1	989	421	411	411
query2	6779	2285	2188	2188
query3	6949	231	225	225
query4	21422	18239	18078	18078
query5	19926	6758	6722	6722
query6	331	262	269	262
query7	4191	339	343	339
query8	496	459	479	459
query9	3234	2799	2743	2743
query10	491	371	378	371
query11	11441	10707	10746	10707
query12	151	109	110	109
query13	5690	733	732	732
query14	18123	13628	13554	13554
query15	395	260	246	246
query16	6510	327	305	305
query17	1623	1450	955	955
query18	2333	483	470	470
query19	267	183	195	183
query20	109	113	114	113
query21	241	151	145	145
query22	5273	5085	5059	5059
query23	32603	32242	31850	31850
query24	7012	6547	6699	6547
query25	573	464	481	464
query26	560	191	194	191
query27	1847	339	326	326
query28	6052	2429	2373	2373
query29	2901	2740	2787	2740
query30	285	215	207	207
query31	1032	861	834	834
query32	101	92	92	92
query33	481	321	349	321
query34	903	519	533	519
query35	1176	1019	991	991
query36	1310	1387	1099	1099
query37	118	92	91	91
query38	3116	2969	2973	2969
query39	1481	1443	1430	1430
query40	258	146	149	146
query41	158	160	156	156
query42	109	112	116	112
query43	723	670	666	666
query44	1180	789	784	784
query45	285	266	269	266
query46	1275	1003	1021	1003
query47	1947	1794	1920	1794
query48	1031	754	740	740
query49	852	586	594	586
query50	947	732	660	660
query51	4902	4729	4742	4729
query52	117	123	102	102
query53	509	382	375	375
query54	2725	2497	2521	2497
query55	108	99	104	99
query56	319	299	278	278
query57	1234	1238	1387	1238
query58	328	302	309	302
query59	3570	3472	3389	3389
query60	302	281	288	281
query61	156	157	154	154
query62	887	605	574	574
query63	540	401	405	401
query64	2465	1663	1642	1642
query65	3697	3616	3584	3584
query66	1239	811	810	810
query67	15842	15543	15205	15205
query68	8562	699	693	693
query69	633	407	428	407
query70	1603	1500	1352	1352
query71	454	357	364	357
query72	6649	3543	3544	3543
query73	790	363	365	363
query74	6359	5904	5931	5904
query75	4609	3791	3787	3787
query76	4758	1170	1215	1170
query77	744	421	412	412
query78	12415	12148	13345	12148
query79	10695	695	693	693
query80	1937	561	542	542
query81	555	279	275	275
query82	1151	125	130	125
query83	248	218	215	215
query84	290	97	97	97
query85	1372	410	406	406
query86	355	323	341	323
query87	3243	3027	3089	3027
query88	5162	2539	2532	2532
query89	483	340	363	340
query90	1710	254	258	254
query91	196	165	158	158
query92	89	83	84	83
query93	5340	593	610	593
query94	668	243	242	242
query95	1159	1096	1109	1096
query96	653	351	337	337
query97	6499	6431	6401	6401
query98	225	207	199	199
query99	2729	947	919	919
Total cold run time: 318555 ms
Total hot run time: 208986 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.57 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6124c8b06393e762956bcbb9d5bde32c32a68c49, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.03	0.02
query3	0.25	0.06	0.05
query4	1.78	0.08	0.06
query5	0.54	0.53	0.53
query6	1.26	0.61	0.62
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.52	0.48	0.48
query10	0.54	0.53	0.56
query11	0.13	0.10	0.09
query12	0.14	0.10	0.10
query13	0.63	0.61	0.61
query14	0.79	0.80	0.81
query15	0.78	0.77	0.77
query16	0.39	0.38	0.37
query17	1.00	1.03	1.03
query18	0.23	0.25	0.28
query19	1.92	1.86	1.83
query20	0.02	0.01	0.02
query21	15.44	0.57	0.57
query22	1.79	2.22	1.64
query23	17.38	1.13	1.05
query24	3.65	1.22	1.75
query25	0.35	0.10	0.06
query26	0.56	0.15	0.16
query27	0.05	0.05	0.04
query28	8.43	0.75	0.77
query29	12.64	2.21	2.25
query30	0.74	0.71	0.75
query31	2.83	0.39	0.41
query32	3.33	0.50	0.51
query33	3.06	3.04	3.10
query34	15.28	4.78	4.79
query35	4.87	4.86	4.85
query36	1.07	1.03	1.03
query37	0.08	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.03	0.03
query40	0.19	0.17	0.17
query41	0.08	0.03	0.04
query42	0.04	0.03	0.04
query43	0.04	0.03	0.03
Total cold run time: 103.06 s
Total hot run time: 31.57 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 6124c8b06393e762956bcbb9d5bde32c32a68c49 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.71% (8103/21488)
Line Coverage: 29.35% (66396/226212)
Region Coverage: 28.84% (34245/118732)
Branch Coverage: 24.73% (17604/71182)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6124c8b06393e762956bcbb9d5bde32c32a68c49_6124c8b06393e762956bcbb9d5bde32c32a68c49/report/index.html

@sollhui
Copy link
Contributor Author

sollhui commented Aug 18, 2024

run feut

@dataroaring dataroaring merged commit b8f23cb into apache:branch-2.0 Aug 19, 2024
21 of 25 checks passed
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants