Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](dynamic partition) fix create dynamic partition with overlap not throw exception #37924 #37966

Merged
merged 6 commits into from
Jul 17, 2024
Merged

[fix](dynamic partition) fix create dynamic partition with overlap not throw exception #37924 #37966

merged 6 commits into from
Jul 17, 2024

Conversation

yujun777
Copy link
Collaborator

cherry-pick: #37924

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@yujun777
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50014 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 56b675d0e306ead16128d73b2f0cf27ebd8329c2, data reload: false

------ Round 1 ----------------------------------
q1	17867	4398	4325	4325
q2	2058	154	152	152
q3	10266	1914	1901	1901
q4	10326	1266	1316	1266
q5	8882	3932	3950	3932
q6	254	142	128	128
q7	2036	1577	1600	1577
q8	9289	2733	2719	2719
q9	11085	10622	10437	10437
q10	8596	3502	3512	3502
q11	421	235	255	235
q12	469	298	303	298
q13	18343	3954	4058	3954
q14	354	323	330	323
q15	514	465	467	465
q16	688	577	573	573
q17	1132	981	990	981
q18	7428	6833	7024	6833
q19	1793	1644	1588	1588
q20	507	314	311	311
q21	4478	4100	4079	4079
q22	531	435	449	435
Total cold run time: 117317 ms
Total hot run time: 50014 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4335	4312	4369	4312
q2	327	232	231	231
q3	4157	4164	4127	4127
q4	2779	2784	2781	2781
q5	7285	7223	7261	7223
q6	233	122	120	120
q7	3307	2873	2863	2863
q8	4393	4492	4507	4492
q9	17484	17113	17165	17113
q10	4256	4332	4241	4241
q11	780	674	654	654
q12	1039	876	885	876
q13	6737	3775	3772	3772
q14	443	425	439	425
q15	495	472	458	458
q16	727	690	683	683
q17	3849	3825	3861	3825
q18	8904	8887	8895	8887
q19	1732	1740	1711	1711
q20	2387	2130	2097	2097
q21	8502	8491	8597	8491
q22	1115	1002	1005	1002
Total cold run time: 85266 ms
Total hot run time: 80384 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203943 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 56b675d0e306ead16128d73b2f0cf27ebd8329c2, data reload: false

query1	946	419	375	375
query2	6551	2801	2626	2626
query3	7568	217	217	217
query4	21715	18393	18199	18199
query5	21992	6597	6480	6480
query6	422	225	232	225
query7	5093	311	328	311
query8	425	413	398	398
query9	3087	2645	2627	2627
query10	431	303	288	288
query11	11317	10689	10669	10669
query12	135	74	78	74
query13	5613	717	699	699
query14	19754	13233	13758	13233
query15	365	264	256	256
query16	6436	286	266	266
query17	1568	1460	890	890
query18	2299	412	420	412
query19	225	156	156	156
query20	84	76	89	76
query21	195	110	102	102
query22	5174	4992	4939	4939
query23	32597	32041	32044	32041
query24	6783	6579	6517	6517
query25	529	442	424	424
query26	512	168	164	164
query27	1679	306	309	306
query28	6155	2354	2316	2316
query29	3009	2837	2709	2709
query30	247	165	167	165
query31	911	758	739	739
query32	72	66	64	64
query33	402	272	256	256
query34	845	492	485	485
query35	1159	947	969	947
query36	1505	1108	1302	1108
query37	91	62	64	62
query38	3078	2944	2968	2944
query39	1395	1335	1327	1327
query40	203	101	101	101
query41	49	44	47	44
query42	85	93	83	83
query43	727	676	648	648
query44	1144	721	728	721
query45	250	245	235	235
query46	1234	969	977	969
query47	1878	1832	1666	1666
query48	1025	731	728	728
query49	634	379	384	379
query50	858	603	625	603
query51	4799	4695	4635	4635
query52	94	83	90	83
query53	450	325	320	320
query54	2663	2444	2433	2433
query55	94	83	90	83
query56	242	221	215	215
query57	1122	1088	1131	1088
query58	228	223	215	215
query59	4663	4042	3913	3913
query60	212	238	201	201
query61	99	100	101	100
query62	844	493	451	451
query63	494	345	345	345
query64	2432	1560	1523	1523
query65	3627	3575	3560	3560
query66	801	388	382	382
query67	16172	15219	14921	14921
query68	8633	690	669	669
query69	597	361	351	351
query70	1657	1415	1353	1353
query71	408	309	328	309
query72	6527	3487	3511	3487
query73	731	316	324	316
query74	6358	5831	5970	5831
query75	5213	3736	3674	3674
query76	5160	1153	1202	1153
query77	906	258	265	258
query78	12676	11750	11942	11750
query79	7594	647	650	647
query80	1059	423	421	421
query81	489	242	245	242
query82	1618	108	105	105
query83	197	139	140	139
query84	258	74	74	74
query85	951	355	345	345
query86	342	321	302	302
query87	3202	3028	3049	3028
query88	4971	2330	2328	2328
query89	373	302	300	300
query90	1974	212	218	212
query91	179	144	146	144
query92	59	60	61	60
query93	3546	603	575	575
query94	741	206	217	206
query95	1127	1076	1083	1076
query96	659	326	326	326
query97	6575	6519	6536	6519
query98	197	180	175	175
query99	2993	831	887	831
Total cold run time: 317457 ms
Total hot run time: 203943 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.71 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 56b675d0e306ead16128d73b2f0cf27ebd8329c2, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.02	0.02
query3	0.24	0.05	0.05
query4	1.78	0.06	0.08
query5	0.54	0.53	0.53
query6	1.27	0.62	0.62
query7	0.01	0.00	0.00
query8	0.03	0.02	0.02
query9	0.54	0.49	0.49
query10	0.54	0.53	0.55
query11	0.12	0.09	0.09
query12	0.12	0.09	0.09
query13	0.63	0.62	0.61
query14	0.79	0.79	0.79
query15	0.79	0.78	0.77
query16	0.37	0.36	0.36
query17	1.03	0.99	1.01
query18	0.23	0.24	0.26
query19	1.89	1.86	1.86
query20	0.02	0.01	0.01
query21	15.48	0.58	0.55
query22	2.07	2.11	1.43
query23	17.05	1.10	0.97
query24	6.10	1.04	1.17
query25	0.43	0.06	0.05
query26	0.72	0.16	0.17
query27	0.05	0.03	0.03
query28	6.82	0.80	0.73
query29	12.73	2.34	2.23
query30	0.62	0.50	0.52
query31	2.81	0.40	0.38
query32	3.34	0.50	0.50
query33	3.07	3.06	3.07
query34	15.26	4.84	4.80
query35	4.86	4.86	4.86
query36	1.08	1.02	1.03
query37	0.06	0.04	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.15	0.14
query41	0.07	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 103.9 s
Total hot run time: 30.71 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 56b675d0e306ead16128d73b2f0cf27ebd8329c2 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       22.0 seconds inserted 10000000 Rows, about 454K ops/s

@dataroaring dataroaring merged commit 79b4154 into apache:branch-2.0 Jul 17, 2024
24 of 25 checks passed
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@yujun777
[fix](dynamic partition) fix create dynamic partition with overlap no…
c74a1aa 
…t throw exception apache#37924 (apache#37966)

cherry-pick: apache#37924
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
kind/test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@yujun777 @doris-robot @dataroaring