Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](be) Check MD5 when downloading snapshot in http #36726

Merged
merged 1 commit into from
Jun 24, 2024
Merged

[fix](be) Check MD5 when downloading snapshot in http #36726

merged 1 commit into from
Jun 24, 2024

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented Jun 24, 2024

No description provided.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@w41ter
Copy link
Contributor Author

w41ter commented Jun 24, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39771 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6453dfa74c22541ecd0c707360c1dca50faab94b, data reload: false

------ Round 1 ----------------------------------
q1	17608	4368	4310	4310
q2	2014	197	198	197
q3	10441	1085	1144	1085
q4	10194	766	774	766
q5	7456	2710	2633	2633
q6	221	135	137	135
q7	939	595	601	595
q8	9241	2080	2092	2080
q9	8965	6558	6515	6515
q10	9008	3721	3793	3721
q11	456	257	230	230
q12	459	224	227	224
q13	18890	2995	3006	2995
q14	267	220	215	215
q15	509	466	481	466
q16	529	377	396	377
q17	983	657	752	657
q18	8105	7613	7355	7355
q19	5536	1452	1374	1374
q20	657	323	325	323
q21	4999	3853	3180	3180
q22	395	340	338	338
Total cold run time: 117872 ms
Total hot run time: 39771 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4395	4247	4283	4247
q2	375	278	258	258
q3	3001	2926	2896	2896
q4	2024	1732	1740	1732
q5	5592	5554	5489	5489
q6	230	126	128	126
q7	2280	1883	1882	1882
q8	3274	3475	3459	3459
q9	8738	8736	8805	8736
q10	4188	3732	3769	3732
q11	599	508	511	508
q12	799	638	655	638
q13	15996	3173	3162	3162
q14	324	281	276	276
q15	525	497	478	478
q16	476	451	436	436
q17	1877	1544	1502	1502
q18	8089	8185	7769	7769
q19	1862	1640	1592	1592
q20	3033	1896	1895	1895
q21	5104	4948	5001	4948
q22	604	567	539	539
Total cold run time: 73385 ms
Total hot run time: 56300 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174842 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6453dfa74c22541ecd0c707360c1dca50faab94b, data reload: false

query1	925	388	371	371
query2	6459	2357	2317	2317
query3	6629	220	227	220
query4	18764	17436	17268	17268
query5	3698	480	471	471
query6	235	173	170	170
query7	4596	297	289	289
query8	315	287	292	287
query9	8542	2366	2350	2350
query10	570	297	294	294
query11	10548	9987	9988	9987
query12	117	89	91	89
query13	1653	384	375	375
query14	10345	7128	8006	7128
query15	234	192	188	188
query16	7867	297	288	288
query17	1856	575	543	543
query18	1926	292	279	279
query19	204	158	164	158
query20	90	88	83	83
query21	213	127	132	127
query22	4271	4183	4122	4122
query23	34028	33787	33758	33758
query24	10864	2960	2940	2940
query25	622	419	405	405
query26	713	164	163	163
query27	2261	322	332	322
query28	5996	2155	2129	2129
query29	909	681	652	652
query30	269	162	166	162
query31	1001	771	776	771
query32	102	57	56	56
query33	680	301	315	301
query34	894	492	489	489
query35	769	667	635	635
query36	1145	982	972	972
query37	148	79	79	79
query38	2958	2821	2838	2821
query39	932	849	840	840
query40	221	147	143	143
query41	58	56	58	56
query42	111	106	101	101
query43	613	565	552	552
query44	1089	761	761	761
query45	201	163	169	163
query46	1089	719	708	708
query47	1859	1790	1757	1757
query48	388	304	308	304
query49	865	429	438	429
query50	781	407	398	398
query51	6738	6777	6690	6690
query52	103	92	96	92
query53	366	297	300	297
query54	881	477	459	459
query55	83	75	77	75
query56	309	280	287	280
query57	1155	1062	1018	1018
query58	347	252	262	252
query59	3338	3041	3086	3041
query60	305	276	278	276
query61	97	110	93	93
query62	614	435	445	435
query63	325	287	287	287
query64	8588	2255	1784	1784
query65	3194	3123	3113	3113
query66	754	334	331	331
query67	15811	15074	15019	15019
query68	8535	553	568	553
query69	746	488	409	409
query70	1185	1129	1015	1015
query71	541	282	276	276
query72	8695	5372	5753	5372
query73	1827	328	331	328
query74	5921	5520	5506	5506
query75	5194	2682	2690	2682
query76	4504	942	962	942
query77	807	302	310	302
query78	10502	9914	10172	9914
query79	7161	506	521	506
query80	871	467	460	460
query81	546	220	217	217
query82	270	103	107	103
query83	341	172	174	172
query84	277	85	89	85
query85	869	285	272	272
query86	340	324	314	314
query87	3277	3092	3141	3092
query88	4015	2459	2442	2442
query89	484	379	378	378
query90	1907	190	183	183
query91	129	103	102	102
query92	56	48	50	48
query93	4585	506	510	506
query94	1279	195	190	190
query95	399	315	320	315
query96	605	272	267	267
query97	3257	3039	3057	3039
query98	223	199	192	192
query99	1072	849	847	847
Total cold run time: 285051 ms
Total hot run time: 174842 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.32 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6453dfa74c22541ecd0c707360c1dca50faab94b, data reload: false

query1	0.04	0.04	0.04
query2	0.09	0.04	0.04
query3	0.22	0.05	0.06
query4	1.67	0.08	0.08
query5	0.49	0.47	0.48
query6	1.13	0.72	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.53	0.50	0.49
query10	0.55	0.54	0.55
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.59	0.60	0.58
query14	0.76	0.77	0.77
query15	0.83	0.81	0.81
query16	0.38	0.36	0.37
query17	0.98	0.99	1.06
query18	0.21	0.27	0.24
query19	1.80	1.80	1.69
query20	0.01	0.01	0.01
query21	15.42	0.65	0.66
query22	4.44	7.18	2.42
query23	18.24	1.42	1.33
query24	2.08	0.24	0.22
query25	0.16	0.08	0.08
query26	0.27	0.18	0.17
query27	0.08	0.08	0.09
query28	13.22	1.04	1.02
query29	12.59	3.34	3.34
query30	0.26	0.06	0.06
query31	2.86	0.38	0.40
query32	3.27	0.47	0.48
query33	2.90	2.87	2.95
query34	17.02	4.45	4.47
query35	4.51	4.54	4.53
query36	0.65	0.48	0.48
query37	0.19	0.15	0.16
query38	0.15	0.15	0.14
query39	0.05	0.03	0.04
query40	0.16	0.14	0.15
query41	0.08	0.04	0.04
query42	0.05	0.06	0.06
query43	0.04	0.05	0.05
Total cold run time: 109.34 s
Total hot run time: 31.32 s

dataroaring
dataroaring approved these changes Jun 24, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 24, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit 89e9001 into apache:master Jun 24, 2024
26 of 30 checks passed
@w41ter w41ter deleted the check_md5_before_apply_snapshot branch June 24, 2024 13:00
dataroaring pushed a commit that referenced this pull request Jun 26, 2024
@w41ter @dataroaring
[fix](be) Check MD5 when downloading snapshot in http (#36726)
ca7bd33
w41ter added a commit to w41ter/incubator-doris that referenced this pull request Jul 8, 2024
@w41ter
[fix](be) Check MD5 when downloading snapshot in http
0757402 
Cherry-pick apache#36726
@w41ter w41ter mentioned this pull request Jul 8, 2024
Merged
w41ter added a commit to w41ter/incubator-doris that referenced this pull request Jul 8, 2024
@w41ter
[fix](be) Check MD5 when downloading snapshot in http
af18208 
Cherry-pick apache#36726
@w41ter w41ter mentioned this pull request Jul 8, 2024
Merged
w41ter added a commit to w41ter/incubator-doris that referenced this pull request Jul 8, 2024
@w41ter
[fix](be) Check MD5 when downloading snapshot in http
439d792 
Cherry-pick apache#36726
dataroaring pushed a commit that referenced this pull request Jul 8, 2024
@w41ter
[chore](be) Acquire and check MD5 digest of the file to download (#37418
5280e27 
)

Cherry-pick #35807, #36621, #36726
@w41ter w41ter removed the dev/2.1.x label Jul 8, 2024
w41ter added a commit to w41ter/incubator-doris that referenced this pull request Jul 26, 2024
@w41ter
[fix](be) Check MD5 when downloading snapshot in http
6a39e8d 
Cherry-pick apache#36726
w41ter added a commit that referenced this pull request Jul 26, 2024
@w41ter
[chore](be) Acquire and check MD5 digest of the file to download (#37420
7fc4ddc 
)

Cherry-pick #35807, #36621, #36726
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@w41ter @Baymine
[chore](be) Acquire and check MD5 digest of the file to download (apa…
8939fe7 
…che#37420)

Cherry-pick apache#35807, apache#36621, apache#36726
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

@gavinchou gavinchou gavinchou approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.14-merged dev/2.1.5-merged dev/3.0.0-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@w41ter @doris-robot @gavinchou @dataroaring