Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix](smooth-upgrade) Fix incompatibility when upgrade from 2.0 to 2.1 #32162

Merged
merged 1 commit into from
Mar 13, 2024
Merged

[Fix](smooth-upgrade) Fix incompatibility when upgrade from 2.0 to 2.1 #32162

merged 1 commit into from
Mar 13, 2024

Conversation

zclllyybb
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@zclllyybb
Copy link
Contributor Author

run buildall

HappenLee
HappenLee previously approved these changes Mar 13, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 13, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 36450 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 95d7643c082fff1fadd7328714ddaca0c4878c44, data reload: false

------ Round 1 ----------------------------------
q1	17814	4790	4115	4115
q2	2023	154	143	143
q3	10639	900	886	886
q4	4647	701	682	682
q5	7583	2796	2772	2772
q6	184	126	124	124
q7	1171	832	799	799
q8	9488	2029	2052	2029
q9	7361	6496	6436	6436
q10	8136	2468	2467	2467
q11	425	211	216	211
q12	789	275	293	275
q13	17958	3026	2974	2974
q14	275	245	248	245
q15	491	456	454	454
q16	474	401	412	401
q17	950	550	566	550
q18	6657	5841	5788	5788
q19	1581	1333	1418	1333
q20	566	279	270	270
q21	7314	3204	3202	3202
q22	807	326	294	294
Total cold run time: 107333 ms
Total hot run time: 36450 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4135	4095	4124	4095
q2	324	231	219	219
q3	3002	2824	2836	2824
q4	1876	1586	1517	1517
q5	5219	5246	5227	5227
q6	210	114	117	114
q7	2264	1840	1819	1819
q8	3167	3274	3276	3274
q9	8515	8523	8585	8523
q10	6116	3648	3647	3647
q11	527	452	432	432
q12	718	533	560	533
q13	12642	2840	2855	2840
q14	280	242	254	242
q15	482	446	450	446
q16	448	395	407	395
q17	1725	1476	1452	1452
q18	7693	7231	6990	6990
q19	4163	1564	1540	1540
q20	1906	1680	1720	1680
q21	4792	4791	4855	4791
q22	503	438	474	438
Total cold run time: 70707 ms
Total hot run time: 53038 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186687 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 95d7643c082fff1fadd7328714ddaca0c4878c44, data reload: false

query1	924	350	348	348
query2	7487	2062	1908	1908
query3	6706	218	224	218
query4	31001	20628	20986	20628
query5	4429	397	410	397
query6	262	181	195	181
query7	4628	295	299	295
query8	239	179	175	175
query9	9902	2286	2249	2249
query10	449	228	242	228
query11	17101	14113	14145	14113
query12	133	90	80	80
query13	1616	410	425	410
query14	11214	10445	10412	10412
query15	249	210	203	203
query16	7933	263	263	263
query17	1944	562	543	543
query18	2015	284	274	274
query19	219	166	154	154
query20	93	92	86	86
query21	198	130	123	123
query22	4584	4405	4411	4405
query23	31873	31062	31154	31062
query24	11260	2822	2836	2822
query25	660	382	375	375
query26	1262	158	155	155
query27	2758	354	343	343
query28	7534	1899	1880	1880
query29	889	651	637	637
query30	320	151	150	150
query31	996	727	722	722
query32	94	61	56	56
query33	775	266	249	249
query34	1098	472	489	472
query35	838	606	600	600
query36	992	864	858	858
query37	128	79	77	77
query38	3305	3141	3103	3103
query39	1406	1355	1378	1355
query40	209	116	111	111
query41	49	48	51	48
query42	97	98	92	92
query43	505	467	481	467
query44	1295	735	722	722
query45	276	253	266	253
query46	1073	677	679	677
query47	1668	1628	1582	1582
query48	437	356	351	351
query49	1124	336	342	336
query50	756	376	382	376
query51	6819	6715	6786	6715
query52	103	93	93	93
query53	344	273	278	273
query54	294	237	277	237
query55	81	75	77	75
query56	249	222	227	222
query57	1068	997	1010	997
query58	219	202	201	201
query59	2733	2522	2742	2522
query60	267	261	238	238
query61	93	90	91	90
query62	576	383	396	383
query63	296	323	280	280
query64	5555	3816	3750	3750
query65	3137	3027	3012	3012
query66	830	321	328	321
query67	14896	14390	14246	14246
query68	7164	523	549	523
query69	621	370	373	370
query70	1184	1139	1205	1139
query71	498	276	264	264
query72	7053	2655	2520	2520
query73	719	324	322	322
query74	7893	6575	6584	6575
query75	9670	8178	7952	7952
query76	4726	949	926	926
query77	674	275	258	258
query78	10230	9827	9653	9653
query79	8392	530	527	527
query80	1485	419	430	419
query81	540	212	209	209
query82	870	197	231	197
query83	214	154	148	148
query84	286	79	84	79
query85	1550	376	367	367
query86	471	308	286	286
query87	3449	3197	3199	3197
query88	4835	2323	2293	2293
query89	525	364	376	364
query90	1946	178	172	172
query91	158	127	146	127
query92	57	46	46	46
query93	6968	502	487	487
query94	1145	172	176	172
query95	425	328	331	328
query96	601	275	276	275
query97	4388	4249	4296	4249
query98	230	213	212	212
query99	1100	757	752	752
Total cold run time: 314630 ms
Total hot run time: 186687 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.67 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 95d7643c082fff1fadd7328714ddaca0c4878c44, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.02	0.02
query3	0.23	0.06	0.06
query4	1.66	0.10	0.10
query5	0.50	0.51	0.51
query6	1.25	0.64	0.62
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.52	0.50
query10	0.56	0.54	0.53
query11	0.12	0.09	0.09
query12	0.12	0.09	0.09
query13	0.59	0.56	0.55
query14	0.75	0.77	0.76
query15	0.80	0.78	0.76
query16	0.38	0.36	0.40
query17	0.95	1.03	1.00
query18	0.20	0.27	0.21
query19	1.74	1.68	1.63
query20	0.02	0.01	0.01
query21	15.41	0.56	0.57
query22	2.44	1.86	1.64
query23	17.23	0.96	0.68
query24	3.29	1.61	0.62
query25	0.19	0.15	0.19
query26	0.46	0.11	0.11
query27	0.04	0.02	0.03
query28	10.81	0.88	0.83
query29	12.53	3.22	3.22
query30	0.67	0.66	0.60
query31	2.81	0.34	0.33
query32	3.41	0.45	0.43
query33	2.76	2.77	2.83
query34	16.32	4.27	4.28
query35	4.29	4.28	4.29
query36	1.08	1.01	1.01
query37	0.06	0.05	0.05
query38	0.04	0.03	0.03
query39	0.03	0.01	0.01
query40	0.17	0.13	0.13
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.02	0.02
Total cold run time: 104.78 s
Total hot run time: 29.67 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 95d7643c082fff1fadd7328714ddaca0c4878c44 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       17.3 seconds inserted 10000000 Rows, about 578K ops/s

@zclllyybb
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Mar 13, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 36472 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3184fd1cc216faa40e037eba254325697d4e32a6, data reload: false

------ Round 1 ----------------------------------
q1	17617	4711	4120	4120
q2	2018	145	145	145
q3	10580	889	903	889
q4	4652	690	731	690
q5	7577	2741	2810	2741
q6	183	126	123	123
q7	1177	807	802	802
q8	9475	2058	2011	2011
q9	7369	6512	6415	6415
q10	8183	2494	2497	2494
q11	431	217	219	217
q12	790	307	298	298
q13	17947	2995	2960	2960
q14	277	251	250	250
q15	502	467	455	455
q16	488	389	395	389
q17	947	560	533	533
q18	6681	5763	5925	5763
q19	1566	1406	1468	1406
q20	536	279	281	279
q21	7427	3184	3248	3184
q22	788	321	308	308
Total cold run time: 107211 ms
Total hot run time: 36472 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4114	4056	4126	4056
q2	320	229	232	229
q3	2993	2812	2865	2812
q4	1841	1559	1568	1559
q5	5193	5232	5214	5214
q6	192	114	117	114
q7	2220	1847	1820	1820
q8	3128	3286	3252	3252
q9	8530	8579	8540	8540
q10	6134	3697	3660	3660
q11	538	446	428	428
q12	721	553	528	528
q13	11044	2827	2854	2827
q14	270	243	245	243
q15	471	442	446	442
q16	434	405	396	396
q17	1744	1458	1475	1458
q18	7700	7283	7093	7093
q19	4557	1500	1513	1500
q20	1878	1717	1682	1682
q21	4896	4690	4810	4690
q22	554	479	459	459
Total cold run time: 69472 ms
Total hot run time: 53002 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185386 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3184fd1cc216faa40e037eba254325697d4e32a6, data reload: false

query1	926	355	338	338
query2	7637	1976	1984	1976
query3	6703	219	214	214
query4	30856	20436	20502	20436
query5	4407	402	391	391
query6	262	182	169	169
query7	4630	299	287	287
query8	227	176	171	171
query9	9906	2283	2246	2246
query10	452	241	246	241
query11	17094	13980	13973	13973
query12	133	85	79	79
query13	1627	407	439	407
query14	11034	10075	10596	10075
query15	260	206	185	185
query16	8050	263	259	259
query17	1951	573	536	536
query18	2067	286	273	273
query19	217	164	152	152
query20	92	82	111	82
query21	198	131	122	122
query22	4679	4483	4382	4382
query23	31749	31094	30887	30887
query24	10864	2857	2779	2779
query25	620	378	385	378
query26	1241	166	153	153
query27	2530	353	351	351
query28	7428	1879	1880	1879
query29	877	644	627	627
query30	307	142	149	142
query31	963	712	720	712
query32	89	62	55	55
query33	762	263	255	255
query34	1081	465	484	465
query35	821	601	606	601
query36	1003	876	854	854
query37	136	79	78	78
query38	3296	3172	3136	3136
query39	1455	1366	1371	1366
query40	218	151	107	107
query41	50	46	48	46
query42	105	94	95	94
query43	491	460	454	454
query44	1324	729	707	707
query45	265	249	249	249
query46	1072	679	686	679
query47	1662	1597	1574	1574
query48	432	361	345	345
query49	1139	345	329	329
query50	760	362	372	362
query51	6821	6711	6605	6605
query52	101	94	84	84
query53	342	279	273	273
query54	313	236	236	236
query55	90	99	82	82
query56	268	229	228	228
query57	1075	1001	1004	1001
query58	225	207	205	205
query59	2667	2635	2732	2635
query60	267	236	249	236
query61	124	114	111	111
query62	595	437	418	418
query63	305	274	272	272
query64	5535	3801	3740	3740
query65	3067	2991	2980	2980
query66	839	317	320	317
query67	14730	14767	14266	14266
query68	9096	534	539	534
query69	635	381	364	364
query70	1231	1183	1093	1093
query71	529	272	272	272
query72	6856	2686	2483	2483
query73	1447	315	309	309
query74	6804	6566	6583	6566
query75	9671	8281	7900	7900
query76	5433	854	1015	854
query77	665	264	259	259
query78	10236	9619	9620	9619
query79	11107	540	534	534
query80	1998	387	375	375
query81	493	205	210	205
query82	350	196	192	192
query83	238	146	144	144
query84	292	75	72	72
query85	1230	327	325	325
query86	355	309	300	300
query87	3418	3246	3206	3206
query88	5266	2275	2272	2272
query89	470	369	361	361
query90	2052	181	180	180
query91	151	128	128	128
query92	61	44	45	44
query93	5957	497	480	480
query94	1352	170	177	170
query95	420	327	321	321
query96	598	275	268	268
query97	4356	4262	4291	4262
query98	225	218	204	204
query99	968	753	743	743
Total cold run time: 317472 ms
Total hot run time: 185386 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.02 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3184fd1cc216faa40e037eba254325697d4e32a6, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.03	0.03
query3	0.22	0.06	0.06
query4	1.68	0.10	0.10
query5	0.48	0.50	0.51
query6	1.23	0.62	0.63
query7	0.02	0.02	0.01
query8	0.04	0.03	0.02
query9	0.56	0.49	0.49
query10	0.54	0.56	0.55
query11	0.12	0.09	0.09
query12	0.11	0.09	0.10
query13	0.57	0.56	0.57
query14	0.76	0.75	0.76
query15	0.80	0.77	0.77
query16	0.38	0.39	0.39
query17	0.96	0.96	1.04
query18	0.21	0.25	0.23
query19	1.75	1.66	1.69
query20	0.01	0.01	0.01
query21	15.41	0.59	0.57
query22	2.58	2.10	1.92
query23	17.08	0.93	0.74
query24	3.23	1.95	0.62
query25	0.24	0.13	0.09
query26	0.66	0.13	0.11
query27	0.04	0.02	0.04
query28	11.36	0.85	0.85
query29	12.54	3.14	3.20
query30	0.64	0.55	0.61
query31	2.80	0.34	0.33
query32	3.40	0.45	0.44
query33	2.77	2.84	2.80
query34	16.06	4.31	4.34
query35	4.41	4.40	4.34
query36	1.08	1.00	0.99
query37	0.07	0.05	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.14	0.14
query41	0.07	0.02	0.01
query42	0.03	0.02	0.01
query43	0.03	0.03	0.02
Total cold run time: 105.27 s
Total hot run time: 30.02 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 3184fd1cc216faa40e037eba254325697d4e32a6 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.8 seconds inserted 10000000 Rows, about 724K ops/s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.00% (8574/24495)
Line Coverage: 26.76% (69458/259558)
Region Coverage: 26.02% (36060/138568)
Branch Coverage: 22.99% (18420/80128)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3184fd1cc216faa40e037eba254325697d4e32a6_3184fd1cc216faa40e037eba254325697d4e32a6/report/index.html

mrhhsg
mrhhsg approved these changes Mar 13, 2024
View reviewed changes
Copy link
Member

@mrhhsg mrhhsg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

HappenLee
HappenLee approved these changes Mar 13, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 13, 2024
@HappenLee HappenLee merged commit 954750b into apache:master Mar 13, 2024
25 of 30 checks passed
@zclllyybb zclllyybb deleted the fix_upgrade branch March 13, 2024 16:35
@zclllyybb zclllyybb mentioned this pull request Mar 14, 2024
Merged
@yiguolei yiguolei added the usercase Important user case type label label Mar 14, 2024
@mrhhsg mrhhsg mentioned this pull request Mar 15, 2024
Merged
@yiguolei yiguolei mentioned this pull request Mar 24, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HappenLee HappenLee HappenLee approved these changes

@mrhhsg mrhhsg mrhhsg approved these changes

@yiguolei yiguolei Awaiting requested review from yiguolei

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. reviewed usercase Important user case type label
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@zclllyybb @doris-robot @mrhhsg @HappenLee @yiguolei