Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](map)fix upgrade behavior from 1.2 version #36937

Merged
merged 2 commits into from
Jul 1, 2024

Conversation

amorynan
Copy link
Contributor

Proposed changes

see also with https://github.com/apache/doris/pull/36635/files
Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@amorynan
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39631 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bedf5e8a5f3a4fe1e62c1155bde9350bc422f284, data reload: false

------ Round 1 ----------------------------------
q1	17627	4379	4269	4269
q2	2016	191	191	191
q3	10476	1215	1063	1063
q4	10189	908	790	790
q5	7464	2615	2596	2596
q6	218	134	134	134
q7	952	595	603	595
q8	9217	2077	2052	2052
q9	8739	6525	6508	6508
q10	9009	3688	3692	3688
q11	449	232	231	231
q12	419	229	232	229
q13	18841	2949	2990	2949
q14	275	228	222	222
q15	510	482	489	482
q16	510	378	368	368
q17	965	679	690	679
q18	7928	7514	7365	7365
q19	7281	1516	1561	1516
q20	650	309	334	309
q21	4908	3055	3880	3055
q22	388	340	340	340
Total cold run time: 119031 ms
Total hot run time: 39631 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4435	4270	4352	4270
q2	375	275	306	275
q3	3122	2921	2967	2921
q4	2018	1815	1709	1709
q5	5596	5531	5524	5524
q6	232	138	136	136
q7	2271	1872	1805	1805
q8	3329	3466	3420	3420
q9	8730	8816	8859	8816
q10	4341	3861	4060	3861
q11	621	520	517	517
q12	860	704	679	679
q13	16372	3397	3293	3293
q14	331	282	329	282
q15	546	488	489	488
q16	492	438	427	427
q17	1856	1513	1539	1513
q18	8688	7945	7790	7790
q19	1778	1714	1658	1658
q20	2115	1888	1844	1844
q21	9579	4864	4888	4864
q22	612	544	552	544
Total cold run time: 78299 ms
Total hot run time: 56636 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 171922 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bedf5e8a5f3a4fe1e62c1155bde9350bc422f284, data reload: false

query1	922	379	372	372
query2	6444	2456	2362	2362
query3	6626	214	215	214
query4	20608	17465	17407	17407
query5	3693	469	469	469
query6	270	166	169	166
query7	4591	287	284	284
query8	304	287	301	287
query9	8633	2339	2337	2337
query10	575	301	286	286
query11	10678	9982	10180	9982
query12	117	83	83	83
query13	1638	372	369	369
query14	10259	7762	7605	7605
query15	242	192	192	192
query16	7732	274	276	274
query17	1898	570	546	546
query18	1926	281	285	281
query19	208	157	170	157
query20	89	85	81	81
query21	208	134	130	130
query22	4324	4141	4001	4001
query23	33714	33592	33811	33592
query24	10858	2962	2931	2931
query25	614	370	365	365
query26	956	158	153	153
query27	2332	317	323	317
query28	6778	2080	2099	2080
query29	889	624	625	624
query30	258	160	162	160
query31	1004	768	730	730
query32	98	56	54	54
query33	761	281	281	281
query34	1051	472	491	472
query35	745	632	611	611
query36	1143	985	979	979
query37	147	77	74	74
query38	2979	2881	2846	2846
query39	858	818	864	818
query40	198	124	129	124
query41	54	50	54	50
query42	106	105	99	99
query43	607	542	537	537
query44	1192	717	724	717
query45	188	171	164	164
query46	1077	731	773	731
query47	1857	1759	1766	1759
query48	376	322	297	297
query49	846	398	404	398
query50	774	386	386	386
query51	7184	6829	6752	6752
query52	98	95	87	87
query53	367	292	283	283
query54	902	461	434	434
query55	73	71	73	71
query56	271	254	257	254
query57	1113	1034	1058	1034
query58	244	254	252	252
query59	3310	3278	3088	3088
query60	293	268	266	266
query61	91	91	95	91
query62	600	445	444	444
query63	311	293	300	293
query64	8838	2315	1746	1746
query65	3162	3098	3073	3073
query66	744	328	321	321
query67	15508	14968	15120	14968
query68	4764	519	532	519
query69	670	407	378	378
query70	1172	1127	1074	1074
query71	439	274	267	267
query72	8290	5185	2785	2785
query73	764	326	330	326
query74	5893	5622	5563	5563
query75	3733	2645	2641	2641
query76	3004	1021	912	912
query77	672	334	293	293
query78	10390	9808	9776	9776
query79	2970	505	518	505
query80	1503	472	463	463
query81	548	224	239	224
query82	1406	104	111	104
query83	431	171	169	169
query84	268	96	87	87
query85	1415	271	268	268
query86	468	308	293	293
query87	3281	3091	3076	3076
query88	3909	2452	2435	2435
query89	465	387	374	374
query90	1756	197	198	197
query91	127	99	97	97
query92	58	49	47	47
query93	2576	501	493	493
query94	1141	191	185	185
query95	396	375	318	318
query96	600	275	273	273
query97	3287	3065	3043	3043
query98	225	200	195	195
query99	1192	858	851	851
Total cold run time: 275367 ms
Total hot run time: 171922 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.13 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bedf5e8a5f3a4fe1e62c1155bde9350bc422f284, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.04	0.05
query4	1.67	0.08	0.07
query5	0.50	0.50	0.49
query6	1.13	0.72	0.72
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.55	0.49	0.47
query10	0.55	0.54	0.55
query11	0.16	0.12	0.12
query12	0.16	0.12	0.12
query13	0.58	0.59	0.60
query14	0.80	0.77	0.78
query15	0.84	0.81	0.82
query16	0.36	0.37	0.37
query17	1.04	0.97	1.04
query18	0.22	0.26	0.25
query19	1.81	1.81	1.72
query20	0.01	0.01	0.01
query21	15.43	0.77	0.65
query22	3.74	8.47	1.52
query23	18.33	1.36	1.27
query24	2.21	0.22	0.22
query25	0.15	0.08	0.09
query26	0.27	0.18	0.18
query27	0.08	0.08	0.09
query28	13.14	1.01	1.00
query29	12.63	3.36	3.36
query30	0.25	0.05	0.06
query31	2.88	0.37	0.38
query32	3.29	0.46	0.47
query33	2.87	2.91	2.86
query34	17.01	4.33	4.38
query35	4.51	4.50	4.46
query36	0.64	0.46	0.48
query37	0.19	0.15	0.14
query38	0.15	0.14	0.14
query39	0.04	0.04	0.04
query40	0.18	0.15	0.14
query41	0.09	0.05	0.05
query42	0.05	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 108.97 s
Total hot run time: 30.13 s

Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 28, 2024
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xiaokang
Copy link
Contributor

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39509 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f87ae16019ebca0dd650c2038f659337811faeb8, data reload: false

------ Round 1 ----------------------------------
q1	17617	4288	4217	4217
q2	2013	193	184	184
q3	10484	1179	1083	1083
q4	10193	706	821	706
q5	7496	2671	2700	2671
q6	214	140	138	138
q7	953	600	605	600
q8	9221	2057	2030	2030
q9	9044	6465	6426	6426
q10	8922	3640	3674	3640
q11	452	238	243	238
q12	475	237	228	228
q13	17756	3014	2993	2993
q14	262	224	227	224
q15	519	468	477	468
q16	505	372	373	372
q17	944	715	712	712
q18	7996	7389	7306	7306
q19	2747	1392	1546	1392
q20	633	317	323	317
q21	4954	3227	3920	3227
q22	391	337	337	337
Total cold run time: 113791 ms
Total hot run time: 39509 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4372	4244	4219	4219
q2	367	273	265	265
q3	2977	2741	2766	2741
q4	1962	1710	1690	1690
q5	5509	5682	5456	5456
q6	225	139	129	129
q7	2186	1807	1832	1807
q8	3268	3375	3403	3375
q9	8656	8672	8712	8672
q10	4111	3888	3916	3888
q11	578	498	492	492
q12	765	617	606	606
q13	15900	3186	3209	3186
q14	301	281	283	281
q15	528	503	484	484
q16	492	460	420	420
q17	1784	1512	1507	1507
q18	8022	8065	7711	7711
q19	1798	1672	1708	1672
q20	2083	1839	1866	1839
q21	5202	5033	4892	4892
q22	630	581	564	564
Total cold run time: 71716 ms
Total hot run time: 55896 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173429 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f87ae16019ebca0dd650c2038f659337811faeb8, data reload: false

query1	908	375	369	369
query2	6465	2388	2354	2354
query3	6641	207	220	207
query4	19716	17464	17429	17429
query5	3639	504	475	475
query6	254	189	165	165
query7	4587	300	295	295
query8	310	287	278	278
query9	8416	2385	2362	2362
query10	557	333	281	281
query11	10376	10078	9941	9941
query12	119	89	84	84
query13	1649	370	369	369
query14	10289	6977	7093	6977
query15	237	188	192	188
query16	7726	267	288	267
query17	1879	551	514	514
query18	1946	275	268	268
query19	192	147	148	147
query20	88	85	82	82
query21	212	134	132	132
query22	4211	3916	4070	3916
query23	33827	33549	33492	33492
query24	10729	2882	2877	2877
query25	586	387	369	369
query26	721	161	158	158
query27	2347	324	336	324
query28	5568	2167	2153	2153
query29	879	633	620	620
query30	259	162	161	161
query31	975	759	767	759
query32	94	59	53	53
query33	670	286	281	281
query34	884	478	493	478
query35	773	622	635	622
query36	1117	962	968	962
query37	156	74	77	74
query38	2982	2858	2870	2858
query39	910	829	831	829
query40	214	130	138	130
query41	60	57	53	53
query42	115	101	109	101
query43	618	564	563	563
query44	1070	743	732	732
query45	200	204	164	164
query46	1068	695	735	695
query47	1836	1768	1749	1749
query48	367	308	296	296
query49	843	411	404	404
query50	769	384	384	384
query51	6917	6715	6740	6715
query52	106	91	91	91
query53	360	291	298	291
query54	871	444	438	438
query55	75	72	74	72
query56	290	259	251	251
query57	1105	1069	1057	1057
query58	247	247	245	245
query59	3263	3239	3242	3239
query60	298	295	281	281
query61	90	91	92	91
query62	647	445	454	445
query63	320	290	289	289
query64	8482	2242	1726	1726
query65	3180	3082	3409	3082
query66	743	328	330	328
query67	15171	14823	14992	14823
query68	6801	555	554	554
query69	625	467	413	413
query70	1235	1070	1136	1070
query71	484	281	276	276
query72	7708	5243	5482	5243
query73	838	330	326	326
query74	6072	5549	5467	5467
query75	4025	2712	2665	2665
query76	4007	946	911	911
query77	651	312	302	302
query78	10227	9817	9702	9702
query79	3830	513	583	513
query80	1075	494	474	474
query81	506	220	219	219
query82	388	104	104	104
query83	199	173	176	173
query84	269	88	89	88
query85	857	279	265	265
query86	368	311	296	296
query87	3255	3085	3080	3080
query88	4022	2393	2360	2360
query89	488	374	390	374
query90	1840	188	187	187
query91	127	101	99	99
query92	56	50	48	48
query93	5144	520	522	520
query94	1229	189	190	189
query95	417	322	329	322
query96	596	261	263	261
query97	3234	2988	3100	2988
query98	205	198	191	191
query99	1164	848	844	844
Total cold run time: 275161 ms
Total hot run time: 173429 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.48 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f87ae16019ebca0dd650c2038f659337811faeb8, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.03
query3	0.23	0.04	0.04
query4	1.67	0.07	0.07
query5	0.50	0.50	0.49
query6	1.14	0.73	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.51	0.50
query10	0.54	0.55	0.55
query11	0.16	0.12	0.11
query12	0.14	0.11	0.12
query13	0.59	0.59	0.60
query14	0.77	0.79	0.78
query15	0.84	0.81	0.82
query16	0.37	0.37	0.35
query17	0.98	0.94	0.94
query18	0.22	0.26	0.24
query19	1.76	1.75	1.84
query20	0.01	0.01	0.01
query21	15.45	0.73	0.65
query22	4.29	7.43	1.73
query23	18.30	1.39	1.45
query24	2.11	0.23	0.24
query25	0.17	0.09	0.08
query26	0.26	0.17	0.18
query27	0.09	0.08	0.08
query28	13.22	1.02	1.00
query29	12.63	3.29	3.30
query30	0.25	0.06	0.06
query31	2.86	0.40	0.38
query32	3.30	0.47	0.46
query33	2.88	2.88	2.90
query34	17.12	4.40	4.47
query35	4.50	4.47	4.54
query36	0.65	0.45	0.46
query37	0.17	0.15	0.15
query38	0.15	0.14	0.14
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.09	0.04	0.05
query42	0.05	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.46 s
Total hot run time: 30.48 s

@xiaokang xiaokang merged commit 6f43278 into apache:master Jul 1, 2024
24 of 27 checks passed
dataroaring pushed a commit that referenced this pull request Jul 2, 2024
fix core dump when upgrading from 1.2.x to 2.0.x with map datatype column
@yiguolei yiguolei mentioned this pull request Jul 19, 2024
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants