Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bugfix](paimon)upgrade paimon to 0.9 #41337

Merged
merged 3 commits into from
Oct 17, 2024

Conversation

wuwenchi
Copy link
Contributor

Proposed changes

upgrade paimon to 0.9

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@wuwenchi
Copy link
Contributor Author

run buildall

@wuwenchi
Copy link
Contributor Author

run buildall

1 similar comment
@CalvinKirs
Copy link
Member

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40928 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e18df769c5b8b08a1b6bd50782842f4191b035fc, data reload: false

------ Round 1 ----------------------------------
q1	17579	7493	7277	7277
q2	2018	275	276	275
q3	11842	1080	1210	1080
q4	10567	771	718	718
q5	7743	2871	2862	2862
q6	234	149	146	146
q7	990	612	626	612
q8	9347	2016	1982	1982
q9	6527	6397	6411	6397
q10	6994	2263	2275	2263
q11	440	250	253	250
q12	426	218	216	216
q13	17817	2946	3026	2946
q14	233	212	216	212
q15	583	514	518	514
q16	637	589	559	559
q17	973	619	599	599
q18	7223	6621	6742	6621
q19	1393	1040	1178	1040
q20	485	199	204	199
q21	4024	3169	3255	3169
q22	1098	991	1004	991
Total cold run time: 109173 ms
Total hot run time: 40928 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7288	7227	7267	7227
q2	328	230	235	230
q3	3028	2991	2969	2969
q4	2036	1815	1801	1801
q5	6086	5770	5718	5718
q6	236	147	143	143
q7	2191	1830	1815	1815
q8	3360	3535	3438	3438
q9	8944	8900	8846	8846
q10	3601	3558	3556	3556
q11	599	490	495	490
q12	883	632	609	609
q13	8247	3210	3195	3195
q14	316	271	280	271
q15	573	535	527	527
q16	670	640	652	640
q17	1849	1598	1638	1598
q18	8288	7738	7522	7522
q19	1728	1458	1424	1424
q20	2115	1879	1900	1879
q21	5630	5466	5420	5420
q22	1192	1069	1061	1061
Total cold run time: 69188 ms
Total hot run time: 60379 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192606 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e18df769c5b8b08a1b6bd50782842f4191b035fc, data reload: false

query1	848	409	402	402
query2	6241	2094	2083	2083
query3	8695	195	198	195
query4	33457	23606	23634	23606
query5	3436	475	474	474
query6	266	166	156	156
query7	4187	304	308	304
query8	276	215	209	209
query9	9371	2693	2697	2693
query10	454	289	293	289
query11	17842	15133	15252	15133
query12	157	99	98	98
query13	1555	445	418	418
query14	9715	7675	7346	7346
query15	247	171	177	171
query16	7755	465	451	451
query17	1627	627	612	612
query18	1989	331	341	331
query19	377	150	161	150
query20	124	112	117	112
query21	216	115	107	107
query22	4600	4538	4572	4538
query23	35795	34242	34026	34026
query24	11034	2874	2837	2837
query25	545	414	423	414
query26	736	164	166	164
query27	2067	313	303	303
query28	6681	2427	2430	2427
query29	696	436	433	433
query30	255	149	162	149
query31	1043	775	791	775
query32	104	57	56	56
query33	786	293	304	293
query34	925	495	495	495
query35	912	729	735	729
query36	1108	952	946	946
query37	153	90	97	90
query38	4052	3940	3928	3928
query39	1509	1425	1398	1398
query40	212	99	100	99
query41	54	48	49	48
query42	123	98	101	98
query43	572	513	494	494
query44	1249	826	816	816
query45	199	167	167	167
query46	1156	779	705	705
query47	1939	1830	1841	1830
query48	470	377	378	377
query49	886	418	421	418
query50	805	421	422	421
query51	6985	6916	6980	6916
query52	101	96	86	86
query53	250	185	184	184
query54	1271	476	484	476
query55	79	80	77	77
query56	292	270	267	267
query57	1221	1088	1078	1078
query58	250	266	243	243
query59	3545	3347	3047	3047
query60	287	270	267	267
query61	103	115	105	105
query62	853	668	661	661
query63	228	181	181	181
query64	3938	644	614	614
query65	3293	3305	3237	3237
query66	833	305	306	305
query67	15831	15548	15586	15548
query68	4466	587	577	577
query69	560	311	296	296
query70	1201	1118	1113	1113
query71	383	282	290	282
query72	7353	3990	4028	3990
query73	775	364	358	358
query74	10456	9040	8959	8959
query75	3428	2696	2673	2673
query76	3299	954	881	881
query77	588	303	298	298
query78	10396	9661	9497	9497
query79	4044	610	597	597
query80	3133	449	445	445
query81	596	239	242	239
query82	674	141	139	139
query83	317	134	135	134
query84	286	71	80	71
query85	2028	304	286	286
query86	504	299	294	294
query87	4513	4337	4353	4337
query88	3897	2428	2392	2392
query89	421	291	288	288
query90	2105	188	185	185
query91	180	140	144	140
query92	64	50	49	49
query93	3104	562	555	555
query94	1114	311	290	290
query95	360	254	257	254
query96	623	277	290	277
query97	3296	3111	3127	3111
query98	230	199	196	196
query99	1666	1297	1314	1297
Total cold run time: 304921 ms
Total hot run time: 192606 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.68 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e18df769c5b8b08a1b6bd50782842f4191b035fc, data reload: false

query1	0.04	0.05	0.04
query2	0.06	0.02	0.03
query3	0.23	0.06	0.06
query4	1.64	0.10	0.10
query5	0.50	0.51	0.52
query6	1.14	0.72	0.73
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.57	0.49	0.49
query10	0.55	0.56	0.55
query11	0.14	0.11	0.11
query12	0.14	0.10	0.10
query13	0.60	0.60	0.60
query14	2.71	2.73	2.83
query15	0.91	0.83	0.82
query16	0.38	0.39	0.38
query17	0.95	1.02	1.04
query18	0.20	0.20	0.20
query19	1.97	1.88	2.00
query20	0.01	0.02	0.01
query21	15.35	0.58	0.58
query22	2.83	2.15	1.69
query23	16.89	0.98	0.82
query24	2.36	1.20	1.00
query25	0.21	0.22	0.12
query26	0.26	0.14	0.14
query27	0.04	0.04	0.04
query28	11.02	1.09	1.07
query29	12.55	3.23	3.25
query30	0.24	0.05	0.06
query31	2.89	0.39	0.38
query32	3.25	0.47	0.47
query33	2.99	3.04	3.07
query34	17.04	4.51	4.44
query35	4.49	4.47	4.50
query36	0.67	0.50	0.47
query37	0.08	0.06	0.05
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.35 s
Total hot run time: 32.68 s

@wuwenchi
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41073 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2e3fc0a04e8d0d257edc5c6ffbe7188067807184, data reload: false

------ Round 1 ----------------------------------
q1	17632	7432	7278	7278
q2	2027	283	284	283
q3	12059	1051	1152	1051
q4	10570	874	884	874
q5	7754	3082	3059	3059
q6	237	155	153	153
q7	1036	634	607	607
q8	9353	1908	1948	1908
q9	6572	6488	6377	6377
q10	7032	2386	2434	2386
q11	431	247	243	243
q12	412	229	225	225
q13	17777	2977	3009	2977
q14	239	218	218	218
q15	575	526	516	516
q16	644	584	582	582
q17	969	668	474	474
q18	7261	6741	6586	6586
q19	1340	887	935	887
q20	479	190	186	186
q21	3942	3199	3239	3199
q22	1094	1019	1004	1004
Total cold run time: 109435 ms
Total hot run time: 41073 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7306	7299	7346	7299
q2	336	235	239	235
q3	3006	2933	2927	2927
q4	2155	1866	1856	1856
q5	5731	5727	5778	5727
q6	239	152	148	148
q7	2265	1820	1812	1812
q8	3392	3518	3437	3437
q9	8883	8830	8824	8824
q10	3582	3573	3529	3529
q11	584	514	509	509
q12	845	645	639	639
q13	8658	3246	3204	3204
q14	316	276	300	276
q15	597	532	524	524
q16	680	629	651	629
q17	1872	1614	1645	1614
q18	8238	7785	7718	7718
q19	1747	1438	1559	1438
q20	2141	1884	1912	1884
q21	5560	5538	5368	5368
q22	1109	1094	1042	1042
Total cold run time: 69242 ms
Total hot run time: 60639 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192068 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2e3fc0a04e8d0d257edc5c6ffbe7188067807184, data reload: false

query1	937	383	403	383
query2	6273	2038	1981	1981
query3	8682	193	214	193
query4	34222	23638	23519	23519
query5	3598	503	466	466
query6	269	164	162	162
query7	4209	303	284	284
query8	280	221	247	221
query9	9380	2786	2773	2773
query10	432	280	267	267
query11	17888	15457	15426	15426
query12	165	102	97	97
query13	1588	457	427	427
query14	10282	7110	7786	7110
query15	252	167	176	167
query16	7531	490	449	449
query17	1612	597	589	589
query18	2038	315	334	315
query19	367	163	158	158
query20	122	117	114	114
query21	221	111	110	110
query22	4787	4451	4489	4451
query23	38285	34421	34037	34037
query24	11034	2761	2827	2761
query25	527	419	410	410
query26	934	158	169	158
query27	2462	305	289	289
query28	7497	2438	2422	2422
query29	681	432	428	428
query30	260	157	156	156
query31	1054	772	801	772
query32	102	55	56	55
query33	763	292	288	288
query34	932	524	517	517
query35	882	760	751	751
query36	1098	955	954	954
query37	150	95	91	91
query38	4005	4002	4017	4002
query39	1474	1437	1438	1437
query40	262	98	100	98
query41	50	50	46	46
query42	126	100	101	100
query43	535	493	489	489
query44	1264	826	834	826
query45	196	165	165	165
query46	1133	729	724	724
query47	1944	1876	1845	1845
query48	412	329	318	318
query49	953	419	441	419
query50	804	415	405	405
query51	7177	6903	6871	6871
query52	104	93	86	86
query53	262	185	188	185
query54	1162	453	437	437
query55	84	82	82	82
query56	296	259	259	259
query57	1278	1156	1127	1127
query58	232	252	229	229
query59	3189	2862	2880	2862
query60	303	262	265	262
query61	105	100	108	100
query62	825	672	671	671
query63	232	193	189	189
query64	4617	632	620	620
query65	3274	3227	3203	3203
query66	860	299	310	299
query67	15809	15712	15511	15511
query68	4469	578	561	561
query69	517	293	294	293
query70	1182	1119	1126	1119
query71	337	272	281	272
query72	7070	4030	3961	3961
query73	775	364	363	363
query74	10229	8995	9050	8995
query75	3464	2715	2685	2685
query76	2927	913	885	885
query77	687	310	294	294
query78	10516	9702	9626	9626
query79	1197	604	606	604
query80	2033	464	464	464
query81	569	245	242	242
query82	702	141	140	140
query83	306	143	141	141
query84	289	68	72	68
query85	1328	303	292	292
query86	362	295	307	295
query87	4538	4362	4334	4334
query88	3380	2232	2189	2189
query89	406	290	292	290
query90	2161	189	190	189
query91	142	109	108	108
query92	65	46	49	46
query93	1068	531	535	531
query94	979	305	297	297
query95	353	253	250	250
query96	616	275	288	275
query97	3310	3144	3148	3144
query98	225	214	197	197
query99	1507	1300	1314	1300
Total cold run time: 302472 ms
Total hot run time: 192068 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.57 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2e3fc0a04e8d0d257edc5c6ffbe7188067807184, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.24	0.06	0.06
query4	1.64	0.10	0.10
query5	0.49	0.48	0.52
query6	1.13	0.72	0.73
query7	0.01	0.01	0.01
query8	0.04	0.04	0.03
query9	0.56	0.52	0.50
query10	0.54	0.55	0.54
query11	0.13	0.10	0.10
query12	0.14	0.12	0.11
query13	0.62	0.61	0.60
query14	2.83	2.73	2.72
query15	0.91	0.84	0.84
query16	0.38	0.38	0.38
query17	1.04	1.03	0.99
query18	0.20	0.20	0.21
query19	1.98	1.90	2.07
query20	0.01	0.00	0.00
query21	15.36	0.55	0.57
query22	2.80	2.66	1.54
query23	16.97	0.94	0.73
query24	2.99	0.27	1.19
query25	0.18	0.06	0.07
query26	0.45	0.15	0.12
query27	0.05	0.04	0.05
query28	11.33	1.12	1.07
query29	12.57	3.28	3.23
query30	0.24	0.06	0.06
query31	2.85	0.39	0.39
query32	3.26	0.47	0.45
query33	3.01	3.03	3.03
query34	17.09	4.52	4.46
query35	4.49	4.45	4.49
query36	0.69	0.49	0.48
query37	0.08	0.08	0.06
query38	0.05	0.03	0.04
query39	0.04	0.02	0.02
query40	0.16	0.13	0.12
query41	0.07	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.79 s
Total hot run time: 31.57 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 17, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@suxiaogang223 suxiaogang223 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@morningman morningman merged commit 0bbf502 into apache:master Oct 17, 2024
25 of 27 checks passed
morningman added a commit that referenced this pull request Oct 18, 2024
morningman added a commit that referenced this pull request Oct 18, 2024
Reverts #41337
paimon 0.9 introduce a breaking change that user can not read bucket
table without bucket key.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants