Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](gson) Fix Expr deserialize compatibility #38799

Merged
merged 1 commit into from
Aug 4, 2024
Merged

[fix](gson) Fix Expr deserialize compatibility #38799

merged 1 commit into from
Aug 4, 2024

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented Aug 2, 2024

Since GsonBuilder.create() adds all registered factories to GSON in reverse order, and ExprAdapterFactory is registered before the RuntimeTypeAdapterFactory for Expr, ExprAdapterFactory will not be executed.

This PR adjusts their registration order. Now, it will first check-in ExprAdapterFactory whether to use the pre-134 deserialize method and then attempt to use the RuntimeTypeAdapterFactory for Expr.class.

@w41ter
[fix](gson) Fix Expr deserialize compatibility
eae0069 
Since GsonBuilder.create() adds all registered factories to GSON in reverse
order, and ExprAdapterFactory is registered before the RuntimeTypeAdapterFactory
for Expr, ExprAdapterFactory will not be executed.

This PR adjusts their registration order. Now, it will first check in
ExprAdapterFactory whether to use the pre-134 deserialize method, and then
attempt to use the RuntimeTypeAdapterFactory for Expr.class.
@w41ter w41ter requested a review from morrySnow August 2, 2024 11:44
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@w41ter
Copy link
Contributor Author

w41ter commented Aug 2, 2024

run buildall

@github-actions github-actions bot added the doing label Aug 2, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 41893 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eae00691cd2d3d78f451c488c42b590e6be0df87, data reload: false

------ Round 1 ----------------------------------
q1	18166	4212	4200	4200
q2	2458	217	205	205
q3	11618	1335	1367	1335
q4	10798	843	905	843
q5	7854	3014	3024	3014
q6	225	139	140	139
q7	1055	627	620	620
q8	9441	1847	1953	1847
q9	8493	6610	6597	6597
q10	8745	3846	3812	3812
q11	422	250	248	248
q12	412	231	229	229
q13	17785	2935	2931	2931
q14	279	241	245	241
q15	528	479	498	479
q16	483	399	387	387
q17	947	930	936	930
q18	8041	7372	7255	7255
q19	1380	1221	1225	1221
q20	566	317	328	317
q21	5257	4763	4801	4763
q22	365	286	280	280
Total cold run time: 115318 ms
Total hot run time: 41893 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4053	4034	4046	4034
q2	329	233	219	219
q3	3006	3008	2976	2976
q4	1882	1878	1872	1872
q5	5202	5296	5256	5256
q6	218	133	132	132
q7	2020	1690	1690	1690
q8	3179	3278	3233	3233
q9	8307	8247	8318	8247
q10	3740	3835	3836	3835
q11	546	472	453	453
q12	734	539	571	539
q13	11389	2951	2926	2926
q14	292	255	258	255
q15	526	486	483	483
q16	446	397	408	397
q17	1709	1707	1700	1700
q18	7807	7396	7165	7165
q19	1676	1697	1674	1674
q20	1959	1769	1768	1768
q21	5439	5154	5159	5154
q22	516	469	468	468
Total cold run time: 64975 ms
Total hot run time: 54476 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169426 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eae00691cd2d3d78f451c488c42b590e6be0df87, data reload: false

query1	907	371	376	371
query2	6481	1668	1678	1668
query3	6674	211	221	211
query4	19359	17442	17141	17141
query5	4365	515	530	515
query6	303	188	179	179
query7	4610	304	292	292
query8	240	208	195	195
query9	8511	2357	2349	2349
query10	448	286	276	276
query11	10525	9958	10110	9958
query12	142	91	87	87
query13	1620	396	393	393
query14	9353	7365	7086	7086
query15	211	163	170	163
query16	7118	457	471	457
query17	974	576	573	573
query18	1846	292	290	290
query19	201	150	149	149
query20	96	85	87	85
query21	204	101	101	101
query22	4180	3954	3925	3925
query23	33782	32964	32927	32927
query24	10401	3051	3021	3021
query25	703	406	428	406
query26	1746	157	157	157
query27	2837	286	282	282
query28	6938	1980	1954	1954
query29	1279	455	434	434
query30	287	153	157	153
query31	926	784	804	784
query32	103	54	58	54
query33	711	319	329	319
query34	906	495	495	495
query35	872	740	726	726
query36	1016	883	886	883
query37	294	80	82	80
query38	2874	2797	2804	2797
query39	873	830	823	823
query40	282	118	115	115
query41	49	47	51	47
query42	124	102	105	102
query43	472	422	416	416
query44	1203	760	747	747
query45	208	184	180	180
query46	1085	833	820	820
query47	1809	1709	1776	1709
query48	378	297	297	297
query49	1189	438	428	428
query50	897	452	448	448
query51	6842	6820	6722	6722
query52	98	95	89	89
query53	251	186	183	183
query54	655	448	456	448
query55	77	78	84	78
query56	284	252	288	252
query57	1160	1072	1027	1027
query58	269	262	264	262
query59	2545	2559	2439	2439
query60	316	276	270	270
query61	95	90	96	90
query62	916	668	657	657
query63	220	184	187	184
query64	5875	1892	1889	1889
query65	3226	3116	3136	3116
query66	1430	333	343	333
query67	15372	15001	14886	14886
query68	4837	587	588	587
query69	727	384	331	331
query70	1134	1072	1099	1072
query71	482	278	287	278
query72	8068	2679	2519	2519
query73	839	348	338	338
query74	6010	5667	5663	5663
query75	4030	2743	2745	2743
query76	3739	1252	1280	1252
query77	691	309	311	309
query78	9513	8872	8934	8872
query79	2040	543	542	542
query80	942	501	548	501
query81	545	223	231	223
query82	742	140	133	133
query83	316	168	178	168
query84	279	81	81	81
query85	1434	318	289	289
query86	416	290	263	263
query87	3252	3133	3129	3129
query88	3209	2534	2545	2534
query89	398	291	294	291
query90	2002	196	204	196
query91	131	101	101	101
query92	59	50	54	50
query93	2305	620	623	620
query94	889	293	289	289
query95	427	267	275	267
query96	626	292	284	284
query97	3217	3021	3071	3021
query98	227	208	196	196
query99	1597	1297	1291	1291
Total cold run time: 267366 ms
Total hot run time: 169426 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.24 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eae00691cd2d3d78f451c488c42b590e6be0df87, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.04	0.04
query4	1.68	0.06	0.07
query5	0.49	0.48	0.48
query6	1.14	0.72	0.72
query7	0.02	0.01	0.02
query8	0.06	0.04	0.05
query9	0.57	0.53	0.50
query10	0.56	0.57	0.56
query11	0.14	0.12	0.11
query12	0.16	0.12	0.13
query13	0.60	0.61	0.59
query14	0.78	0.79	0.80
query15	0.90	0.86	0.86
query16	0.35	0.35	0.35
query17	1.00	0.99	0.98
query18	0.23	0.21	0.21
query19	1.81	1.78	1.72
query20	0.01	0.01	0.00
query21	15.39	0.75	0.64
query22	4.06	7.20	1.67
query23	17.90	1.29	1.31
query24	2.28	0.23	0.22
query25	0.18	0.08	0.07
query26	0.32	0.21	0.21
query27	0.46	0.22	0.22
query28	13.22	0.99	0.97
query29	12.50	3.33	3.35
query30	0.26	0.06	0.06
query31	2.86	0.40	0.40
query32	3.24	0.49	0.49
query33	2.97	2.88	2.97
query34	15.43	4.25	4.24
query35	4.27	4.26	4.26
query36	0.68	0.48	0.49
query37	0.19	0.16	0.17
query38	0.16	0.16	0.15
query39	0.04	0.04	0.03
query40	0.17	0.13	0.13
query41	0.09	0.04	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 107.61 s
Total hot run time: 30.24 s

dataroaring
dataroaring approved these changes Aug 2, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 2, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 2, 2024

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit 0b61263 into apache:master Aug 4, 2024
30 of 31 checks passed
dataroaring pushed a commit that referenced this pull request Aug 4, 2024
@w41ter @dataroaring
[fix](gson) Fix Expr deserialize compatibility (#38799)
5f67a85 
Since GsonBuilder.create() adds all registered factories to GSON in
reverse order, and ExprAdapterFactory is registered before the
RuntimeTypeAdapterFactory for Expr, ExprAdapterFactory will not be
executed.

This PR adjusts their registration order. Now, it will first check-in
ExprAdapterFactory whether to use the pre-134 deserialize method and
then attempt to use the RuntimeTypeAdapterFactory for Expr.class.
@w41ter w41ter deleted the fix_expr_deserialize branch August 5, 2024 01:34
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

@gavinchou gavinchou gavinchou approved these changes

@morrySnow morrySnow Awaiting requested review from morrySnow

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.1-merged doing p0_b reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@w41ter @doris-robot @gavinchou @dataroaring