Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[bug](fold) fix fold constant core dump with variant type #32265

Merged
merged 15 commits into from
Mar 22, 2024
Merged

[bug](fold) fix fold constant core dump with variant type #32265

merged 15 commits into from
Mar 22, 2024

Conversation

zhangstar333
Copy link
Contributor

@zhangstar333 zhangstar333 commented Mar 14, 2024 •
edited
Loading

Proposed changes

  1. variant type core dump at call get_data_at function, as not impl this function.
  2. some case can't pass at old planner and fold_constant_by_be = on.
  3. open enable_fold_constant_by_be = true.
F20240314 19:24:30.485731 1887808 column_object.h:415] should not call the method in column object
*** Check failure stack trace: ***
    @     0x55a27aeb85f6  google::LogMessage::SendToLog()
    @     0x55a27aeb5040  google::LogMessage::Flush()
    @     0x55a27aeb8e39  google::LogMessageFatal::~LogMessageFatal()
    @     0x55a2744d0521  doris::vectorized::ColumnObject::get_data_at()
    @     0x55a2744b9f28  doris::vectorized::ColumnNullable::get_data_at()
    @     0x55a2716979b8  doris::FoldConstantExecutor::fold_constant_vexpr()
    @     0x55a271661126  doris::PInternalServiceImpl::_fold_constant_expr()
    @     0x55a27166e01d  std::_Function_handler<>::_M_invoke()
    @     0x55a27167d9cb  doris::WorkThreadPool<>::work_thread()
    @     0x55a27dc23050  execute_native_thread_routine
    @     0x7f01f659c609  start_thread
    @     0x7f01f6849133  clone
    @              (nil)  (unknown)

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 34.99% (8573/24499)
Line Coverage: 26.74% (69464/259805)
Region Coverage: 26.02% (36081/138691)
Branch Coverage: 22.97% (18421/80208)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b1139731961f19fc45ad48bb97a29e1bf0c58c5d_b1139731961f19fc45ad48bb97a29e1bf0c58c5d/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 37923 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b1139731961f19fc45ad48bb97a29e1bf0c58c5d, data reload: false

------ Round 1 ----------------------------------
q1	17704	4170	4093	4093
q2	2121	156	153	153
q3	10587	1092	897	897
q4	7791	707	728	707
q5	7486	2574	2630	2574
q6	181	123	123	123
q7	1186	810	803	803
q8	9364	2005	1993	1993
q9	7172	6391	6383	6383
q10	8514	3480	3601	3480
q11	444	220	218	218
q12	747	294	290	290
q13	17957	2851	2862	2851
q14	270	248	252	248
q15	504	452	457	452
q16	499	391	395	391
q17	940	504	540	504
q18	7061	6462	6336	6336
q19	1581	1456	1377	1377
q20	561	278	274	274
q21	6060	3480	3510	3480
q22	345	317	296	296
Total cold run time: 109075 ms
Total hot run time: 37923 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4068	4045	4063	4045
q2	327	232	227	227
q3	2955	2852	2824	2824
q4	1807	1524	1528	1524
q5	5186	5199	5232	5199
q6	194	114	122	114
q7	2194	1800	1836	1800
q8	3139	3286	3264	3264
q9	8529	8545	8472	8472
q10	3688	3704	3685	3685
q11	542	448	453	448
q12	734	564	531	531
q13	16927	2846	2856	2846
q14	272	239	252	239
q15	478	439	442	439
q16	452	436	418	418
q17	1704	1482	1476	1476
q18	7451	7158	7108	7108
q19	1715	1564	1523	1523
q20	1882	1723	1709	1709
q21	4847	4643	4775	4643
q22	525	458	466	458
Total cold run time: 69616 ms
Total hot run time: 52992 ms

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37947 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 89d58a5d5045ccc9f0ab4328e4b5f8c34d9e1332, data reload: false

------ Round 1 ----------------------------------
q1	17663	4184	4123	4123
q2	2101	147	145	145
q3	10606	1049	871	871
q4	7079	668	710	668
q5	7456	2704	2680	2680
q6	185	129	121	121
q7	1159	795	797	795
q8	9322	2005	1948	1948
q9	7213	6486	6366	6366
q10	8542	3483	3613	3483
q11	431	223	212	212
q12	756	280	279	279
q13	18147	2844	2841	2841
q14	278	247	246	246
q15	494	441	448	441
q16	519	390	388	388
q17	938	510	591	510
q18	7072	6465	6422	6422
q19	4738	1362	1399	1362
q20	541	276	285	276
q21	6195	3486	3523	3486
q22	343	284	308	284
Total cold run time: 111778 ms
Total hot run time: 37947 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4142	4081	4070	4070
q2	320	227	231	227
q3	2933	2807	2807	2807
q4	1828	1514	1507	1507
q5	5180	5200	5198	5198
q6	189	114	113	113
q7	2198	1810	1819	1810
q8	3137	3269	3244	3244
q9	8552	8533	8590	8533
q10	3679	3668	3619	3619
q11	537	439	424	424
q12	709	515	549	515
q13	16936	2830	2833	2830
q14	284	246	258	246
q15	478	438	442	438
q16	455	394	425	394
q17	1718	1469	1447	1447
q18	7466	7139	7071	7071
q19	1593	1468	1526	1468
q20	1907	1701	1689	1689
q21	4892	4619	4642	4619
q22	577	466	434	434
Total cold run time: 69710 ms
Total hot run time: 52703 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 34.95% (8574/24531)
Line Coverage: 26.66% (69483/260587)
Region Coverage: 25.96% (36092/139046)
Branch Coverage: 22.92% (18433/80434)
Coverage Report: http://coverage.selectdb-in.cc/coverage/89d58a5d5045ccc9f0ab4328e4b5f8c34d9e1332_89d58a5d5045ccc9f0ab4328e4b5f8c34d9e1332/report/index.html

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 34.95% (8574/24534)
Line Coverage: 26.65% (69484/260725)
Region Coverage: 25.95% (36098/139116)
Branch Coverage: 22.90% (18430/80484)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3d0ab498338027909fe3ad706f506e1bce524559_3d0ab498338027909fe3ad706f506e1bce524559/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 38266 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3d0ab498338027909fe3ad706f506e1bce524559, data reload: false

------ Round 1 ----------------------------------
q1	17691	4216	4107	4107
q2	2113	150	146	146
q3	10595	1087	891	891
q4	7755	731	696	696
q5	7474	2749	2744	2744
q6	185	122	120	120
q7	1157	810	797	797
q8	9367	2002	1999	1999
q9	7130	6446	6429	6429
q10	8483	3471	3596	3471
q11	431	229	213	213
q12	667	299	285	285
q13	17804	2871	2841	2841
q14	291	248	242	242
q15	504	454	457	454
q16	516	390	397	390
q17	935	537	625	537
q18	7072	6474	6438	6438
q19	3198	1454	1385	1385
q20	550	282	277	277
q21	6174	3500	3521	3500
q22	340	313	304	304
Total cold run time: 110432 ms
Total hot run time: 38266 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4115	4046	4080	4046
q2	331	234	221	221
q3	2952	2863	2843	2843
q4	1843	1508	1540	1508
q5	5184	5225	5225	5225
q6	196	122	121	121
q7	2215	1832	1849	1832
q8	3104	3253	3248	3248
q9	8533	8513	8537	8513
q10	3679	3695	3644	3644
q11	526	457	429	429
q12	725	527	565	527
q13	16915	2870	2854	2854
q14	270	256	255	255
q15	472	436	447	436
q16	460	402	404	402
q17	1714	1490	1445	1445
q18	7436	7117	7080	7080
q19	1597	1536	1498	1498
q20	1901	1752	1710	1710
q21	4841	4650	4739	4650
q22	542	443	429	429
Total cold run time: 69551 ms
Total hot run time: 52916 ms

@zhangstar333
Copy link
Contributor Author

run buildall

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38342 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4cb4b62ae5318e92fe875c7335d9092367625c0c, data reload: false

------ Round 1 ----------------------------------
q1	17638	4486	4105	4105
q2	2028	156	147	147
q3	10596	1121	896	896
q4	7781	769	709	709
q5	7475	2712	2700	2700
q6	183	121	123	121
q7	1187	822	812	812
q8	9329	2029	1977	1977
q9	7183	6449	6486	6449
q10	8521	3544	3646	3544
q11	437	219	220	219
q12	784	306	301	301
q13	18297	2838	2843	2838
q14	279	249	252	249
q15	504	465	449	449
q16	518	389	393	389
q17	980	523	583	523
q18	7183	6473	6391	6391
q19	1700	1438	1506	1438
q20	559	291	284	284
q21	6198	3545	3493	3493
q22	360	308	333	308
Total cold run time: 109720 ms
Total hot run time: 38342 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4156	4064	4054	4054
q2	324	226	219	219
q3	2973	2820	2855	2820
q4	1818	1593	1562	1562
q5	5229	5279	5209	5209
q6	206	116	118	116
q7	2225	1851	1831	1831
q8	3162	3315	3271	3271
q9	8596	8526	8593	8526
q10	3726	3713	3617	3617
q11	554	436	445	436
q12	708	535	525	525
q13	16928	2852	2844	2844
q14	279	244	254	244
q15	481	444	447	444
q16	450	404	424	404
q17	1737	1484	1450	1450
q18	7525	7292	7203	7203
q19	1638	1513	1428	1428
q20	1906	1715	1702	1702
q21	4862	4760	4720	4720
q22	529	480	450	450
Total cold run time: 70012 ms
Total hot run time: 53075 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 34.94% (8574/24540)
Line Coverage: 26.64% (69484/260787)
Region Coverage: 25.94% (36100/139146)
Branch Coverage: 22.89% (18428/80516)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4cb4b62ae5318e92fe875c7335d9092367625c0c_4cb4b62ae5318e92fe875c7335d9092367625c0c/report/index.html

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.25% (8700/24683)
Line Coverage: 27.08% (71176/262882)
Region Coverage: 26.32% (36914/140245)
Branch Coverage: 23.24% (18884/81256)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b20c64152f4c37e1f476f8a472681f3b188569eb_b20c64152f4c37e1f476f8a472681f3b188569eb/report/index.html

@zhangstar333
Copy link
Contributor Author

run buildall

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.25% (8700/24683)
Line Coverage: 27.08% (71176/262882)
Region Coverage: 26.32% (36913/140245)
Branch Coverage: 23.24% (18883/81256)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4921ad6a4b00ba112b300cbbaf5bbce22b5b33ef_4921ad6a4b00ba112b300cbbaf5bbce22b5b33ef/report/index.html

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.27% (8728/24746)
Line Coverage: 27.08% (71422/263764)
Region Coverage: 26.32% (37063/140816)
Branch Coverage: 23.24% (18958/81588)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d13e832506186a4cb045665bbe7cb17f2b03725c_d13e832506186a4cb045665bbe7cb17f2b03725c/report/index.html

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38264 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f537f58ac57d6a7b21929e1e6865236338f5a3d8, data reload: false

------ Round 1 ----------------------------------
q1	17749	4463	4226	4226
q2	2311	161	157	157
q3	11192	1135	1203	1135
q4	10512	825	793	793
q5	7983	3105	3028	3028
q6	208	127	128	127
q7	1041	594	604	594
q8	9397	2075	2013	2013
q9	7211	6681	6570	6570
q10	8569	3541	3550	3541
q11	427	226	228	226
q12	368	197	195	195
q13	17798	2858	2857	2857
q14	241	215	203	203
q15	498	467	446	446
q16	463	362	376	362
q17	980	595	555	555
q18	7331	6440	6417	6417
q19	1613	1418	1469	1418
q20	546	275	254	254
q21	3557	2854	2962	2854
q22	342	293	294	293
Total cold run time: 110337 ms
Total hot run time: 38264 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4119	4058	4078	4058
q2	326	230	232	230
q3	2986	2893	2990	2893
q4	1927	1631	1607	1607
q5	5234	5314	5379	5314
q6	204	120	124	120
q7	2265	1851	1850	1850
q8	3185	3343	3278	3278
q9	8620	8626	8644	8626
q10	3696	3666	3617	3617
q11	541	432	442	432
q12	730	553	551	551
q13	16904	2819	2838	2819
q14	266	249	254	249
q15	485	445	443	443
q16	453	422	410	410
q17	1763	1496	1497	1496
q18	7515	7139	7087	7087
q19	1859	1530	1484	1484
q20	1884	1719	1677	1677
q21	4617	4750	4722	4722
q22	526	451	464	451
Total cold run time: 70105 ms
Total hot run time: 53414 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.28% (8732/24751)
Line Coverage: 27.10% (71513/263924)
Region Coverage: 26.32% (37091/140917)
Branch Coverage: 23.24% (18975/81652)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f537f58ac57d6a7b21929e1e6865236338f5a3d8_f537f58ac57d6a7b21929e1e6865236338f5a3d8/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 180448 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f537f58ac57d6a7b21929e1e6865236338f5a3d8, data reload: false

query1	927	355	344	344
query2	7359	2013	2047	2013
query3	6701	210	209	209
query4	31328	20613	20605	20605
query5	4264	390	406	390
query6	273	181	177	177
query7	4629	288	297	288
query8	229	167	173	167
query9	9908	2270	2266	2266
query10	568	237	263	237
query11	17138	14144	14102	14102
query12	129	95	88	88
query13	1633	430	407	407
query14	11496	10345	10254	10254
query15	271	197	194	194
query16	8162	270	257	257
query17	1946	571	534	534
query18	2083	267	269	267
query19	228	143	145	143
query20	91	85	88	85
query21	205	121	130	121
query22	4626	4475	4453	4453
query23	31816	31065	30919	30919
query24	11415	2818	2791	2791
query25	606	373	375	373
query26	1680	147	151	147
query27	3154	342	336	336
query28	8187	1869	1884	1869
query29	991	642	624	624
query30	301	146	145	145
query31	966	709	756	709
query32	93	58	56	56
query33	773	264	333	264
query34	994	478	477	477
query35	826	621	613	613
query36	999	858	831	831
query37	123	87	77	77
query38	3568	3457	3425	3425
query39	1437	1370	1356	1356
query40	299	112	112	112
query41	49	47	50	47
query42	100	96	97	96
query43	485	440	467	440
query44	1208	730	710	710
query45	289	259	261	259
query46	1084	679	672	672
query47	1635	1584	1584	1584
query48	425	345	359	345
query49	1185	344	341	341
query50	778	364	372	364
query51	6703	6598	6671	6598
query52	97	97	88	88
query53	344	271	276	271
query54	306	261	257	257
query55	85	83	79	79
query56	245	247	236	236
query57	1084	1005	1005	1005
query58	230	209	206	206
query59	2841	2672	2660	2660
query60	269	242	259	242
query61	117	113	110	110
query62	599	392	390	390
query63	299	278	273	273
query64	6480	3944	3839	3839
query65	3089	3025	2988	2988
query66	1446	366	375	366
query67	14935	14715	14228	14228
query68	8948	520	537	520
query69	612	375	374	374
query70	1222	1170	1199	1170
query71	522	286	291	286
query72	6859	2652	2490	2490
query73	1117	319	316	316
query74	8328	6600	6586	6586
query75	4288	2841	2756	2756
query76	5382	983	1002	983
query77	645	276	258	258
query78	10488	9606	9620	9606
query79	11779	518	521	518
query80	2544	416	383	383
query81	493	213	207	207
query82	360	199	202	199
query83	240	144	139	139
query84	288	76	80	76
query85	1219	332	316	316
query86	363	289	290	289
query87	3712	3523	3521	3521
query88	5360	2296	2283	2283
query89	476	366	376	366
query90	2037	177	177	177
query91	173	135	134	134
query92	63	48	48	48
query93	7321	495	478	478
query94	1365	174	172	172
query95	453	330	333	330
query96	623	263	272	263
query97	3024	2855	2866	2855
query98	229	210	201	201
query99	1062	729	741	729
Total cold run time: 320424 ms
Total hot run time: 180448 ms

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit f537f58ac57d6a7b21929e1e6865236338f5a3d8 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       22.3 seconds inserted 10000000 Rows, about 448K ops/s

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38111 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f7f34eed10928e45d67a9fc9a451517f387ff5cd, data reload: false

------ Round 1 ----------------------------------
q1	17896	4677	4225	4225
q2	2724	165	157	157
q3	11756	1132	1184	1132
q4	10677	825	824	824
q5	8422	3023	3034	3023
q6	211	130	129	129
q7	1059	590	573	573
q8	9654	2067	2026	2026
q9	7255	6672	6462	6462
q10	8396	3421	3468	3421
q11	431	230	215	215
q12	364	192	193	192
q13	17792	2845	2852	2845
q14	232	200	199	199
q15	483	452	457	452
q16	478	364	354	354
q17	968	645	573	573
q18	7227	6556	6684	6556
q19	3080	1432	1458	1432
q20	571	252	248	248
q21	3536	2783	2970	2783
q22	345	303	290	290
Total cold run time: 113557 ms
Total hot run time: 38111 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4156	4081	4077	4077
q2	325	232	227	227
q3	3000	2951	2840	2840
q4	1911	1601	1593	1593
q5	5235	5281	5264	5264
q6	197	118	114	114
q7	2249	1841	1835	1835
q8	3163	3317	3291	3291
q9	8624	8590	8592	8590
q10	3670	3702	3677	3677
q11	535	440	434	434
q12	713	553	549	549
q13	16890	2852	2833	2833
q14	276	251	270	251
q15	472	434	437	434
q16	457	414	418	414
q17	1731	1488	1447	1447
q18	7417	7329	7093	7093
q19	1608	1518	1533	1518
q20	1873	1717	1680	1680
q21	4754	4625	4650	4625
q22	535	458	434	434
Total cold run time: 69791 ms
Total hot run time: 53220 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.28% (8732/24753)
Line Coverage: 27.09% (71507/264006)
Region Coverage: 26.33% (37104/140910)
Branch Coverage: 23.23% (18973/81680)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f7f34eed10928e45d67a9fc9a451517f387ff5cd_f7f34eed10928e45d67a9fc9a451517f387ff5cd/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 180867 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f7f34eed10928e45d67a9fc9a451517f387ff5cd, data reload: false

query1	905	352	343	343
query2	7453	2048	2023	2023
query3	6707	203	207	203
query4	31130	20649	20713	20649
query5	4431	412	395	395
query6	284	177	173	173
query7	4622	285	284	284
query8	228	164	179	164
query9	9936	2250	2250	2250
query10	563	238	247	238
query11	17055	13987	14235	13987
query12	135	92	85	85
query13	1634	425	411	411
query14	11212	10795	10899	10795
query15	264	192	188	188
query16	8147	263	256	256
query17	1949	568	530	530
query18	2074	284	278	278
query19	265	154	147	147
query20	90	87	84	84
query21	202	123	123	123
query22	4739	4508	4562	4508
query23	31779	31105	31358	31105
query24	10862	2803	2776	2776
query25	619	367	369	367
query26	1426	148	147	147
query27	3066	345	335	335
query28	7999	1858	1832	1832
query29	886	634	612	612
query30	301	143	144	143
query31	950	738	703	703
query32	85	57	53	53
query33	773	254	246	246
query34	1057	464	474	464
query35	826	607	606	606
query36	1015	886	877	877
query37	124	84	77	77
query38	3530	3376	3379	3376
query39	1444	1373	1373	1373
query40	220	123	109	109
query41	51	49	47	47
query42	101	94	99	94
query43	480	445	447	445
query44	1291	705	702	702
query45	285	249	258	249
query46	1070	682	680	680
query47	1649	1629	1593	1593
query48	448	357	348	348
query49	1136	339	342	339
query50	761	361	370	361
query51	6661	6575	6558	6558
query52	109	89	90	89
query53	340	270	273	270
query54	306	248	245	245
query55	79	79	87	79
query56	298	239	238	238
query57	1086	1008	1012	1008
query58	232	206	208	206
query59	2749	2569	2642	2569
query60	277	272	256	256
query61	118	129	129	129
query62	610	396	400	396
query63	307	279	281	279
query64	5804	3987	3959	3959
query65	3141	3001	2999	2999
query66	1440	360	375	360
query67	15134	14426	14354	14354
query68	8662	535	534	534
query69	619	385	380	380
query70	1268	1179	1117	1117
query71	531	281	277	277
query72	6962	2658	2492	2492
query73	738	308	317	308
query74	6890	6513	6588	6513
query75	4326	2838	2797	2797
query76	5030	866	1004	866
query77	657	254	252	252
query78	10530	9738	9567	9567
query79	9962	521	508	508
query80	1725	386	369	369
query81	509	213	209	209
query82	662	196	202	196
query83	214	143	141	141
query84	282	78	76	76
query85	1409	335	311	311
query86	417	307	278	278
query87	3751	3462	3496	3462
query88	4476	2288	2294	2288
query89	496	350	365	350
query90	2002	175	171	171
query91	166	136	136	136
query92	61	48	49	48
query93	7340	501	492	492
query94	1258	174	171	171
query95	428	329	317	317
query96	599	265	273	265
query97	3050	2863	2863	2863
query98	223	219	200	200
query99	1111	746	752	746
Total cold run time: 313304 ms
Total hot run time: 180867 ms

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit f7f34eed10928e45d67a9fc9a451517f387ff5cd with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.9 seconds inserted 10000000 Rows, about 456K ops/s

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37509 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a2e719c1ba2acb921c315de186735b967375a660, data reload: false

------ Round 1 ----------------------------------
q1	17634	4207	4089	4089
q2	2099	159	154	154
q3	10586	1107	1167	1107
q4	10221	731	763	731
q5	7445	3015	2925	2925
q6	211	125	126	125
q7	1016	577	562	562
q8	9336	1969	1980	1969
q9	7192	6460	6436	6436
q10	8393	3329	3436	3329
q11	432	220	219	219
q12	456	192	189	189
q13	17806	2851	2849	2849
q14	239	199	212	199
q15	499	448	452	448
q16	476	378	363	363
q17	937	535	610	535
q18	7111	6558	6451	6451
q19	4364	1430	1403	1403
q20	544	255	246	246
q21	3492	2965	2878	2878
q22	338	306	302	302
Total cold run time: 110827 ms
Total hot run time: 37509 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4152	4112	4042	4042
q2	328	232	231	231
q3	2927	2817	2813	2813
q4	1850	1600	1525	1525
q5	5232	5256	5288	5256
q6	196	116	115	115
q7	2185	1847	1841	1841
q8	3134	3300	3305	3300
q9	8539	8519	8519	8519
q10	3699	3703	3660	3660
q11	544	424	436	424
q12	740	552	536	536
q13	16925	2840	2868	2840
q14	279	247	245	245
q15	494	462	461	461
q16	464	414	412	412
q17	1718	1485	1478	1478
q18	7510	7094	7001	7001
q19	1628	1508	1569	1508
q20	1903	1719	1703	1703
q21	4867	4543	4760	4543
q22	538	440	452	440
Total cold run time: 69852 ms
Total hot run time: 52893 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 182618 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a2e719c1ba2acb921c315de186735b967375a660, data reload: false

query1	932	363	348	348
query2	7399	1979	1953	1953
query3	6698	211	214	211
query4	31393	20659	20740	20659
query5	4333	399	407	399
query6	268	174	177	174
query7	4630	287	290	287
query8	237	191	166	166
query9	9353	2277	2295	2277
query10	556	257	259	257
query11	15460	14157	14186	14157
query12	139	95	85	85
query13	1628	411	413	411
query14	12015	11343	11281	11281
query15	253	221	190	190
query16	8005	259	259	259
query17	1992	561	526	526
query18	1652	283	281	281
query19	350	163	160	160
query20	98	88	87	87
query21	205	130	126	126
query22	4780	4545	4617	4545
query23	31833	31273	31008	31008
query24	10575	2835	2808	2808
query25	636	399	383	383
query26	1095	155	158	155
query27	2460	367	359	359
query28	6962	1907	1908	1907
query29	865	637	629	629
query30	301	149	149	149
query31	980	741	750	741
query32	92	64	60	60
query33	774	259	254	254
query34	1047	475	484	475
query35	835	605	627	605
query36	1007	861	887	861
query37	119	79	79	79
query38	3549	3476	3477	3476
query39	1438	1389	1371	1371
query40	214	116	114	114
query41	51	49	47	47
query42	102	100	104	100
query43	476	441	446	441
query44	1195	719	728	719
query45	272	274	257	257
query46	1105	709	693	693
query47	1666	1615	1634	1615
query48	446	376	375	375
query49	1125	345	361	345
query50	763	371	377	371
query51	6765	6573	6546	6546
query52	119	88	97	88
query53	347	275	283	275
query54	313	248	254	248
query55	89	83	79	79
query56	256	233	240	233
query57	1083	1048	1021	1021
query58	234	212	214	212
query59	2780	2587	2593	2587
query60	279	263	263	263
query61	119	117	131	117
query62	614	401	410	401
query63	306	278	289	278
query64	5582	3920	3908	3908
query65	3028	3006	3027	3006
query66	888	356	360	356
query67	15104	14532	14625	14532
query68	8871	540	552	540
query69	657	390	383	383
query70	1219	1176	1160	1160
query71	517	281	273	273
query72	6364	2688	2497	2497
query73	771	313	316	313
query74	8570	6631	6662	6631
query75	4507	2935	2870	2870
query76	5516	871	902	871
query77	759	264	257	257
query78	10468	9682	9620	9620
query79	9551	513	518	513
query80	1404	400	388	388
query81	528	209	214	209
query82	820	211	203	203
query83	203	150	146	146
query84	286	83	75	75
query85	1453	327	310	310
query86	417	309	314	309
query87	3722	3604	3545	3545
query88	4732	2296	2298	2296
query89	514	372	362	362
query90	2054	176	176	176
query91	173	138	160	138
query92	61	49	49	49
query93	6878	519	486	486
query94	1182	184	176	176
query95	433	338	333	333
query96	611	268	267	267
query97	3046	2917	2876	2876
query98	234	211	217	211
query99	1126	749	782	749
Total cold run time: 309912 ms
Total hot run time: 182618 ms

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit a2e719c1ba2acb921c315de186735b967375a660 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.28% (8733/24756)
Line Coverage: 27.08% (71519/264132)
Region Coverage: 26.32% (37112/140989)
Branch Coverage: 23.23% (18985/81738)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a2e719c1ba2acb921c315de186735b967375a660_a2e719c1ba2acb921c315de186735b967375a660/report/index.html

HappenLee
HappenLee approved these changes Mar 22, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Mar 22, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@zhangstar333 zhangstar333 merged commit 8d966fa into apache:master Mar 22, 2024
24 of 30 checks passed
yiguolei pushed a commit that referenced this pull request Mar 22, 2024
@zhangstar333 @Doris-Extras
[bug](fold) fix fold constant core dump with variant type (#32265)
e41311d 
1. variant type core dump at call get_data_at function, as not impl this function.
2. some case can't pass at old planner and fold_constant_by_be = on.
3. open enable_fold_constant_by_be = true.
@yiguolei yiguolei mentioned this pull request Mar 24, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HappenLee HappenLee HappenLee approved these changes

@BiteTheDDDDt BiteTheDDDDt BiteTheDDDDt approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. reviewed variant
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@zhangstar333 @doris-robot @BiteTheDDDDt @HappenLee @xiaokang