Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix](inverted index) Resolve null processing issue in arrays_overlap #41495

Merged
merged 1 commit into from
Oct 9, 2024

Conversation

airborne12
Copy link
Member

Proposed changes

Fix problem "Runtime Error: Null pointer passed to 'StringTypeInvertedIndexReader::query', which requires a non-null argument."


Stack Trace:
#0 doris::segment_v2::StringTypeInvertedIndexReader::query(...) inverted_index_reader.cpp:473
#1 doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(...) inverted_index_reader.cpp:1237
#2 doris::vectorized::FunctionArraysOverlap::evaluate_inverted_index(...) function_arrays_overlap.h:192
#3 doris::vectorized::DefaultFunction::evaluate_inverted_index(...) function.h:532
#4 doris::vectorized::VExpr::_evaluate_inverted_index(...) vexpr.cpp:708
#5 doris::vectorized::VectorizedFnCall::evaluate_inverted_index(...) vectorized_fn_call.cpp:143
#6 doris::vectorized::VExprContext::evaluate_inverted_index(...) vexpr_context.cpp:126

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@airborne12
Copy link
Member Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.32% (9630/25805)
Line Coverage: 28.70% (79860/278277)
Region Coverage: 28.15% (41303/146745)
Branch Coverage: 24.76% (21038/84968)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c23ba1abe59c58a131a71c835989c91e24eae59c_c23ba1abe59c58a131a71c835989c91e24eae59c/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 42598 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c23ba1abe59c58a131a71c835989c91e24eae59c, data reload: false

------ Round 1 ----------------------------------
q1	17669	8350	8030	8030
q2	2016	328	181	181
q3	10462	1231	1318	1231
q4	10417	777	899	777
q5	7760	3050	2917	2917
q6	261	156	157	156
q7	1124	629	601	601
q8	9349	2070	2000	2000
q9	6989	6861	6754	6754
q10	7018	2345	2342	2342
q11	458	247	250	247
q12	455	226	223	223
q13	17783	3003	2994	2994
q14	250	221	211	211
q15	600	523	534	523
q16	655	593	576	576
q17	1031	581	671	581
q18	7351	6684	6674	6674
q19	1370	1041	1130	1041
q20	494	210	204	204
q21	4216	3350	3519	3350
q22	1105	985	990	985
Total cold run time: 108833 ms
Total hot run time: 42598 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7953	7867	7824	7824
q2	366	241	241	241
q3	3204	3091	3054	3054
q4	2255	1890	1841	1841
q5	5843	5879	5921	5879
q6	254	145	151	145
q7	2405	1783	1848	1783
q8	3551	3782	3815	3782
q9	9046	9057	9014	9014
q10	3749	3699	3753	3699
q11	635	487	488	487
q12	829	575	632	575
q13	11412	3178	3205	3178
q14	315	294	285	285
q15	581	531	537	531
q16	707	644	657	644
q17	2052	1755	1700	1700
q18	8328	7897	7608	7608
q19	1871	1473	1645	1473
q20	2199	1871	1866	1866
q21	5821	5493	5457	5457
q22	1188	1093	1014	1014
Total cold run time: 74564 ms
Total hot run time: 62080 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191992 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c23ba1abe59c58a131a71c835989c91e24eae59c, data reload: false

query1	877	409	416	409
query2	6250	2062	2027	2027
query3	8687	194	206	194
query4	33555	23523	23446	23446
query5	3994	479	481	479
query6	291	167	173	167
query7	4204	302	307	302
query8	271	211	215	211
query9	9288	2711	2712	2711
query10	471	295	293	293
query11	17828	15163	15206	15163
query12	152	100	99	99
query13	1573	487	429	429
query14	9857	7534	7495	7495
query15	265	172	176	172
query16	8062	505	486	486
query17	1640	638	586	586
query18	2172	312	323	312
query19	386	159	155	155
query20	120	113	114	113
query21	216	114	106	106
query22	4896	4543	4407	4407
query23	35038	33964	33912	33912
query24	11004	2857	2838	2838
query25	611	415	398	398
query26	1152	160	164	160
query27	2257	307	302	302
query28	7416	2442	2414	2414
query29	845	449	437	437
query30	254	152	149	149
query31	1060	812	823	812
query32	98	57	56	56
query33	774	298	310	298
query34	916	504	538	504
query35	913	725	731	725
query36	1099	931	955	931
query37	149	83	87	83
query38	4003	3995	3995	3995
query39	1455	1390	1430	1390
query40	209	98	99	98
query41	47	43	43	43
query42	113	96	100	96
query43	530	480	472	472
query44	1250	824	812	812
query45	198	164	168	164
query46	1133	714	748	714
query47	1937	1823	1837	1823
query48	429	366	348	348
query49	921	413	412	412
query50	832	409	418	409
query51	6985	6963	6882	6882
query52	103	88	91	88
query53	254	182	180	180
query54	1203	484	474	474
query55	84	78	76	76
query56	290	261	271	261
query57	1268	1156	1160	1156
query58	237	228	237	228
query59	3089	2981	3074	2981
query60	308	281	257	257
query61	111	120	107	107
query62	879	672	662	662
query63	226	183	184	183
query64	4039	650	619	619
query65	3274	3244	3253	3244
query66	847	296	309	296
query67	15783	15489	15603	15489
query68	4593	568	566	566
query69	535	334	301	301
query70	1198	1115	1127	1115
query71	361	277	278	277
query72	7175	3922	4014	3922
query73	781	345	357	345
query74	10404	9019	8945	8945
query75	3437	2667	2661	2661
query76	2974	883	920	883
query77	409	297	295	295
query78	10560	9597	9433	9433
query79	2208	600	605	600
query80	1217	463	480	463
query81	594	238	240	238
query82	694	140	138	138
query83	267	136	135	135
query84	252	72	83	72
query85	1367	302	292	292
query86	376	289	288	288
query87	4560	4384	4267	4267
query88	3975	2438	2414	2414
query89	412	288	297	288
query90	2031	191	186	186
query91	139	108	119	108
query92	66	45	49	45
query93	1586	546	542	542
query94	1101	291	300	291
query95	366	249	251	249
query96	612	283	279	279
query97	3289	3104	3137	3104
query98	211	201	198	198
query99	1542	1312	1317	1312
Total cold run time: 299970 ms
Total hot run time: 191992 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.2 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c23ba1abe59c58a131a71c835989c91e24eae59c, data reload: false

query1	0.05	0.04	0.04
query2	0.06	0.03	0.02
query3	0.24	0.06	0.06
query4	1.64	0.10	0.10
query5	0.50	0.52	0.53
query6	1.12	0.72	0.72
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.57	0.52	0.50
query10	0.56	0.55	0.52
query11	0.14	0.10	0.11
query12	0.14	0.12	0.11
query13	0.61	0.60	0.59
query14	2.73	2.72	2.69
query15	0.90	0.83	0.82
query16	0.39	0.40	0.38
query17	1.06	1.03	1.07
query18	0.20	0.20	0.20
query19	1.95	1.86	1.95
query20	0.01	0.01	0.01
query21	15.48	0.63	0.60
query22	2.83	1.87	1.78
query23	17.03	0.96	0.89
query24	2.91	1.45	0.47
query25	0.23	0.10	0.04
query26	0.50	0.14	0.14
query27	0.04	0.03	0.04
query28	10.73	1.09	1.07
query29	12.56	3.30	3.26
query30	0.26	0.06	0.06
query31	2.88	0.39	0.38
query32	3.27	0.48	0.46
query33	2.97	3.01	3.01
query34	16.72	4.50	4.50
query35	4.47	4.49	4.43
query36	0.67	0.49	0.47
query37	0.08	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.02	0.03
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.04	0.02
Total cold run time: 106.94 s
Total hot run time: 32.2 s

Copy link
Contributor

@amorynan amorynan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

github-actions bot commented Oct 8, 2024

PR approved by anyone and no changes requested.

Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 9, 2024
Copy link
Contributor

github-actions bot commented Oct 9, 2024

PR approved by at least one committer and no changes requested.

@airborne12 airborne12 merged commit bc9b87d into apache:master Oct 9, 2024
24 of 29 checks passed
@airborne12 airborne12 deleted the fix-array branch October 9, 2024 01:28
dataroaring pushed a commit that referenced this pull request Oct 10, 2024
## Proposed changes
feature for array overlaps
backports: #41161 
#41495
Issue Number: close #xxx
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Oct 10, 2024
…apache#41495)

## Proposed changes

Fix problem "Runtime Error: Null pointer passed to
'StringTypeInvertedIndexReader::query', which requires a non-null
argument."
```

Stack Trace:
#0 doris::segment_v2::StringTypeInvertedIndexReader::query(...) inverted_index_reader.cpp:473
apache#1 doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(...) inverted_index_reader.cpp:1237
apache#2 doris::vectorized::FunctionArraysOverlap::evaluate_inverted_index(...) function_arrays_overlap.h:192
apache#3 doris::vectorized::DefaultFunction::evaluate_inverted_index(...) function.h:532
apache#4 doris::vectorized::VExpr::_evaluate_inverted_index(...) vexpr.cpp:708
apache#5 doris::vectorized::VectorizedFnCall::evaluate_inverted_index(...) vectorized_fn_call.cpp:143
apache#6 doris::vectorized::VExprContext::evaluate_inverted_index(...) vexpr_context.cpp:126

```
cjj2010 pushed a commit to cjj2010/doris that referenced this pull request Oct 12, 2024
…apache#41495)

## Proposed changes

Fix problem "Runtime Error: Null pointer passed to
'StringTypeInvertedIndexReader::query', which requires a non-null
argument."
```

Stack Trace:
#0 doris::segment_v2::StringTypeInvertedIndexReader::query(...) inverted_index_reader.cpp:473
apache#1 doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(...) inverted_index_reader.cpp:1237
apache#2 doris::vectorized::FunctionArraysOverlap::evaluate_inverted_index(...) function_arrays_overlap.h:192
apache#3 doris::vectorized::DefaultFunction::evaluate_inverted_index(...) function.h:532
apache#4 doris::vectorized::VExpr::_evaluate_inverted_index(...) vexpr.cpp:708
apache#5 doris::vectorized::VectorizedFnCall::evaluate_inverted_index(...) vectorized_fn_call.cpp:143
apache#6 doris::vectorized::VExprContext::evaluate_inverted_index(...) vexpr_context.cpp:126

```
amorynan pushed a commit to amorynan/doris that referenced this pull request Oct 12, 2024
…apache#41495)

## Proposed changes

Fix problem "Runtime Error: Null pointer passed to
'StringTypeInvertedIndexReader::query', which requires a non-null
argument."
```

Stack Trace:
#0 doris::segment_v2::StringTypeInvertedIndexReader::query(...) inverted_index_reader.cpp:473
#1 doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(...) inverted_index_reader.cpp:1237
apache#2 doris::vectorized::FunctionArraysOverlap::evaluate_inverted_index(...) function_arrays_overlap.h:192
apache#3 doris::vectorized::DefaultFunction::evaluate_inverted_index(...) function.h:532
apache#4 doris::vectorized::VExpr::_evaluate_inverted_index(...) vexpr.cpp:708
apache#5 doris::vectorized::VectorizedFnCall::evaluate_inverted_index(...) vectorized_fn_call.cpp:143
apache#6 doris::vectorized::VExprContext::evaluate_inverted_index(...) vexpr_context.cpp:126

```
yiguolei pushed a commit that referenced this pull request Oct 18, 2024
)

## Proposed changes
backport : #41286
#41495
Issue Number: close #xxx

<!--Describe your changes.-->
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.3-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants