Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](inverted index) disable range query in StringTypeInvertedIndexReader #38218

Merged
merged 3 commits into from
Jul 25, 2024
Merged

[fix](inverted index) disable range query in StringTypeInvertedIndexReader #38218

merged 3 commits into from
Jul 25, 2024

Conversation

csun5285
Copy link
Contributor

Proposed changes

disable range query in StringTypeInvertedIndexReader

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@csun5285
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@csun5285
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39864 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 589fa8be3696a6f010f28a1dd89ce0a936dfa940, data reload: false

------ Round 1 ----------------------------------
q1	8366	4903	4256	4256
q2	1318	193	192	192
q3	6634	1100	1084	1084
q4	5246	741	814	741
q5	3331	2779	2657	2657
q6	222	140	135	135
q7	946	606	602	602
q8	3939	2040	2113	2040
q9	6702	6575	6528	6528
q10	4285	3771	3802	3771
q11	384	227	235	227
q12	382	225	217	217
q13	6748	3028	2985	2985
q14	283	231	229	229
q15	532	483	467	467
q16	483	374	385	374
q17	946	742	729	729
q18	8098	7560	7522	7522
q19	3460	1410	1376	1376
q20	674	319	331	319
q21	4852	3136	3256	3136
q22	348	291	277	277
Total cold run time: 68179 ms
Total hot run time: 39864 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4243	4228	4201	4201
q2	394	268	270	268
q3	3023	2821	2785	2785
q4	1864	1580	1639	1580
q5	5309	5281	5327	5281
q6	213	128	130	128
q7	2125	1723	1727	1723
q8	3190	3325	3289	3289
q9	8407	8366	8400	8366
q10	3868	3708	3685	3685
q11	580	488	490	488
q12	757	587	650	587
q13	5243	3004	2989	2989
q14	322	277	268	268
q15	516	486	475	475
q16	466	416	412	412
q17	1754	1459	1458	1458
q18	7619	7594	7357	7357
q19	1662	1394	1500	1394
q20	1971	1810	1780	1780
q21	4786	4552	4835	4552
q22	586	479	501	479
Total cold run time: 58898 ms
Total hot run time: 53545 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173994 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 589fa8be3696a6f010f28a1dd89ce0a936dfa940, data reload: false

query1	925	374	370	370
query2	6445	2019	1886	1886
query3	6670	205	212	205
query4	28464	17505	17179	17179
query5	4268	492	497	492
query6	285	182	168	168
query7	4602	306	292	292
query8	244	200	201	200
query9	8515	2477	2473	2473
query10	451	294	298	294
query11	11911	10058	9934	9934
query12	135	89	82	82
query13	1656	387	366	366
query14	10328	7510	7765	7510
query15	233	161	164	161
query16	7808	463	398	398
query17	1568	546	522	522
query18	1955	271	279	271
query19	209	150	149	149
query20	92	87	84	84
query21	211	126	126	126
query22	4404	4159	4119	4119
query23	33915	33317	33096	33096
query24	11762	2803	2900	2803
query25	674	363	370	363
query26	1842	156	159	156
query27	2931	277	282	277
query28	7674	2028	1989	1989
query29	1162	625	614	614
query30	285	152	148	148
query31	958	736	746	736
query32	95	52	54	52
query33	769	325	356	325
query34	928	490	511	490
query35	847	731	721	721
query36	1110	912	936	912
query37	194	80	79	79
query38	2864	2765	2739	2739
query39	903	805	808	805
query40	268	116	119	116
query41	49	44	45	44
query42	123	101	109	101
query43	548	475	499	475
query44	1197	719	716	716
query45	191	164	162	162
query46	1100	747	700	700
query47	1851	1805	1821	1805
query48	396	311	290	290
query49	1193	408	412	408
query50	788	407	412	407
query51	6682	6730	6752	6730
query52	101	93	100	93
query53	363	287	292	287
query54	910	460	453	453
query55	75	77	75	75
query56	298	279	269	269
query57	1182	1026	1037	1026
query58	269	244	270	244
query59	2959	2850	2738	2738
query60	294	282	292	282
query61	97	96	93	93
query62	831	650	654	650
query63	318	292	293	292
query64	10600	2201	1630	1630
query65	3158	3057	3083	3057
query66	1368	333	335	333
query67	15410	15168	15030	15030
query68	4603	538	538	538
query69	467	338	352	338
query70	1136	1088	1151	1088
query71	392	282	282	282
query72	7001	5701	5620	5620
query73	747	338	332	332
query74	5961	5776	5695	5695
query75	3449	2658	2726	2658
query76	2520	957	892	892
query77	506	314	319	314
query78	10592	10204	9776	9776
query79	2825	529	538	529
query80	1651	479	491	479
query81	566	223	218	218
query82	632	136	142	136
query83	315	170	167	167
query84	276	92	91	91
query85	1784	337	297	297
query86	469	325	302	302
query87	3276	3115	3122	3115
query88	4144	2463	2466	2463
query89	489	402	388	388
query90	1834	194	195	194
query91	128	97	100	97
query92	61	53	49	49
query93	3566	500	494	494
query94	991	287	298	287
query95	408	325	331	325
query96	606	284	276	276
query97	3252	3025	3081	3025
query98	212	211	195	195
query99	1667	1283	1242	1242
Total cold run time: 289437 ms
Total hot run time: 173994 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.26 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 589fa8be3696a6f010f28a1dd89ce0a936dfa940, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.06	0.05
query4	1.68	0.08	0.08
query5	0.50	0.50	0.50
query6	1.13	0.74	0.72
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.55	0.49	0.50
query10	0.55	0.54	0.55
query11	0.16	0.12	0.11
query12	0.14	0.12	0.13
query13	0.61	0.58	0.57
query14	0.76	0.78	0.77
query15	0.85	0.81	0.82
query16	0.35	0.35	0.36
query17	0.99	1.00	0.98
query18	0.22	0.21	0.22
query19	1.73	1.73	1.68
query20	0.01	0.01	0.01
query21	15.39	0.77	0.66
query22	3.82	6.62	2.53
query23	18.29	1.35	1.22
query24	2.15	0.22	0.22
query25	0.16	0.08	0.08
query26	0.28	0.20	0.21
query27	0.45	0.24	0.23
query28	13.24	1.01	1.00
query29	12.62	3.37	3.36
query30	0.25	0.06	0.05
query31	2.88	0.40	0.39
query32	3.26	0.47	0.47
query33	2.88	2.94	2.90
query34	17.07	4.37	4.35
query35	4.40	4.46	4.44
query36	0.65	0.47	0.48
query37	0.19	0.15	0.16
query38	0.16	0.15	0.15
query39	0.05	0.03	0.04
query40	0.16	0.12	0.12
query41	0.10	0.04	0.05
query42	0.05	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.18 s
Total hot run time: 31.26 s

xiaokang
xiaokang previously approved these changes Jul 23, 2024
View reviewed changes
Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 23, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 23, 2024
@csun5285
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39814 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d216ad1dca9e673a821f178c61a4200e5dcf6c86, data reload: false

------ Round 1 ----------------------------------
q1	17920	5355	4364	4364
q2	2337	189	188	188
q3	10473	1240	1040	1040
q4	10187	802	757	757
q5	7556	2733	2669	2669
q6	226	152	140	140
q7	951	597	600	597
q8	9226	2060	2088	2060
q9	8676	6545	6566	6545
q10	8808	3850	3806	3806
q11	456	230	241	230
q12	396	224	229	224
q13	18880	2980	3009	2980
q14	277	239	260	239
q15	523	469	480	469
q16	486	404	373	373
q17	944	607	713	607
q18	8088	7506	7400	7400
q19	5851	1448	1429	1429
q20	704	329	326	326
q21	4916	3118	3092	3092
q22	356	294	279	279
Total cold run time: 118237 ms
Total hot run time: 39814 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4405	4234	4349	4234
q2	383	260	263	260
q3	3040	2871	2734	2734
q4	1915	1641	1643	1641
q5	5312	5320	5315	5315
q6	220	136	141	136
q7	2156	1798	1732	1732
q8	3198	3355	3306	3306
q9	8477	8431	8392	8392
q10	3911	3713	3753	3713
q11	592	488	484	484
q12	765	588	596	588
q13	16594	2986	3015	2986
q14	324	268	269	268
q15	509	482	469	469
q16	467	423	408	408
q17	1808	1504	1464	1464
q18	7724	7600	7487	7487
q19	1695	1521	1481	1481
q20	1997	1790	1792	1790
q21	5008	4912	4653	4653
q22	540	519	509	509
Total cold run time: 71040 ms
Total hot run time: 54050 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173747 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d216ad1dca9e673a821f178c61a4200e5dcf6c86, data reload: false

query1	912	370	363	363
query2	6433	1869	1759	1759
query3	6648	205	217	205
query4	28477	17398	17540	17398
query5	4197	485	484	484
query6	282	191	157	157
query7	4600	307	285	285
query8	243	195	195	195
query9	8435	2464	2417	2417
query10	440	263	277	263
query11	11482	10237	10157	10157
query12	135	85	84	84
query13	1634	377	377	377
query14	10345	8101	7777	7777
query15	226	178	169	169
query16	7784	480	505	480
query17	1593	581	550	550
query18	1879	301	285	285
query19	208	189	148	148
query20	90	87	84	84
query21	212	131	122	122
query22	4325	4145	3989	3989
query23	33903	33180	33344	33180
query24	12064	2810	2819	2810
query25	674	364	369	364
query26	1805	151	152	151
query27	2946	272	268	268
query28	7599	2041	2024	2024
query29	1034	619	613	613
query30	287	147	161	147
query31	964	766	733	733
query32	91	55	53	53
query33	766	323	344	323
query34	897	486	495	486
query35	864	742	717	717
query36	1117	948	921	921
query37	253	78	80	78
query38	2871	2763	2782	2763
query39	888	827	831	827
query40	279	119	120	119
query41	52	46	45	45
query42	116	97	103	97
query43	498	458	448	448
query44	1114	733	729	729
query45	190	165	160	160
query46	1080	720	720	720
query47	1861	1776	1772	1772
query48	371	294	289	289
query49	1209	410	410	410
query50	778	401	390	390
query51	6815	6830	6695	6695
query52	104	93	94	93
query53	357	287	292	287
query54	1041	447	454	447
query55	75	77	72	72
query56	297	273	271	271
query57	1138	1020	1063	1020
query58	274	241	245	241
query59	2897	2660	2502	2502
query60	294	275	274	274
query61	97	94	94	94
query62	837	665	648	648
query63	328	312	289	289
query64	10555	2226	1668	1668
query65	3185	3167	3131	3131
query66	1371	340	330	330
query67	15602	15244	14990	14990
query68	6051	558	553	553
query69	663	433	360	360
query70	1196	1102	1076	1076
query71	473	282	280	280
query72	7967	5440	5279	5279
query73	777	331	329	329
query74	6158	5643	5671	5643
query75	3775	2717	2696	2696
query76	3719	960	967	960
query77	683	329	323	323
query78	12880	9429	9880	9429
query79	3466	522	530	522
query80	1199	476	481	476
query81	565	220	223	220
query82	878	139	133	133
query83	197	173	176	173
query84	278	88	92	88
query85	731	341	313	313
query86	470	317	320	317
query87	3294	3165	3174	3165
query88	4379	2389	2366	2366
query89	490	391	394	391
query90	1904	200	193	193
query91	126	103	100	100
query92	67	54	49	49
query93	2934	521	519	519
query94	1138	300	301	300
query95	412	331	327	327
query96	603	270	274	270
query97	3152	3071	2985	2985
query98	231	205	250	205
query99	1769	1270	1276	1270
Total cold run time: 295043 ms
Total hot run time: 173747 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.3 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d216ad1dca9e673a821f178c61a4200e5dcf6c86, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.05	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.49	0.48	0.49
query6	1.16	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.56	0.49	0.50
query10	0.55	0.56	0.55
query11	0.15	0.12	0.11
query12	0.15	0.13	0.12
query13	0.58	0.58	0.59
query14	0.77	0.77	0.82
query15	0.85	0.82	0.81
query16	0.36	0.37	0.38
query17	1.06	1.03	0.97
query18	0.23	0.23	0.23
query19	1.85	1.70	1.73
query20	0.01	0.01	0.01
query21	15.44	0.78	0.67
query22	4.49	6.96	2.37
query23	18.24	1.37	1.36
query24	2.13	0.22	0.21
query25	0.17	0.10	0.09
query26	0.29	0.20	0.20
query27	0.45	0.24	0.23
query28	13.26	1.03	1.00
query29	12.64	3.34	3.28
query30	0.27	0.07	0.05
query31	2.87	0.39	0.39
query32	3.26	0.48	0.48
query33	2.91	2.96	2.90
query34	17.15	4.46	4.40
query35	4.48	4.44	4.46
query36	0.64	0.47	0.46
query37	0.18	0.16	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.03
query40	0.15	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.29 s
Total hot run time: 31.3 s

airborne12
airborne12 approved these changes Jul 25, 2024
View reviewed changes
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 25, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

zzzxl1993
zzzxl1993 approved these changes Jul 25, 2024
View reviewed changes
Copy link
Contributor

@zzzxl1993 zzzxl1993 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@airborne12 airborne12 merged commit 00fca3e into apache:master Jul 25, 2024
30 of 34 checks passed
@xiaokang xiaokang added the p0_b label Jul 29, 2024
dataroaring pushed a commit that referenced this pull request Jul 31, 2024
@csun5285 @dataroaring
[fix](inverted index) disable range query in StringTypeInvertedIndexR…
2e7fffb 
…eader (#38218)

## Proposed changes
disable range query in StringTypeInvertedIndexReader
@csun5285 csun5285 mentioned this pull request Aug 1, 2024
Merged
xiaokang pushed a commit that referenced this pull request Aug 1, 2024
@csun5285
[fix](inverted index) disable range query in StringTypeInvertedIndexR…
5cb03e9 
…eader (#38218) (#38722)
@csun5285 csun5285 mentioned this pull request Aug 6, 2024
Merged
yiguolei pushed a commit that referenced this pull request Aug 7, 2024
@csun5285
[fix](inverted index) disable range query in StringTypeInvertedIndexR…
b856530 
…eader (#38218)  (#38926)

## Proposed changes

pick from master #38218

Issue Number: close #xxx

<!--Describe your changes.-->
@xiaokang xiaokang removed the doing label Aug 12, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@csun5285 @Baymine
[fix](inverted index) disable range query in StringTypeInvertedIndexR…
c09cc08 
…eader (apache#38218) (apache#38722)
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zzzxl1993 zzzxl1993 zzzxl1993 approved these changes

@airborne12 airborne12 airborne12 approved these changes

@xiaokang xiaokang xiaokang approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.14-merged dev/2.1.6-merged dev/3.0.1-merged p0_b reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@csun5285 @doris-robot @xiaokang @airborne12 @zzzxl1993 @yiguolei @dataroaring