Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhance](mtmv)support replace materialized view #36749

Merged
merged 11 commits into from
Jul 2, 2024

Conversation

zddr
Copy link
Contributor

@zddr zddr commented Jun 24, 2024

  • support alter MATERIALIZED VIEW mv1 replace with MATERIALIZED VIEW mv2 PROPERTIES('swap' = 'false');
  • fix rename MATERIALIZED VIEW not change state of high level MATERIALIZED VIEW

Logic is the same as: https://doris.apache.org/zh-CN/docs/dev/sql-manual/sql-statements/Data-Definition-Statements/Alter/ALTER-TABLE-REPLACE/

The difference is that this statement can only operate on materialized views

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zddr
Copy link
Contributor Author

zddr commented Jun 24, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40179 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d0156a6f49ef89d6771bf1c501b52441224ce051, data reload: false

------ Round 1 ----------------------------------
q1	17668	4440	4321	4321
q2	2014	206	195	195
q3	10513	1135	1165	1135
q4	10234	810	767	767
q5	7486	2685	2760	2685
q6	221	135	135	135
q7	956	599	624	599
q8	9233	2081	2080	2080
q9	8974	6515	6510	6510
q10	9000	3733	3729	3729
q11	470	244	245	244
q12	467	233	234	233
q13	18120	3033	3010	3010
q14	260	242	235	235
q15	515	460	478	460
q16	490	380	388	380
q17	978	639	689	639
q18	8170	7557	7501	7501
q19	5175	1576	1505	1505
q20	667	321	336	321
q21	4983	3155	3905	3155
q22	414	340	343	340
Total cold run time: 117008 ms
Total hot run time: 40179 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4411	4225	4203	4203
q2	369	258	266	258
q3	2990	2965	2899	2899
q4	1971	1741	1700	1700
q5	5567	5509	5511	5509
q6	222	132	133	132
q7	2199	1875	1834	1834
q8	3297	3438	3449	3438
q9	8774	8769	8831	8769
q10	4162	3918	3633	3633
q11	631	508	511	508
q12	848	637	625	625
q13	15819	3235	3150	3150
q14	312	284	261	261
q15	528	474	486	474
q16	497	430	439	430
q17	1880	1506	1496	1496
q18	8213	7987	7864	7864
q19	1796	1642	1547	1547
q20	2125	1905	1904	1904
q21	5135	4987	4973	4973
q22	640	564	574	564
Total cold run time: 72386 ms
Total hot run time: 56171 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173914 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d0156a6f49ef89d6771bf1c501b52441224ce051, data reload: false

query1	914	386	378	378
query2	7502	2300	2330	2300
query3	6620	221	221	221
query4	19159	17428	17376	17376
query5	3607	471	462	462
query6	236	163	163	163
query7	4575	299	289	289
query8	306	288	282	282
query9	8531	2492	2462	2462
query10	563	299	280	280
query11	10528	10193	10198	10193
query12	118	91	87	87
query13	1643	384	385	384
query14	9498	6730	7433	6730
query15	234	185	190	185
query16	7788	263	264	263
query17	1929	528	506	506
query18	1898	272	265	265
query19	191	151	170	151
query20	86	81	78	78
query21	204	124	123	123
query22	4436	4097	3999	3999
query23	34030	33716	33669	33669
query24	10716	2861	2891	2861
query25	599	378	374	374
query26	719	158	156	156
query27	2263	333	324	324
query28	6360	2186	2200	2186
query29	889	620	644	620
query30	254	155	154	154
query31	963	768	756	756
query32	98	53	55	53
query33	674	279	283	279
query34	930	504	473	473
query35	751	624	645	624
query36	1153	935	990	935
query37	142	75	72	72
query38	2956	2884	2828	2828
query39	919	846	875	846
query40	218	128	122	122
query41	56	49	53	49
query42	119	103	98	98
query43	570	534	522	522
query44	1106	750	722	722
query45	198	165	168	165
query46	1053	702	717	702
query47	1887	1795	1749	1749
query48	372	313	303	303
query49	862	404	412	404
query50	783	385	398	385
query51	6891	6675	6767	6675
query52	107	90	90	90
query53	360	282	291	282
query54	877	451	437	437
query55	73	73	73	73
query56	306	256	283	256
query57	1173	1040	1078	1040
query58	242	250	244	244
query59	3395	3260	3382	3260
query60	287	278	271	271
query61	92	107	109	107
query62	605	436	436	436
query63	316	289	299	289
query64	8614	2233	1728	1728
query65	3158	3110	3088	3088
query66	745	322	331	322
query67	15465	14981	14954	14954
query68	4506	540	542	540
query69	452	313	304	304
query70	1193	1099	1125	1099
query71	389	277	269	269
query72	7078	5641	5430	5430
query73	745	337	323	323
query74	6156	5631	5449	5449
query75	3355	2648	2685	2648
query76	2527	978	914	914
query77	428	305	288	288
query78	10404	9805	9773	9773
query79	2716	514	515	514
query80	1145	459	456	456
query81	594	219	221	219
query82	1093	109	101	101
query83	235	170	168	168
query84	234	88	87	87
query85	1143	283	267	267
query86	462	311	299	299
query87	3317	3113	3085	3085
query88	4313	2427	2427	2427
query89	469	391	388	388
query90	1671	187	184	184
query91	127	102	101	101
query92	57	47	49	47
query93	1912	514	519	514
query94	1123	196	187	187
query95	407	326	337	326
query96	612	274	270	270
query97	3212	3043	3060	3043
query98	221	209	204	204
query99	1299	830	830	830
Total cold run time: 268507 ms
Total hot run time: 173914 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.96 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d0156a6f49ef89d6771bf1c501b52441224ce051, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.05
query3	0.21	0.05	0.05
query4	1.69	0.07	0.07
query5	0.52	0.48	0.50
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.55	0.49	0.50
query10	0.55	0.54	0.54
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.59	0.59	0.59
query14	0.75	0.77	0.79
query15	0.82	0.82	0.81
query16	0.36	0.37	0.35
query17	1.01	1.03	1.04
query18	0.22	0.23	0.25
query19	1.81	1.72	1.76
query20	0.02	0.01	0.01
query21	15.40	0.67	0.65
query22	4.02	6.05	3.05
query23	18.31	1.43	1.36
query24	2.17	0.23	0.22
query25	0.15	0.08	0.08
query26	0.27	0.17	0.17
query27	0.08	0.07	0.08
query28	13.19	1.03	0.99
query29	12.65	3.30	3.28
query30	0.25	0.06	0.06
query31	2.86	0.40	0.39
query32	3.28	0.47	0.47
query33	2.87	2.94	2.91
query34	17.26	4.47	4.44
query35	4.52	4.56	4.56
query36	0.65	0.46	0.47
query37	0.18	0.15	0.15
query38	0.16	0.14	0.15
query39	0.04	0.04	0.03
query40	0.17	0.14	0.14
query41	0.10	0.05	0.05
query42	0.04	0.04	0.04
query43	0.05	0.03	0.04
Total cold run time: 109.38 s
Total hot run time: 31.96 s

@zddr
Copy link
Contributor Author

zddr commented Jun 25, 2024

run p0

Copy link
Contributor

@zfr9527 zfr9527 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LTGM

}

sql """
alter MATERIALIZED VIEW ${mvName1} replace with MATERIALIZED VIEW ${mvName2};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will an exception occur when this statement is executed while mtmv is being updated?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no,will waiting for the lock in very little time

Copy link
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 2, 2024
Copy link
Contributor

github-actions bot commented Jul 2, 2024

PR approved by at least one committer and no changes requested.

@morrySnow morrySnow merged commit 9223349 into apache:master Jul 2, 2024
33 of 35 checks passed
zddr added a commit to zddr/incubator-doris that referenced this pull request Jul 2, 2024
- support alter MATERIALIZED VIEW mv1 replace with MATERIALIZED VIEW mv2 PROPERTIES('swap' = 'false');
- fix rename MATERIALIZED VIEW not change state of high level MATERIALIZED VIEW

Logic is the same as ALTER-TABLE-REPLACE. more detail pelase read doris document.
The difference is that this statement can only operate on materialized views
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
- support alter MATERIALIZED VIEW mv1 replace with MATERIALIZED VIEW mv2 PROPERTIES('swap' = 'false');
- fix rename MATERIALIZED VIEW not change state of high level MATERIALIZED VIEW

Logic is the same as ALTER-TABLE-REPLACE. more detail pelase read doris document.
The difference is that this statement can only operate on materialized views
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants