Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improvement](statistics)Support show column partition update rows info. #37124

Merged
merged 1 commit into from
Jul 2, 2024
Merged

[improvement](statistics)Support show column partition update rows info. #37124

merged 1 commit into from
Jul 2, 2024

Conversation

Jibing-Li
Copy link
Contributor

Support show column update rows of each partition. This may help when investigate online issues.

mysql> show table stats part2 partition(p1, p2) (id, colint);
+------------+-------------+----------------+--------------+
| index_name | column_name | partition_name | updated_rows |
+------------+-------------+----------------+--------------+
| part2      | id          | p1             | 6            |
| part2      | id          | p2             | 6            |
| part2      | colint      | p1             | 6            |
| part2      | colint      | p2             | 6            |
+------------+-------------+----------------+--------------+
4 rows in set (0.01 sec)

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Jibing-Li Jibing-Li marked this pull request as ready for review July 2, 2024 06:39
@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40186 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f3934830194c7ff53b722cfb0ce3c50300865f55, data reload: false

------ Round 1 ----------------------------------
q1	17974	4673	4455	4455
q2	2666	197	196	196
q3	11162	1273	1129	1129
q4	10756	952	800	800
q5	7696	2773	2720	2720
q6	247	144	141	141
q7	977	617	627	617
q8	9285	2120	2115	2115
q9	9027	6570	6516	6516
q10	8982	3776	3708	3708
q11	474	246	229	229
q12	429	240	232	232
q13	17753	3005	2978	2978
q14	267	226	216	216
q15	527	482	507	482
q16	507	376	372	372
q17	992	663	731	663
q18	8045	7443	7348	7348
q19	6526	1574	1461	1461
q20	665	315	323	315
q21	4989	3157	3861	3157
q22	411	345	336	336
Total cold run time: 120357 ms
Total hot run time: 40186 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4410	4288	4247	4247
q2	366	265	262	262
q3	2989	2738	2695	2695
q4	1931	1597	1543	1543
q5	5250	5327	5299	5299
q6	221	138	135	135
q7	2064	1695	1766	1695
q8	3213	3356	3327	3327
q9	8391	8382	8387	8382
q10	3908	3658	3636	3636
q11	571	483	505	483
q12	815	602	588	588
q13	16321	2974	2994	2974
q14	280	273	259	259
q15	531	482	487	482
q16	473	417	407	407
q17	1802	1476	1490	1476
q18	7610	7426	7416	7416
q19	1703	1495	1623	1495
q20	1997	1794	1770	1770
q21	4882	4644	4718	4644
q22	647	575	542	542
Total cold run time: 70375 ms
Total hot run time: 53757 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 170152 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f3934830194c7ff53b722cfb0ce3c50300865f55, data reload: false

query1	928	375	367	367
query2	6457	2484	2496	2484
query3	6663	212	213	212
query4	19063	17556	17307	17307
query5	4192	474	478	474
query6	303	167	174	167
query7	4603	294	299	294
query8	300	290	297	290
query9	8452	2342	2335	2335
query10	605	304	280	280
query11	10505	10029	10214	10029
query12	135	90	87	87
query13	1661	387	365	365
query14	9325	6665	6787	6665
query15	223	195	185	185
query16	7814	279	273	273
query17	1763	550	535	535
query18	1885	278	281	278
query19	204	150	158	150
query20	91	81	85	81
query21	217	135	129	129
query22	4307	4109	3973	3973
query23	33776	33082	33066	33066
query24	11355	2894	2852	2852
query25	674	384	384	384
query26	1670	157	157	157
query27	3003	319	324	319
query28	7602	2062	2048	2048
query29	1045	653	634	634
query30	277	151	154	151
query31	963	723	740	723
query32	101	55	54	54
query33	784	308	301	301
query34	925	469	479	469
query35	736	628	610	610
query36	1060	938	906	906
query37	154	80	82	80
query38	2887	2747	2743	2743
query39	894	815	816	815
query40	280	128	128	128
query41	58	55	55	55
query42	119	100	99	99
query43	597	569	529	529
query44	1207	721	721	721
query45	194	162	163	162
query46	1082	741	728	728
query47	1865	1783	1772	1772
query48	376	304	292	292
query49	1189	432	427	427
query50	776	385	384	384
query51	6896	6853	6805	6805
query52	111	91	97	91
query53	358	289	293	289
query54	1034	452	443	443
query55	75	83	73	73
query56	307	281	281	281
query57	1170	1048	1075	1048
query58	266	251	268	251
query59	3242	3171	3091	3091
query60	304	271	271	271
query61	93	95	94	94
query62	654	443	473	443
query63	318	290	294	290
query64	9874	2234	1745	1745
query65	3194	3125	3116	3116
query66	1359	339	338	338
query67	15713	14990	15056	14990
query68	8369	549	554	549
query69	704	493	339	339
query70	1161	1090	1048	1048
query71	532	273	282	273
query72	9024	5201	2778	2778
query73	1060	321	325	321
query74	5907	5540	5512	5512
query75	5171	2632	2663	2632
query76	5128	902	915	902
query77	752	311	297	297
query78	10575	9851	9684	9684
query79	8498	514	508	508
query80	997	479	461	461
query81	541	222	217	217
query82	746	107	106	106
query83	359	175	167	167
query84	322	85	84	84
query85	1397	289	272	272
query86	415	305	306	305
query87	3302	3104	3106	3104
query88	4411	2361	2369	2361
query89	525	381	377	377
query90	1994	181	179	179
query91	125	99	99	99
query92	65	50	47	47
query93	6934	509	498	498
query94	1223	184	219	184
query95	403	307	312	307
query96	600	267	263	263
query97	3198	3034	3014	3014
query98	242	202	198	198
query99	1149	838	858	838
Total cold run time: 296142 ms
Total hot run time: 170152 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.44 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f3934830194c7ff53b722cfb0ce3c50300865f55, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.05
query4	1.66	0.08	0.09
query5	0.51	0.51	0.48
query6	1.14	0.74	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.54	0.54	0.53
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.61	0.58	0.58
query14	0.84	0.78	0.78
query15	0.85	0.81	0.82
query16	0.36	0.36	0.36
query17	0.94	0.97	0.95
query18	0.23	0.23	0.27
query19	1.77	1.71	1.70
query20	0.02	0.00	0.00
query21	15.45	0.76	0.66
query22	4.30	7.54	1.77
query23	18.25	1.40	1.28
query24	2.14	0.24	0.22
query25	0.15	0.09	0.08
query26	0.27	0.18	0.18
query27	0.08	0.08	0.08
query28	13.28	1.01	1.02
query29	12.59	3.32	3.30
query30	0.25	0.06	0.06
query31	2.88	0.39	0.39
query32	3.25	0.47	0.46
query33	2.90	2.89	2.88
query34	17.21	4.40	4.46
query35	4.49	4.52	4.51
query36	0.65	0.46	0.49
query37	0.19	0.15	0.16
query38	0.16	0.15	0.15
query39	0.04	0.03	0.04
query40	0.18	0.14	0.14
query41	0.10	0.05	0.05
query42	0.06	0.04	0.05
query43	0.05	0.04	0.03
Total cold run time: 109.67 s
Total hot run time: 30.44 s

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 2, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 2, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 2, 2024

PR approved by anyone and no changes requested.

morningman
morningman approved these changes Jul 2, 2024
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Jibing-Li Jibing-Li merged commit 80f6583 into apache:master Jul 2, 2024
27 of 30 checks passed
@Jibing-Li Jibing-Li deleted the show branch July 3, 2024 02:27
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
@Jibing-Li @dataroaring
[improvement](statistics)Support show column partition update rows in…
0463e86 
…fo. (#37124)

Support show column update rows of each partition. This may help when
investigate online issues.
```
mysql> show table stats part2 partition(p1, p2) (id, colint);
+------------+-------------+----------------+--------------+
| index_name | column_name | partition_name | updated_rows |
+------------+-------------+----------------+--------------+
| part2      | id          | p1             | 6            |
| part2      | id          | p2             | 6            |
| part2      | colint      | p1             | 6            |
| part2      | colint      | p2             | 6            |
+------------+-------------+----------------+--------------+
4 rows in set (0.01 sec)
```
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrySnow morrySnow morrySnow approved these changes

@morningman morningman morningman approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.1-merged not-merge/2.1 reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Jibing-Li @doris-robot @morningman @morrySnow @dataroaring