Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature](audit) add new FE config to skip audit for certain user #38310

Merged
merged 1 commit into from
Jul 25, 2024

Conversation

morningman
Copy link
Contributor

@morningman morningman commented Jul 24, 2024

Sometime we don't want to audit operation from certain user in audit log or audit table.
Add a new FE config skip_audit_user_list.
Default is empty, which means all operations will be recorded.
When you want to ignore some user's operation, you can set this config like:

skip_audit_user_list=user1
--or
skip_audit_user_list=user1,user2

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39816 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 201925c11713cb7d52763d82b907ac9b4f272965, data reload: false

------ Round 1 ----------------------------------
q1	17636	4356	4282	4282
q2	2010	191	193	191
q3	10427	1269	1090	1090
q4	10188	746	761	746
q5	7539	2679	2648	2648
q6	223	138	140	138
q7	957	600	596	596
q8	9218	2071	2079	2071
q9	8716	6595	6592	6592
q10	8863	3743	3818	3743
q11	451	253	242	242
q12	474	225	228	225
q13	18007	3005	2993	2993
q14	284	232	238	232
q15	512	474	469	469
q16	502	392	374	374
q17	969	665	642	642
q18	7986	7446	7554	7446
q19	7573	1474	1352	1352
q20	672	329	306	306
q21	5037	3157	3317	3157
q22	351	291	281	281
Total cold run time: 118595 ms
Total hot run time: 39816 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4411	4268	4238	4238
q2	367	273	265	265
q3	2990	2929	2949	2929
q4	1984	1665	1724	1665
q5	5641	5543	5473	5473
q6	224	134	141	134
q7	2164	1873	1837	1837
q8	3233	3432	3469	3432
q9	8873	8756	8879	8756
q10	4108	3872	3769	3769
q11	616	507	505	505
q12	826	621	657	621
q13	16141	3158	3210	3158
q14	310	279	288	279
q15	539	471	494	471
q16	499	447	437	437
q17	1823	1479	1502	1479
q18	8189	8018	7982	7982
q19	1797	1612	1732	1612
q20	2961	1908	1859	1859
q21	9381	4855	4903	4855
q22	656	536	521	521
Total cold run time: 77733 ms
Total hot run time: 56277 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 175430 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 201925c11713cb7d52763d82b907ac9b4f272965, data reload: false

query1	914	395	372	372
query2	6434	1921	1868	1868
query3	6644	208	220	208
query4	23665	17618	17506	17506
query5	3607	496	480	480
query6	283	177	170	170
query7	4594	303	295	295
query8	239	193	200	193
query9	8457	2427	2407	2407
query10	438	291	284	284
query11	11410	10038	10140	10038
query12	120	85	88	85
query13	1642	366	366	366
query14	10232	7929	7104	7104
query15	213	164	168	164
query16	7207	500	467	467
query17	1098	566	538	538
query18	1793	292	276	276
query19	206	151	150	150
query20	91	82	82	82
query21	210	130	128	128
query22	4287	4058	4019	4019
query23	34227	33898	33884	33884
query24	9287	2935	2968	2935
query25	613	414	398	398
query26	701	160	156	156
query27	2253	285	285	285
query28	5734	2094	2095	2094
query29	927	716	660	660
query30	260	166	177	166
query31	1005	755	780	755
query32	99	54	54	54
query33	589	351	340	340
query34	908	507	529	507
query35	848	772	767	767
query36	1136	992	962	962
query37	148	84	82	82
query38	3024	2913	2813	2813
query39	897	878	843	843
query40	208	128	127	127
query41	49	45	49	45
query42	123	100	102	100
query43	507	492	470	470
query44	1097	734	738	734
query45	213	178	181	178
query46	1134	765	744	744
query47	1892	1754	1779	1754
query48	381	299	298	298
query49	848	437	452	437
query50	799	430	405	405
query51	6856	6803	6716	6716
query52	106	96	94	94
query53	376	305	294	294
query54	673	479	469	469
query55	78	78	78	78
query56	316	306	301	301
query57	1122	1055	1063	1055
query58	272	260	283	260
query59	2801	2646	2527	2527
query60	325	309	301	301
query61	120	117	120	117
query62	814	665	655	655
query63	334	305	301	301
query64	9228	2353	7207	2353
query65	3179	3112	3118	3112
query66	769	353	336	336
query67	15645	15368	15028	15028
query68	6283	575	547	547
query69	763	475	365	365
query70	1221	1163	1114	1114
query71	483	298	286	286
query72	8874	5780	5690	5690
query73	793	328	330	328
query74	6118	5623	5661	5623
query75	4365	2700	2710	2700
query76	3990	1027	932	932
query77	818	310	307	307
query78	9850	9067	9022	9022
query79	8393	536	529	529
query80	2593	479	486	479
query81	584	224	224	224
query82	1126	132	130	130
query83	292	172	175	172
query84	275	87	85	85
query85	1473	309	307	307
query86	434	290	288	288
query87	3348	3101	3101	3101
query88	5325	2396	2383	2383
query89	495	373	380	373
query90	1897	196	230	196
query91	135	102	103	102
query92	67	50	51	50
query93	6582	517	515	515
query94	1218	291	291	291
query95	410	321	318	318
query96	610	272	273	272
query97	3263	3049	3080	3049
query98	210	206	196	196
query99	1534	1266	1296	1266
Total cold run time: 290527 ms
Total hot run time: 175430 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.86 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 201925c11713cb7d52763d82b907ac9b4f272965, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.04	0.05
query4	1.68	0.10	0.10
query5	0.50	0.49	0.48
query6	1.13	0.72	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.56	0.49	0.48
query10	0.55	0.54	0.54
query11	0.16	0.12	0.12
query12	0.16	0.13	0.13
query13	0.60	0.59	0.58
query14	0.78	0.78	0.77
query15	0.86	0.81	0.82
query16	0.38	0.36	0.38
query17	0.99	1.03	1.04
query18	0.22	0.22	0.22
query19	1.93	1.68	1.83
query20	0.02	0.01	0.01
query21	15.42	0.79	0.64
query22	4.38	6.38	2.11
query23	18.36	1.38	1.29
query24	2.11	0.22	0.22
query25	0.15	0.09	0.08
query26	0.30	0.20	0.22
query27	0.45	0.24	0.22
query28	13.32	1.02	1.00
query29	12.60	3.29	3.28
query30	0.24	0.06	0.05
query31	2.87	0.40	0.38
query32	3.26	0.47	0.47
query33	2.94	2.95	2.93
query34	17.02	4.30	4.36
query35	4.45	4.41	4.43
query36	0.64	0.49	0.48
query37	0.20	0.16	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.14	0.12	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.19 s
Total hot run time: 30.86 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 24, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@morningman morningman merged commit 8551ac9 into apache:master Jul 25, 2024
30 of 32 checks passed
morningman added a commit to morningman/doris that referenced this pull request Aug 7, 2024
…ache#38310)

Sometime we don't want to audit operation from certain user in audit log
or audit table.
Add a new FE config `skip_audit_user_list`.
Default is empty, which means all operations will be recorded.
When you want to ignore some user's operation, you can set this config
like:

```
skip_audit_user_list=user1
--or
skip_audit_user_list=user1,user2
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants