[improvement](catalog) optimize ldap and support more character in us…

…er and table name (apache#21968)

- common name support `-` ,reason: MySQL's db name support `-`
- table name support `-`
- username support `.`,reason:LDAP's username support `.`
- ldap doc
- ldap support rbac

docs/en/docs/admin-manual/privilege-ldap/ldap.md

@@ -38,6 +38,30 @@ LDAP group authorization, is to map the group in LDAP to the Role in Doris, if t
 * Privilege: Permissions act on nodes, databases or tables. Different permissions represent different permission to operate.
 * Role: Doris can create custom named roles. A role can be thought of as a collection of permissions.
 
+## LDAP related concepts
+
+In LDAP, data is organized in a tree structure.
+
+### Example (the following introduction will be expanded based on this example)
+- dc=example,dc=com
+- ou = ou1
+  - cn = group1
+  - cn = user1
+- ou = ou2
+  - cn = group2
+    - cn = user2
+- cn = user3
+
+### Explanation of LDAP Terms
+- dc(Domain Component): It can be understood as the domain name of an organization, serving as the root node of a tree
+- dn(Distinguished Name): Equivalent to a unique name, for example, the dn of user1 is cn=user1,ou=ou1,dc=example,dc=com the dn of user2 is cn=user2,cn=group2,ou=ou2,dc=example,dc=com
+- rdn(Relative Distinguished Name): As part of dn, the four rdns of user1 are cn=user1 ou=ou1 dc=example and dc=com
+- ou(Organization Unit): It can be understood as a sub organization, where users can be placed in ou or directly in the example.com domain
+- cn(common name):name
+- group: Group, which can be understood as the role of Doris
+- user: User, equivalent to Doris' user
+- objectClass：It can be understood as the type of data in each row, such as how to distinguish whether group1 is a group or a user. Each type of data requires different attributes below, such as CN and member (user list) for group, CN, password, uid, etc. for user
+
 ## Enable LDAP Authentication
 ### Server-side Configuration
 
@@ -57,7 +81,7 @@ You need to configure the LDAP basic information in the fe/conf/ldap.conf file,
   LDAP administrator account "Distinguished Name". When a user logs into Doris using LDAP authentication, Doris will bind the administrator account to search for user information in LDAP.
 
 * ldap_user_basedn = ou=people,dc=domain,dc=com
-  Doris base dn when searching for user information in LDAP.
+  Doris base dn when searching for user information in LDAP,For example, only user2 in the above example is allowed to log in to Doris, which is configured as ou=ou2, dc=example, dc=com. If user1, user2, and user3 in the above example are allowed to log in to Doris, which is configured as dc=example, dc=com
 
 * ldap_user_filter = (&(uid={login}))
 
@@ -69,7 +93,7 @@ You need to configure the LDAP basic information in the fe/conf/ldap.conf file,
   ldap_user_filter = (&(mail={login}@baidu.com))。
 
 * ldap_group_basedn = ou=group,dc=domain,dc=com
-  base dn when Doris searches for group information in LDAP. if this item is not configured, LDAP group authorization will not be enabled.  
+  base dn when Doris searches for group information in LDAP. if this item is not configured, LDAP group authorization will not be enabled. Same as ldap_ User_ Similar to basedn, it limits the scope of Doris searching for groups.
 
 #### Set the LDAP administrator password:
 After configuring the ldap.conf file, start fe, log in to Doris with the root or admin account, and execute sql:  
@@ -97,7 +121,7 @@ Client-side LDAP authentication requires the mysql client-side explicit authenti
 LDAP password authentication and group authorization are complementary to Doris password authentication and authorization. Enabling LDAP functionality does not completely replace Doris password authentication and authorization, but coexists with Doris password authentication and authorization.
 
 ### LDAP authentication login details
-When LDAP is enabled, users have the following in Doris and DLAP:  
+When LDAP is enabled, users have the following in Doris and LDAP:  
 
 |LDAP User|Doris User|Password|Login Status|Login to Doris users|
 |--|--|--|--|--|
@@ -168,9 +192,26 @@ Then the group name is doris_rd.
 
 If jack also belongs to the LDAP groups doris_qa, doris_pm; Doris exists roles: doris_rd, doris_qa, doris_pm, after logging in using LDAP authentication, the user will not only have the original permissions of the account, but will also get the roles doris_rd, doris_qa and doris _pm privileges.
 
+>Attention:
+>
+>The group to which user belongs is not related to the organizational structure of the LDAP tree, and user2 in the example section may not necessarily belong to group2
+> If you want user2 to belong to group2, you need to add user2 to the member attribute of group2
+
 ### LDAP information cache
 To avoid frequent access to LDAP service, Doris will cache LDAP information into memory, you can specify the cache time for LDAP users through the `ldap_user_cache_timeout_s` configuration item in ldap.conf, the default is 12 hours; after modifying the information in LDAP service or modifying the After modifying the information in the LDAP service or modifying the Role permissions of the LDAP user group, the cache may not take effect in time because of the cache, so you can refresh the cache with the refresh ldap statement, see [REFRESH-LDAP](... /... /sql-manual/sql-reference/Utility-Statements/REFRESH-LDAP.md).
 
 ## Limitations of LDAP authentication
 
 * The current LDAP feature of Doris only supports plaintext password authentication, that is, when a user logs in, the password is transmitted in plaintext between client and fe and between fe and LDAP service.
+
+## FAQ
+
+- How to determine which roles an LDAP user has in Doris?
+
+  Log in to Doris using an LDAP user, ` show grants` Can view which roles the current user has. Among them, ldapDefaultRole is the default role that every ldap user has in Doris.
+- How to troubleshoot when the roles of LDAP users in Doris are less than expected?
+
+  1. Through 'show roles` Check if the expected role exists in Doris. If it does not exist, you need to use the 'CREATE ROLE role'_ Name` Create a character.
+  2. Check if the expected group is in 'ldap'_ Group_ Based on the corresponding organizational structure.
+  3. Check if the expected group contains the member attribute.
+  4. Check if the member attribute of the expected group contains the current user.

docs/zh-CN/docs/admin-manual/privilege-ldap/ldap.md

@@ -38,6 +38,30 @@ LDAP组授权是将LDAP中的group映射到Doris中的Role，如果用户在LDAP
 - 权限 Privilege：权限作用的对象是节点、数据库或表。不同的权限代表不同的操作许可。
 - 角色 Role：Doris可以创建自定义命名的角色。角色可以被看做是一组权限的集合。
 
+## LDAP相关概念
+
+在LDAP中，数据是按照树型结构组织的。
+
+### 示例（下文的介绍都将根据这个例子进行展开）
+ - dc=example,dc=com
+  - ou = ou1
+    - cn = group1
+    - cn = user1
+  - ou = ou2
+    - cn = group2
+      - cn = user2
+  - cn = user3
+
+### LDAP名词解释
+- dc(Domain Component): 可以理解为一个组织的域名，作为树的根结点
+- dn(Distinguished Name): 相当于唯一名称，例如user1的dn为 cn=user1,ou=ou1,dc=example,dc=com user2的dn为 cn=user2,cn=group2,ou=ou2,dc=example,dc=com
+- rdn(Relative Distinguished Name): dn的一部分，user1的四个rdn为cn=user1 ou=ou1 dc=example和dc=com
+- ou(Organization Unit): 可以理解为子组织，user可以放在ou中，也可以直接放在example.com域中
+- cn(common name):名字
+- group: 组，可以理解为doris的角色
+- user: 用户，和doris的用户等价
+- objectClass：可以理解为每行数据的类型，比如怎么区分group1是group还是user，每种类型的数据下面要求有不同的属性，比如group要求有cn和member（user列表），user要求有cn,password,uid等
+
 ## 启用LDAP认证
 
 ### server端配置
@@ -59,7 +83,7 @@ LDAP组授权是将LDAP中的group映射到Doris中的Role，如果用户在LDAP
   LDAP管理员账户“Distinguished Name”。当用户使用LDAP验证登录Doris时，Doris会绑定该管理员账户在LDAP中搜索用户信息。
 
 - ldap_user_basedn = ou=people,dc=domain,dc=com
-  Doris在LDAP中搜索用户信息时的base dn。
+  Doris在LDAP中搜索用户信息时的base dn，例如只允许上例中的user2登陆doris，此处配置为ou=ou2,dc=example,dc=com 如果允许上例中的user1,user2,user3都能登陆doris，此处配置为dc=example,dc=com
 
 - ldap_user_filter = (&(uid={login}))
 
@@ -71,7 +95,7 @@ LDAP组授权是将LDAP中的group映射到Doris中的Role，如果用户在LDAP
   ldap_user_filter = (&(mail={login}@baidu.com))。
 
 - ldap_group_basedn = ou=group,dc=domain,dc=com
-  Doris在LDAP中搜索组信息时的base dn。如果不配置该项，将不启用LDAP组授权。
+  Doris在LDAP中搜索组信息时的base dn。如果不配置该项，将不启用LDAP组授权。同ldap_user_basedn类似，限制doris搜索group时的范围。
 
 #### 设置LDAP管理员密码：
 
@@ -107,7 +131,7 @@ LDAP密码验证和组授权是Doris密码验证和授权的补充，开启LDAP
 
 ### LDAP验证登录详解
 
-开启LDAP后，用户在Doris和DLAP中存在以下几种情况：
+开启LDAP后，用户在Doris和LDAP中存在以下几种情况：
 
 | LDAP用户 | Doris用户 | 密码      | 登录情况 | 登录Doris的用户 |
 | -------- | --------- | --------- | -------- | --------------- |
@@ -185,9 +209,26 @@ member: uid=jack,ou=aidp,dc=domain,dc=com
 
 假如jack还属于LDAP组doris_qa、doris_pm；Doris存在role：doris_rd、doris_qa、doris_pm，在使用LDAP验证登录后，用户不但具有该账户原有的权限，还将获得role doris_rd、doris_qa和doris_pm的权限。
 
+>注意：
+>
+>user属于哪个group和LDAP树的组织结构无关，示例部分的user2并不一定属于group2
+> 若想让user2属于group2，需要在group2的member属性中添加user2
+
 ### LDAP信息缓存
 为了避免频繁访问LDAP服务，Doris会将LDAP信息缓存到内存中，可以通过ldap.conf中的`ldap_user_cache_timeout_s`配置项指定LDAP用户的缓存时间，默认为12小时；在修改了LDAP服务中的信息或者修改了Doris中LDAP用户组对应的Role权限后，可能因为缓存而没有及时生效，可以通过refresh ldap语句刷新缓存，详细查看[REFRESH-LDAP](../../sql-manual/sql-reference/Utility-Statements/REFRESH-LDAP.md)。
 
 ## LDAP验证的局限
 
 - 目前Doris的LDAP功能只支持明文密码验证，即用户登录时，密码在client与fe之间、fe与LDAP服务之间以明文的形式传输。
+
+## 常见问题
+
+- 怎么判断LDAP用户在doris中有哪些角色？
+
+  使用LDAP用户在doris中登陆，`show grants;`能查看当前用户有哪些角色。其中ldapDefaultRole是每个ldap用户在doris中都有的默认角色。
+- LDAP用户在doris中的角色比预期少怎么排查？
+
+  1. 通过`show roles;`查看预期的角色在doris中是否存在，如果不存在，需要通过` CREATE ROLE rol_name;`创建角色。
+  2. 检查预期的group是否在`ldap_group_basedn`对应的组织结构下。
+  3. 检查预期group是否包含member属性。
+  4. 检查预期group的member属性是否包含当前用户。

fe/fe-core/src/main/java/org/apache/doris/common/FeNameFormat.java

@@ -29,13 +29,15 @@
 
 public class FeNameFormat {
     private static final String LABEL_REGEX = "^[-_A-Za-z0-9:]{1,128}$";
-    private static final String COMMON_NAME_REGEX = "^[a-zA-Z][a-zA-Z0-9_]{0,63}$";
-    private static final String TABLE_NAME_REGEX = "^[a-zA-Z][a-zA-Z0-9_]*$";
+    private static final String COMMON_NAME_REGEX = "^[a-zA-Z][a-zA-Z0-9-_]{0,63}$";
+    private static final String TABLE_NAME_REGEX = "^[a-zA-Z][a-zA-Z0-9-_]*$";
+    private static final String USER_NAME_REGEX = "^[a-zA-Z][a-zA-Z0-9.-_]*$";
     private static final String COLUMN_NAME_REGEX = "^[_a-zA-Z@0-9\\s<>/][.a-zA-Z0-9_+-/><?@#$%^&*\"\\s,:]{0,255}$";
 
     private static final String UNICODE_LABEL_REGEX = "^[-_A-Za-z0-9:\\p{L}]{1,128}$";
-    private static final String UNICODE_COMMON_NAME_REGEX = "^[a-zA-Z\\p{L}][a-zA-Z0-9_\\p{L}]{0,63}$";
-    private static final String UNICODE_TABLE_NAME_REGEX = "^[a-zA-Z\\p{L}][a-zA-Z0-9_\\p{L}]*$";
+    private static final String UNICODE_COMMON_NAME_REGEX = "^[a-zA-Z\\p{L}][a-zA-Z0-9-_\\p{L}]{0,63}$";
+    private static final String UNICODE_TABLE_NAME_REGEX = "^[a-zA-Z\\p{L}][a-zA-Z0-9-_\\p{L}]*$";
+    private static final String UNICODE_USER_NAME_REGEX = "^[a-zA-Z\\p{L}][a-zA-Z0-9.-_\\p{L}]*$";
     private static final String UNICODE_COLUMN_NAME_REGEX
             = "^[_a-zA-Z@0-9\\p{L}][.a-zA-Z0-9_+-/><?@#$%^&*\\p{L}]{0,255}$";
 
@@ -102,7 +104,7 @@ public static void checkLabel(String label) throws AnalysisException {
     }
 
     public static void checkUserName(String userName) throws AnalysisException {
-        if (Strings.isNullOrEmpty(userName) || !userName.matches(getCommonNameRegex())) {
+        if (Strings.isNullOrEmpty(userName) || !userName.matches(getUserNameRegex())) {
             throw new AnalysisException("invalid user name: " + userName);
         }
     }
@@ -165,6 +167,14 @@ public static String getTableNameRegex() {
         }
     }
 
+    public static String getUserNameRegex() {
+        if (FeNameFormat.isEnableUnicodeNameSupport()) {
+            return UNICODE_USER_NAME_REGEX;
+        } else {
+            return USER_NAME_REGEX;
+        }
+    }
+
     public static String getLabelRegex() {
         if (FeNameFormat.isEnableUnicodeNameSupport()) {
             return UNICODE_LABEL_REGEX;

fe/fe-core/src/main/java/org/apache/doris/ldap/LdapManager.java

@@ -17,23 +17,30 @@
 
 package org.apache.doris.ldap;
 
+import org.apache.doris.analysis.TablePattern;
 import org.apache.doris.analysis.UserIdentity;
 import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.InfoSchemaDb;
 import org.apache.doris.cluster.ClusterNamespace;
+import org.apache.doris.common.AnalysisException;
 import org.apache.doris.common.DdlException;
 import org.apache.doris.common.LdapConfig;
 import org.apache.doris.mysql.privilege.Auth;
+import org.apache.doris.mysql.privilege.PrivBitSet;
+import org.apache.doris.mysql.privilege.Privilege;
 import org.apache.doris.mysql.privilege.Role;
 
 import com.google.common.base.Strings;
-import com.google.common.collect.Lists;
 import com.google.common.collect.Maps;
+import com.google.common.collect.Sets;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 
+import java.util.Collections;
 import java.util.List;
 import java.util.Map;
 import java.util.Objects;
+import java.util.Set;
 import java.util.concurrent.locks.ReentrantReadWriteLock;
 
 /**
@@ -42,7 +49,7 @@
 public class LdapManager {
     private static final Logger LOG = LogManager.getLogger(LdapManager.class);
 
-    public static final String LDAP_GROUPS_PRIVS_NAME = "ldapGroupsPrivs";
+    public static final String LDAP_DEFAULT_ROLE = "ldapDefaultRole";
 
     private final LdapClient ldapClient = new LdapClient();
 
@@ -122,9 +129,9 @@ public boolean checkUserPasswd(String fullName, String passwd, String remoteIp,
         return false;
     }
 
-    public Role getUserRole(String fullName) {
+    public Set<Role> getUserRoles(String fullName) {
         LdapUserInfo info = getUserInfo(fullName);
-        return !Objects.isNull(info) && info.isExists() ? info.getPaloRole() : new Role(LDAP_GROUPS_PRIVS_NAME);
+        return info == null ? Collections.emptySet() : info.getPaloRoles();
     }
 
     private boolean checkParam(String fullName) {
@@ -142,7 +149,7 @@ private LdapUserInfo getUserInfoAndUpdateCache(String fulName) throws DdlExcepti
         }
         checkTimeoutCleanCache();
 
-        LdapUserInfo ldapUserInfo = new LdapUserInfo(fulName, false, "", getLdapGroupsPrivs(userName, cluster));
+        LdapUserInfo ldapUserInfo = new LdapUserInfo(fulName, false, "", getLdapGroupsRoles(userName, cluster));
         writeLock();
         try {
             ldapUserInfoCache.put(ldapUserInfo.getUserName(), ldapUserInfo);
@@ -198,26 +205,24 @@ private LdapUserInfo getUserInfoFromCache(String fullName) {
     /**
      * Step1: get ldap groups from ldap server;
      * Step2: get roles by ldap groups;
-     * Step3: merge the roles;
+     * Step3: generate default role;
      */
-    private Role getLdapGroupsPrivs(String userName, String clusterName) throws DdlException {
+    private Set<Role> getLdapGroupsRoles(String userName, String clusterName) throws DdlException {
         //get user ldap group. the ldap group name should be the same as the doris role name
         List<String> ldapGroups = ldapClient.getGroups(userName);
-        List<String> rolesNames = Lists.newArrayList();
+        Set<Role> roles = Sets.newHashSet();
         for (String group : ldapGroups) {
             String qualifiedRole = ClusterNamespace.getFullName(clusterName, group);
             if (Env.getCurrentEnv().getAuth().doesRoleExist(qualifiedRole)) {
-                rolesNames.add(qualifiedRole);
+                roles.add(Env.getCurrentEnv().getAuth().getRoleByName(qualifiedRole));
             }
         }
-        LOG.debug("get user:{} ldap groups:{} and doris roles:{}", userName, ldapGroups, rolesNames);
+        LOG.debug("get user:{} ldap groups:{} and doris roles:{}", userName, ldapGroups, roles);
 
-        Role ldapGroupsPrivs = new Role(LDAP_GROUPS_PRIVS_NAME);
-        LdapPrivsChecker.grantDefaultPrivToTempUser(ldapGroupsPrivs, clusterName);
-        if (!rolesNames.isEmpty()) {
-            Env.getCurrentEnv().getAuth().mergeRolesNoCheckName(rolesNames, ldapGroupsPrivs);
-        }
-        return ldapGroupsPrivs;
+        Role ldapGroupsPrivs = new Role(LDAP_DEFAULT_ROLE);
+        grantDefaultPrivToTempUser(ldapGroupsPrivs, clusterName);
+        roles.add(ldapGroupsPrivs);
+        return roles;
     }
 
     public void refresh(boolean isAll, String fullName) {
@@ -235,4 +240,16 @@ public void refresh(boolean isAll, String fullName) {
             writeUnlock();
         }
     }
+
+    // Temporary user has information_schema 'Select_priv' priv by default.
+    public static void grantDefaultPrivToTempUser(Role role, String clusterName) throws DdlException {
+        TablePattern tblPattern = new TablePattern(InfoSchemaDb.DATABASE_NAME, "*");
+        try {
+            tblPattern.analyze(clusterName);
+        } catch (AnalysisException e) {
+            LOG.warn("should not happen.", e);
+        }
+        Role newRole = new Role(role.getRoleName(), tblPattern, PrivBitSet.of(Privilege.SELECT_PRIV));
+        role.merge(newRole);
+    }
 }