Skip to content

Commit

Permalink
Refactor parameter definition of DistSQL for feature db-discovery (#2…
Browse files Browse the repository at this point in the history
…0206)

* Refactor parameter definition of DistSQL for feature db-discovery

* update doc of DistSQL for feature db-discovery

* fix shardingsphere-parser-test fail

* update docs
  • Loading branch information
sunkai-cai authored Aug 16, 2022
1 parent b6fcaeb commit 60610c6
Show file tree
Hide file tree
Showing 11 changed files with 160 additions and 18 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Class name: org.apache.shardingsphere.dbdiscovery.api.config.rule.DatabaseDiscov
Attributes:

| *Name* | *Data Type* | *Description* |
| -------------------------- | -------------------- | --------------------------------------------------------------------------------------- |
|----------------------------|----------------------|-----------------------------------------------------------------------------------------|
| groupName (+) | String | Database discovery group name |
| dataSourceNames (+) | Collection\<String\> | Data source names, multiple data source names separated with comma. Such as: ds_0, ds_1 |
| discoveryHeartbeatName (+) | String | Detect heartbeat name |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,17 @@ property:
key=value
```

### 参数解释
| 名称 | 数据类型 | 说明 |
|:-----------------------|:-----------|:--------------------|
| discoveryTypeName | IDENTIFIER | 数据库发现类型名 |
| ruleName | IDENTIFIER | 规则名称 |
| discoveryHeartbeatName | IDENTIFIER | 监听心跳名称 |
| typeName | STRING | 数据库发现类型,如:MySQL.MGR |
| resourceName | IDENTIFIER | 资源名称 |

### 注意事项

- `discoveryType` 指定数据库发现服务类型,`ShardingSphere` 内置支持 `MySQL.MGR`
- 重复的 `ruleName` 将无法被创建;
- 正在被使用的 `discoveryType``discoveryHeartbeat` 无法被删除;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,18 @@ property:
key=value
```

### Parameters Explained

| name | DateType | Description |
|:-----------------------|:-----------|:--------------------------------------------|
| discoveryTypeName | IDENTIFIER | Database discovery type name |
| ruleName | IDENTIFIER | Rule name |
| discoveryHeartbeatName | IDENTIFIER | Detect heartbeat name |
| typeName | STRING | Database discovery type, such as: MySQL.MGR |
| resourceName | IDENTIFIER | Resource name |

### Notes

- `discoveryType` specifies the database discovery service type, `ShardingSphere` has built-in support for `MySQL.MGR`
- Duplicate `ruleName` will not be created
- The `discoveryType` and `discoveryHeartbeat` being used cannot be deleted
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ ADD RESOURCE ds_0 (
```sql
CREATE DB_DISCOVERY RULE db_discovery_group_0 (
RESOURCES(ds_0, ds_1),
TYPE(NAME=mgr,PROPERTIES('group-name'='92504d5b-6dec')),
TYPE(NAME='MySQL.MGR',PROPERTIES('group-name'='92504d5b-6dec')),
HEARTBEAT(PROPERTIES('keep-alive-cron'='0/5 * * * * ?'))
);
```
Expand All @@ -44,7 +44,7 @@ HEARTBEAT(PROPERTIES('keep-alive-cron'='0/5 * * * * ?'))
```sql
ALTER DB_DISCOVERY RULE db_discovery_group_0 (
RESOURCES(ds_0, ds_1, ds_2),
TYPE(NAME=mgr,PROPERTIES('group-name'='92504d5b-6dec')),
TYPE(NAME='MySQL.MGR',PROPERTIES('group-name'='92504d5b-6dec')),
HEARTBEAT(PROPERTIES('keep-alive-cron'='0/5 * * * * ?'))
);
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ ADD RESOURCE ds_0 (
```sql
CREATE DB_DISCOVERY RULE db_discovery_group_0 (
RESOURCES(ds_0, ds_1),
TYPE(NAME=MySQL.MGR,PROPERTIES('group-name'='92504d5b-6dec')),
TYPE(NAME='MySQL.MGR',PROPERTIES('group-name'='92504d5b-6dec')),
HEARTBEAT(PROPERTIES('keep-alive-cron'='0/5 * * * * ?'))
);
```
Expand All @@ -44,7 +44,7 @@ HEARTBEAT(PROPERTIES('keep-alive-cron'='0/5 * * * * ?'))
```sql
ALTER DB_DISCOVERY RULE db_discovery_group_0 (
RESOURCES(ds_0, ds_1, ds_2),
TYPE(NAME=MySQL.MGR,PROPERTIES('group-name'='92504d5b-6dec')),
TYPE(NAME='MySQL.MGR',PROPERTIES('group-name'='92504d5b-6dec')),
HEARTBEAT(PROPERTIES('keep-alive-cron'='0/5 * * * * ?'))
);
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,11 @@ import Alphabet, Symbol;
IDENTIFIER
: [A-Za-z_$0-9]*?[A-Za-z_$]+?[A-Za-z_$0-9]*
| BQ ~'`'+ BQ
| (DQ ( '\\'. | '""' | ~('"'| '\\') )* DQ)
;

STRING
: (DQ ('""' | ~('"'| '\\') )* DQ)
| (SQ ('\'\'' | ~('\'' | '\\'))* SQ)
: (DQ ('\\'. | '""' | ~('"' | '\\'))* DQ)
| (SQ ('\\'. | '\'\'' | ~('\'' | '\\'))* SQ)
;

INT
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ resourceName
;

typeDefinition
: TYPE LP NAME EQ discoveryTypeName (COMMA PROPERTIES LP properties RP)? RP
: TYPE LP NAME EQ discoveryType (COMMA PROPERTIES LP properties RP)? RP
;

discoveryHeartbeat
Expand All @@ -100,11 +100,15 @@ properties
;

property
: key=(IDENTIFIER | STRING) EQ value=(NUMBER | INT | STRING)
: key=STRING EQ value=(NUMBER | INT | STRING)
;

discoveryType
: STRING
;

discoveryTypeName
: IDENTIFIER | STRING
: IDENTIFIER
;

discoveryHeartbeatName
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ public ASTNode visitDatabaseName(final DatabaseNameContext ctx) {

@Override
public ASTNode visitTypeDefinition(final TypeDefinitionContext ctx) {
return new AlgorithmSegment(getIdentifierValue(ctx.discoveryTypeName()), null == ctx.properties() ? new Properties() : getProperties(ctx.properties()));
return new AlgorithmSegment(getIdentifierValue(ctx.discoveryType()), null == ctx.properties() ? new Properties() : getProperties(ctx.properties()));
}

@Override
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.shardingsphere.dbdiscovery.distsql.parser;

import static org.hamcrest.CoreMatchers.equalTo;
import static org.hamcrest.CoreMatchers.is;
import static org.junit.Assert.assertThat;

import lombok.SneakyThrows;
import org.antlr.v4.runtime.tree.ParseTreeVisitor;
import org.apache.shardingsphere.dbdiscovery.distsql.parser.facade.DatabaseDiscoveryDistSQLStatementParserFacade;
import org.apache.shardingsphere.dbdiscovery.distsql.parser.segment.DatabaseDiscoveryDefinitionSegment;
import org.apache.shardingsphere.dbdiscovery.distsql.parser.segment.DatabaseDiscoveryProviderAlgorithmSegment;
import org.apache.shardingsphere.dbdiscovery.distsql.parser.statement.AlterDatabaseDiscoveryRuleStatement;
import org.apache.shardingsphere.dbdiscovery.distsql.parser.statement.AlterDatabaseDiscoveryTypeStatement;
import org.apache.shardingsphere.dbdiscovery.distsql.parser.statement.CreateDatabaseDiscoveryRuleStatement;
import org.apache.shardingsphere.dbdiscovery.distsql.parser.statement.CreateDatabaseDiscoveryTypeStatement;
import org.apache.shardingsphere.distsql.parser.core.featured.FeaturedDistSQLStatementParserFacadeFactory;
import org.apache.shardingsphere.distsql.parser.statement.DistSQLStatement;
import org.apache.shardingsphere.sql.parser.api.visitor.SQLVisitor;
import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
import org.apache.shardingsphere.sql.parser.core.SQLParserFactory;
import org.junit.Test;

import java.util.Arrays;
import java.util.Iterator;
import java.util.Properties;

public class DatabaseDiscoveryDistSQLTest {

@Test
public void assertCreateDatabaseDiscoveryRule() {
String sql = "CREATE DB_DISCOVERY RULE db_discovery_group_0 ("
+ "RESOURCES(ds_0, ds_1), TYPE(NAME='mgr',PROPERTIES('group-name'='92504d5b')),"
+ "HEARTBEAT(PROPERTIES('keep-alive-cron'='0/5 * * * * ?')))";

CreateDatabaseDiscoveryRuleStatement distSQLStatement = (CreateDatabaseDiscoveryRuleStatement) getDistSQLStatement(sql);
assertThat(distSQLStatement.getRules().size(), is(1));
assertDiscoverySegment((DatabaseDiscoveryDefinitionSegment) distSQLStatement.getRules().iterator().next());
}

@Test
public void assertAlterDatabaseDiscoveryRule() {
String sql = "ALTER DB_DISCOVERY RULE db_discovery_group_0 ("
+ "RESOURCES(ds_0, ds_1), TYPE(NAME='mgr',PROPERTIES('group-name'='92504d5b')),"
+ "HEARTBEAT(PROPERTIES('keep-alive-cron'='0/5 * * * * ?')))";
AlterDatabaseDiscoveryRuleStatement distSQLStatement = (AlterDatabaseDiscoveryRuleStatement) getDistSQLStatement(sql);
assertThat(distSQLStatement.getRules().size(), is(1));
assertDiscoverySegment((DatabaseDiscoveryDefinitionSegment) distSQLStatement.getRules().iterator().next());
}

@Test
public void assertCreateDatabaseDiscoveryType() {
String sql = "CREATE DB_DISCOVERY TYPE primary_replica_ds_mgr(TYPE(NAME='mgr',PROPERTIES('group-name'='92504d5b'))),primary_replica_ds_mgr_2(TYPE(NAME='mgr'))";
CreateDatabaseDiscoveryTypeStatement distSQLStatement = (CreateDatabaseDiscoveryTypeStatement) getDistSQLStatement(sql);
assertThat(distSQLStatement.getProviders().size(), is(2));
assertAlgorithmSegment(distSQLStatement.getProviders().iterator());
}

@Test
public void assertAlterDatabaseDiscoveryType() {
String sql = "ALTER DB_DISCOVERY TYPE primary_replica_ds_mgr(TYPE(NAME='mgr',PROPERTIES('group-name'='92504d5b'))),primary_replica_ds_mgr_2(TYPE(NAME='mgr'))";
AlterDatabaseDiscoveryTypeStatement distSQLStatement = (AlterDatabaseDiscoveryTypeStatement) getDistSQLStatement(sql);
assertThat(distSQLStatement.getProviders().size(), is(2));
assertAlgorithmSegment(distSQLStatement.getProviders().iterator());
}

private void assertDiscoverySegment(final DatabaseDiscoveryDefinitionSegment discoverySegment) {
assertThat(discoverySegment.getName(), is("db_discovery_group_0"));
assertThat(discoverySegment.getDataSources(), is(Arrays.asList("ds_0", "ds_1")));
Properties properties = new Properties();
properties.setProperty("group-name", "92504d5b");
assertThat(discoverySegment.getDiscoveryType().getName(), equalTo("mgr"));
assertThat(discoverySegment.getDiscoveryType().getProps(), equalTo(properties));
Properties heartbeatProps = new Properties();
heartbeatProps.setProperty("keep-alive-cron", "0/5 * * * * ?");
assertThat(discoverySegment.getDiscoveryHeartbeat(), is(heartbeatProps));
}

private void assertAlgorithmSegment(final Iterator<DatabaseDiscoveryProviderAlgorithmSegment> iterator) {
DatabaseDiscoveryProviderAlgorithmSegment providerAlgorithmSegment = iterator.next();
Properties properties = new Properties();
properties.setProperty("group-name", "92504d5b");
assertThat(providerAlgorithmSegment.getDiscoveryProviderName(), is("primary_replica_ds_mgr"));
assertThat(providerAlgorithmSegment.getAlgorithm().getName(), is("mgr"));
assertThat(providerAlgorithmSegment.getAlgorithm().getProps(), is(properties));
DatabaseDiscoveryProviderAlgorithmSegment providerAlgorithmSegment2 = iterator.next();
assertThat(providerAlgorithmSegment2.getDiscoveryProviderName(), is("primary_replica_ds_mgr_2"));
assertThat(providerAlgorithmSegment2.getAlgorithm().getName(), is("mgr"));
assertThat(providerAlgorithmSegment2.getAlgorithm().getProps(), is(new Properties()));
}

@SneakyThrows(ReflectiveOperationException.class)
@SuppressWarnings("rawtypes")
private DistSQLStatement getDistSQLStatement(final String sql) {
DatabaseDiscoveryDistSQLStatementParserFacade facade = new DatabaseDiscoveryDistSQLStatementParserFacade();
ParseASTNode parseASTNode = (ParseASTNode) SQLParserFactory.newInstance(sql, facade.getLexerClass(), facade.getParserClass()).parse();
SQLVisitor visitor = FeaturedDistSQLStatementParserFacadeFactory.getInstance(facade.getType()).getVisitorClass().getDeclaredConstructor().newInstance();
return (DistSQLStatement) ((ParseTreeVisitor) visitor).visit(parseASTNode.getRootNode());
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,10 @@
<distsql-case id="alter-sharding-binding-table-rules" value="ALTER SHARDING BINDING TABLE RULES (t_order,t_order_item), (t_1,t_2)" />
<distsql-case id="alter-sharding-broadcast-table-rules" value="ALTER SHARDING BROADCAST TABLE RULES(t_1,t_2)" />
<distsql-case id="alter-readwrite-splitting-rule" value="ALTER READWRITE_SPLITTING RULE ms_group_0 (AUTO_AWARE_RESOURCE=group_0, TYPE(NAME=random,PROPERTIES(read_weight='2:1'))), ms_group_1 (WRITE_RESOURCE=primary_ds, READ_RESOURCES(replica_ds_0,replica_ds_1),TYPE(NAME=random))" />
<distsql-case id="alter-database-discovery-definition-rule" value="ALTER DB_DISCOVERY RULE ha_group_0 (RESOURCES(resource0,resource1), TYPE(NAME=mgr,PROPERTIES(groupName='92504d5b-6dec')),HEARTBEAT(PROPERTIES('keepAliveCron'='0/5 * * * * ?'))), ha_group_1 (RESOURCES(resource2,resource3), TYPE(NAME=mgr2,PROPERTIES(groupName='92504d5b-6dec-2')),HEARTBEAT(PROPERTIES('keepAliveCron'='0/6 * * * * ?')))" />
<distsql-case id="alter-database-discovery-definition-rule" value="ALTER DB_DISCOVERY RULE ha_group_0 (RESOURCES(resource0,resource1), TYPE(NAME='mgr',PROPERTIES('groupName'='92504d5b-6dec')),HEARTBEAT(PROPERTIES('keepAliveCron'='0/5 * * * * ?'))), ha_group_1 (RESOURCES(resource2,resource3), TYPE(NAME='mgr2',PROPERTIES('groupName'='92504d5b-6dec-2')),HEARTBEAT(PROPERTIES('keepAliveCron'='0/6 * * * * ?')))" />
<distsql-case id="alter-database-discovery-construction-rule" value="ALTER DB_DISCOVERY RULE ha_group_0 (RESOURCES(resource0,resource1), TYPE=ha_group_0_mgr,HEARTBEAT=ha_group_0_heartbeat)" />
<distsql-case id="alter-database-discovery-heartbeat" value="ALTER DB_DISCOVERY HEARTBEAT mgr_heartbeat(PROPERTIES('keepAliveCron'='0/5 * * * * ?'))" />
<distsql-case id="alter-database-discovery-type" value="ALTER DB_DISCOVERY TYPE primary_replica_ds_mgr(TYPE(NAME=mgr,PROPERTIES('groupName'='92504d5b-6dec'))),primary_replica_ds_mgr_2(TYPE(NAME=mgr))" />
<distsql-case id="alter-database-discovery-type" value="ALTER DB_DISCOVERY TYPE primary_replica_ds_mgr(TYPE(NAME='mgr',PROPERTIES('groupName'='92504d5b-6dec'))),primary_replica_ds_mgr_2(TYPE(NAME='mgr'))" />
<distsql-case id="alter-encrypt-rule" value="ALTER ENCRYPT RULE t_encrypt (RESOURCE=ds_1, COLUMNS((NAME=user_id,PLAIN=user_plain,CIPHER=user_cipher,TYPE(NAME='AES',PROPERTIES('aes-key-value'='123456abc'))), (NAME=order_id, CIPHER =order_cipher,TYPE(NAME='MD5'))))" />
<distsql-case id="alter-encrypt-rule-with-assisted-query-column" value="ALTER ENCRYPT RULE t_encrypt (RESOURCE=ds_1, COLUMNS((NAME=user_id,PLAIN=user_plain,CIPHER=user_cipher,ASSISTED_QUERY_COLUMN=assisted_column, TYPE(NAME='AES',PROPERTIES('aes-key-value'='123456abc')), TYPE(NAME='MD5')), (NAME=order_id, CIPHER =order_cipher,TYPE(NAME='MD5'))))" />
<distsql-case id="alter-encrypt-rule-with-query-with-cipher-column" value="ALTER ENCRYPT RULE t_encrypt (RESOURCE=ds_1, COLUMNS((NAME=user_id,PLAIN=user_plain,CIPHER=user_cipher, TYPE(NAME='AES',PROPERTIES('aes-key-value'='123456abc'))), (NAME=order_id, CIPHER =order_cipher,TYPE(NAME='MD5'))), QUERY_WITH_CIPHER_COLUMN=false)" />
Expand Down
Loading

0 comments on commit 60610c6

Please sign in to comment.