Skip to content

Commit

Permalink
[fix](catalog)add custom jar (#23406)
Browse files Browse the repository at this point in the history
- allow put custom jar in `${DORIS_HOME}/lib/java_extensions/custom_extension` such as `paimon-s3-0.4.0-incubating.jar`
- add some note for paimon and fqdn
  • Loading branch information
zddr authored and xiaokang committed Aug 25, 2023
1 parent 3a42885 commit 6deadd6
Show file tree
Hide file tree
Showing 5 changed files with 39 additions and 9 deletions.
15 changes: 8 additions & 7 deletions bin/start_be.sh
Original file line number Diff line number Diff line change
Expand Up @@ -81,13 +81,14 @@ fi
preload_jars=("preload-extensions")
preload_jars+=("java-udf")

for preload_jar in "${preload_jars[@]}"; do
f="${DORIS_HOME}/lib/java_extensions/${preload_jar}/${preload_jar}-jar-with-dependencies.jar"
if [[ -z "${DORIS_CLASSPATH}" ]]; then
export DORIS_CLASSPATH="${f}"
else
export DORIS_CLASSPATH="${DORIS_CLASSPATH}:${f}"
fi
for preload_jar_dir in "${preload_jars[@]}"; do
for f in "${DORIS_HOME}/lib/java_extensions/${preload_jar_dir}"/*.jar; do
if [[ -z "${DORIS_CLASSPATH}" ]]; then
export DORIS_CLASSPATH="${f}"
else
export DORIS_CLASSPATH="${DORIS_CLASSPATH}:${f}"
fi
done
done

if [[ -d "${DORIS_HOME}/lib/hadoop_hdfs/" ]]; then
Expand Down
6 changes: 5 additions & 1 deletion docs/en/docs/admin-manual/cluster-management/fqdn.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,13 +76,17 @@ After deploying the cluster according to 'Enable FQDN for new cluster', if you w
Precondition: The current program supports the syntax 'Alter SYSTEM MODIFY FRONTEND'<fe_ip>:<edit_log_port>'HOSTNAME'<fe_hostname>',
If not, upgrade to a version that supports the syntax

>Note that.
>
> At least three followers are required to perform the following operations, otherwise the cluster may not start properly
Next, follow the steps below:

1. Perform the following operations on the Follower and Observer nodes one by one (and finally on the Master node):

1. Stop the node.
2. Check if the node has stopped. Execute 'show frontends' through the MySQL client to view the Alive status of the FE node until it becomes false
3. set FQDN for node: `ALTER SYSTEM MODIFY FRONTEND "<fe_ip>:<edit_log_port>" HOSTNAME "<fe_hostname>"`
3. set FQDN for node: `ALTER SYSTEM MODIFY FRONTEND "<fe_ip>:<edit_log_port>" HOSTNAME "<fe_hostname>"`(After stopping the master, a new master node will be selected and used to execute SQL statements)
4. Modify node configuration. Modify the 'conf/fe. conf' file in the FE root directory and add the configuration: `enable_fqdn_mode = true`
5. Start the node.

Expand Down
11 changes: 11 additions & 0 deletions docs/en/docs/lakehouse/multi-catalog/paimon.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,11 @@ CREATE CATALOG `paimon_hdfs` PROPERTIES (

#### S3

>Note that.
>
> user need download [paimon-s3-0.4.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-s3/0.4.0-incubating/paimon-s3-0.4.0-incubating.jar)
> Place it in directory ${DORIS_HOME}/be/lib/java_extensions/preload-extensions and restart be
```sql
CREATE CATALOG `paimon_s3` PROPERTIES (
"type" = "paimon",
Expand All @@ -73,6 +78,12 @@ CREATE CATALOG `paimon_s3` PROPERTIES (

#### OSS

>Note that.
>
> user need download [paimon-oss-0.4.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-oss/0.4.0-incubating/paimon-oss-0.4.0-incubating.jar)
> Place it in directory ${DORIS_HOME}/be/lib/java_extensions/preload-extensions and restart be

```sql
CREATE CATALOG `paimon_oss` PROPERTIES (
"type" = "paimon",
Expand Down
6 changes: 5 additions & 1 deletion docs/zh-CN/docs/admin-manual/cluster-management/fqdn.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,13 +76,17 @@ K8s 部署 Doris 的方法请参考[K8s 部署doris](../../install/k8s-deploy.md
前提条件:当前程序支持`ALTER SYSTEM MODIFY FRONTEND "<fe_ip>:<edit_log_port>" HOSTNAME "<fe_hostname>"`语法,
如果不支持,需要升级到支持该语法的版本

>注意:
>
> 至少有三台follower才能进行如下操作,否则会造成集群无法正常启动
接下来按照如下步骤操作:

1. 逐一对 Follower、Observer 节点进行以下操作(最后操作 Master 节点):

1. 停止节点。
2. 检查节点是否停止。通过 MySQL 客户端执行`show frontends`,查看该 FE 节点的 Alive 状态直至变为 false
3. 为节点设置 FQDN: `ALTER SYSTEM MODIFY FRONTEND "<fe_ip>:<edit_log_port>" HOSTNAME "<fe_hostname>"`
3. 为节点设置 FQDN: `ALTER SYSTEM MODIFY FRONTEND "<fe_ip>:<edit_log_port>" HOSTNAME "<fe_hostname>"`(停掉master后,会选举出新的master节点,用新的master节点来执行sql语句)
4. 修改节点配置。修改 FE 根目录中的`conf/fe.conf`文件,添加配置:`enable_fqdn_mode = true`
5. 启动节点。

Expand Down
10 changes: 10 additions & 0 deletions docs/zh-CN/docs/lakehouse/multi-catalog/paimon.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,11 @@ CREATE CATALOG `paimon_hdfs` PROPERTIES (

#### S3

>注意:
>
> 用户需要手动下载[paimon-s3-0.4.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-s3/0.4.0-incubating/paimon-s3-0.4.0-incubating.jar)
> 放在${DORIS_HOME}/be/lib/java_extensions/preload-extensions目录下并重启be
```sql
CREATE CATALOG `paimon_s3` PROPERTIES (
"type" = "paimon",
Expand All @@ -73,6 +78,11 @@ CREATE CATALOG `paimon_s3` PROPERTIES (

#### OSS

>注意:
>
> 用户需要手动下载[paimon-oss-0.4.0-incubating.jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-oss/0.4.0-incubating/paimon-oss-0.4.0-incubating.jar)
> 放在${DORIS_HOME}/be/lib/java_extensions/preload-extensions目录下并重启be
```sql
CREATE CATALOG `paimon_oss` PROPERTIES (
"type" = "paimon",
Expand Down

0 comments on commit 6deadd6

Please sign in to comment.