Skip to content
This repository has been archived by the owner on Oct 18, 2021. It is now read-only.

Commit

Permalink
Merge pull request #88 from Nicole00/readme
Browse files Browse the repository at this point in the history
add submit command for yarn-cluster
  • Loading branch information
Nicole00 authored May 31, 2021
2 parents 7ae60e0 + d6c7edb commit f235882
Show file tree
Hide file tree
Showing 2 changed files with 34 additions and 10 deletions.
23 changes: 18 additions & 5 deletions nebula-exchange/README-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,18 @@ Exchange 2.0 仅支持 Nebula Graph 2.x。

## 如何获取

1. 编译打包 Exchange 2.0
1. 编译打包最新的 Exchange。

```bash
$ git clone -b v2.0.0 https://github.com/vesoft-inc/nebula-spark-utils.git
$ git clone https://github.com/vesoft-inc/nebula-spark-utils.git
$ cd nebula-spark-utils/nebula-exchange
$ mvn clean package -Dmaven.test.skip=true -Dgpg.skip -Dmaven.javadoc.skip=true
```

编译打包完成后,可以在 nebula-spark-utils/nebula-exchange/target/ 目录下看到 nebula-exchange-2.0.0.jar 文件。
编译打包完成后,可以在 nebula-spark-utils/nebula-exchange/target/ 目录下看到 nebula-exchange-2.0-SNAPSHOT.jar 文件。
2. 在 Maven 远程仓库下载

https://repo1.maven.org/maven2/com/vesoft/nebula-exchange/2.0.0/
https://repo1.maven.org/maven2/com/vesoft/nebula-exchange/
## 使用说明

特性 & 注意事项:
Expand All @@ -29,7 +29,7 @@ Exchange 2.0 仅支持 Nebula Graph 2.x。

*2. Exchange 2.0 新增 null、Date、DateTime、Time 类型数据的导入( DateTime 是 UTC 时区,非 Local time)。*

*3. Exchange 2.0 支持 Hive on Spark 以外的 Hive 数据源,需在配置文件中配置 Hive 源,具体配置示例参考 [server_application.conf](https://github.com/vesoft-inc/nebula-spark-utils/blob/main/nebula-exchange/src/main/resources/server_application.conf) 中 Hive 的配置。*
*3. Exchange 2.0 支持 Hive on Spark 以外的 Hive 数据源,需在配置文件中配置 Hive 源,具体配置示例参考 [application.conf](https://github.com/vesoft-inc/nebula-spark-utils/tree/master/nebula-exchange/src/main/resources/application.conf) 中 Hive 的配置。*

*4. Exchange 2.0 将导入失败的 INSERT 语句进行落盘,存于配置文件的 error/output 路径中。*

Expand All @@ -41,6 +41,19 @@ $SPARK_HOME/bin/spark-submit --class com.vesoft.nebula.exchange.Exchange --maste
```
如果数据源有HIVE,则导入命令最后还需要加 `-h` 表示启用HIVE数据源。
注:在Yarn-Cluster模式下提交 Exchange,请使用如下提交命令:
```
$SPARK_HOME/bin/spark-submit --class com.vesoft.nebula.exchange.Exchange \
--master yarn-cluster \
--files application.conf \
--conf spark.driver.extraClassPath=./ \
--conf spark.executor.extraClassPath=./ \
nebula-exchange-2.0.0.jar \
-c application.conf
```
关于 Nebula Exchange 的更多说明,请参考 Exchange 2.0 的[使用手册](https://docs.nebula-graph.com.cn/2.0.1/nebula-exchange/about-exchange/ex-ug-what-is-exchange/) 。
## 贡献
Nebula Exchange 2.0 是一个完全开源的项目,欢迎开源爱好者通过以下方式参与:
Expand Down
21 changes: 16 additions & 5 deletions nebula-exchange/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,18 @@ Exchange 2.0 only supports Nebula Graph 2.0 . If you want to import data for Neb

## How to get

1. Package Exchange 2.0
1. Package latest Exchange

```bash
$ git clone -b v2.0.0 https://github.com/vesoft-inc/nebula-spark-utils.git
$ git clone https://github.com/vesoft-inc/nebula-spark-utils.git
$ cd nebula-spark-utils/nebula-exchange
$ mvn clean package -Dmaven.test.skip=true -Dgpg.skip -Dmaven.javadoc.skip=true
```

After the packaging, you can see the newly generated nebula-exchange-2.0.0.jar under the nebula-spark-utils/nebula-exchange/target/ directory.
After the packaging, you can see the newly generated nebula-exchange-2.0-SNAPSHOT.jar under the nebula-spark-utils/nebula-exchange/target/ directory.
2. Download from Maven repository

https://repo1.maven.org/maven2/com/vesoft/nebula-exchange/2.0.0/
https://repo1.maven.org/maven2/com/vesoft/nebula-exchange/
## How to use

Import command:
Expand All @@ -30,7 +30,18 @@ If your source is HIVE, import command is:
$SPARK_HOME/bin/spark-submit --class com.vesoft.nebula.exchange.Exchange --master local nebula-exchange-2.0.0.jar -c /path/to/application.conf -h
```
For more details about Exchange, please refer to [Exchange 1.0](https://github.com/vesoft-inc/nebula-java/tree/v1.0/tools/exchange) .
Note:Submit Exchange with Yarn-Cluster mode, please use following command:
```
$SPARK_HOME/bin/spark-submit --class com.vesoft.nebula.exchange.Exchange \
--master yarn-cluster \
--files application.conf \
--conf spark.driver.extraClassPath=./ \
--conf spark.executor.extraClassPath=./ \
nebula-exchange-2.0.0.jar \
-c application.conf
```
For more details about Exchange, please refer to [Exchange 2.0](https://docs.nebula-graph.io/2.0.1/16.eco-tools/1.nebula-exchange/) .
## New Features
Expand Down

0 comments on commit f235882

Please sign in to comment.