Skip to content

Commit

Permalink
update pic size
Browse files Browse the repository at this point in the history
  • Loading branch information
yangj1211 committed Feb 7, 2024
1 parent 324c4f2 commit 2b9fcfc
Show file tree
Hide file tree
Showing 11 changed files with 95 additions and 74 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,9 @@ Apache Flink 是一个强大的框架和分布式处理引擎,专注于进行

配置示例如下图所示:

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/flink/matrixone-flink-demo.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/flink/matrixone-flink-demo.png width=50% heigth=50%/>
</div>

2. 添加项目依赖,编辑项目根目录下的 `pom.xml` 文件,将以下内容添加到文件中:

Expand Down
17 changes: 10 additions & 7 deletions docs/MatrixOne/Develop/Ecological-Tools/Computing-Engine/Spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,10 @@ Apache Spark 是一个为高效处理大规模数据而设计的分布式计算
- **Package name**:com.matrixone.demo
- **JDK** 1.8

<div align="center">
<img src=hhttps://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/spark/matrixone-spark-demo.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/spark/matrixone-spark-demo.png width=50% heigth=50%/>
</div>

2. 添加项目依赖,在项目根目录下的 `pom.xml` 内容编辑如下:

```xml
Expand Down Expand Up @@ -314,9 +315,9 @@ select * from test.person;
- **Package name**com.matrixone.demo
- **JDK** 1.8

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/spark/matrixone-spark-demo.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/spark/matrixone-spark-demo.png width=50% heigth=50%/>
</div>

2. 添加项目依赖,在项目根目录下的 `pom.xml` 内容编辑如下:

Expand Down Expand Up @@ -412,7 +413,9 @@ CREATE TABLE `users` (

将 Hadoop 根目录下的 "etc/hadoop/core-site.xml""hdfs-site.xml" 以及 Hive 根目录下的 "conf/hive-site.xml" 这三个配置文件复制到项目的 "resource" 目录中。

![配置文件](https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/spark/config-files.png)
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/spark/config-files.png width=30% heigth=30%/>
</div>

### 步骤五:编写代码

Expand Down
44 changes: 24 additions & 20 deletions docs/MatrixOne/Develop/connect-mo/database-client-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,31 +66,33 @@ MatrixOne 现在支持通过以下几种数据库客户端工具的方式连接
3. 点击 **Save** 保存设置。
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/navicat-config.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/navicat-config.png width=50% heigth=50%/>
</div>
4. 双击左侧数据库目录中的 **MatrixOne**,图标点亮,连接成功。
5. 连接到 MatrixOne 后,在左侧数据库目录栏,你将看到 6 个默认系统数据库:
<img src="https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/navicat-databases.png" style="zoom: 60%;" />
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/navicat-databases.png width=50% heigth=50%/>
</div>
右侧窗口可查看有关此连接的基本信息:
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/navicat-connection.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/navicat-connection.png width=30% heigth=30%/>
</div>
## 通过 DBeaver 连接 MatrixOne 服务
1. 下载安装 [DBeaver](https://dbeaver.io/download/)。
2. 安装 DBeaver 完成后,打开 DBeaver,点击左上角**连接**图标,在弹窗中选择 **MySQL**,点击 **Next**。
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/dbeaver-mysql.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/dbeaver-mysql.png width=50% heigth=50%/>
</div>
在 **Connect to a database** 窗口的 **Main** 区中填写如下参数:
Expand All @@ -101,22 +103,24 @@ MatrixOne 现在支持通过以下几种数据库客户端工具的方式连接
- **Password**: 111
- **Save password locally**: 勾选
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/dbeaver-connection.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/dbeaver-connection.png width=50% heigth=50%/>
</div>
3. 双击左侧目录中的 **MatrixOne**,连接 MatrixOne 服务。你可以在左侧目录树中看到默认的四个系统数据库:
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/dbeaver-databases.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/dbeaver-databases.png width=70% heigth=70%/>
</div>
4. 默认情况下,DBeaver 中不展示视图。如需显示完整的系统数据库,你可以右键单击 **MatrixOne**,选择 **Connection view** 并打开 **Show system objects**:
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/show-system-objects.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/show-system-objects.png width=50% heigth=50%/>
</div>
设置完成后,你将看到 6 个系统数据库。
![dbeaver-databases-with-view](https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/dbeaver-databases-with-view.png)
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/dbeaver-databases-with-view.png width=70% heigth=70%/>
</div>
Original file line number Diff line number Diff line change
Expand Up @@ -27,23 +27,25 @@

2. 在 IDEA 中新建 Java 名称为 **testJDBC** 的项目并选择在 **Build System** 中选择 **Maven** 作为构建系统,点击 **Create**

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/JDBC_connect/JDBC-create-project.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/JDBC_connect/JDBC-create-project.png width=50% heigth=50%/>
</div>

3. 点击 **File > Project Structure**,进入到 **Project Setting**,点选 **Library**,并点击 **+** 按钮,添加 **From Maven**

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/JDBC_connect/JDBC-project-structure.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/JDBC_connect/JDBC-project-structure.png width=40% heigth=40%/>
</div>

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/JDBC_connect/JDBC-from-maven.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/JDBC_connect/JDBC-from-maven.png width=50% heigth=50%/>
</div>

4. 输入框中输入 **mysql-connector-java** 搜索整个库,选择 **mysql:mysql-connector-java:8.0.30**,应用到本项目中。

![JDBC add driver](https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/JDBC_connect/JDBC-add-driver.png)
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/develop/JDBC_connect/JDBC-add-driver.png width=70% heigth=70%/>
</div>

5. 修改 **src/main/java/org/example/Main.java** 中的默认 Java 源代码。如下面的代码示例中所示,这段代码使用连接地址和凭据创建连接。连接到 MatrixOne 后,你可以使用 Java 语言对 MatrixOne 数据库和表进行操作。

Expand Down
42 changes: 21 additions & 21 deletions docs/MatrixOne/Migrate/migrate-from-mysql-to-matrixone.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ MatrixOne 中,数据类型在保持与 MySQL 命名一致的情况下,在精

1. 打开 DBeaver,从 MySQL 中选择待迁移的表,鼠标右键点击后选择**生成 SQL > DDL** 点击**复制**,先将这段 SQL 复制到一个文本编辑器内,给文本编辑器命名为 *tpch_ddl.sql*,保存到跳板机本地。

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-1.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-1.png width=50% heigth=50%/>
</div>

2. 连接 MatrixOne,并在 MatrixOne 中创建新的数据库和表:

Expand All @@ -41,29 +41,29 @@ MatrixOne 中,数据类型在保持与 MySQL 命名一致的情况下,在精

1. 打开 DBeaver,从 MySQL 中选择待迁移的表,鼠标右键点击后选择**导出数据**

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-2.png width=40% heigth=40%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-2.png width=40% heigth=40%/>
</div>

2. 在**转化目标 > 导出目标**窗口选择**数据库**,点击**下一步**;在**表映射**窗口选择**目标容器**,目标容器选择 MatrixOne 的数据库 *tpch*

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-3.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-3.png width=50% heigth=50%/>
</div>

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-4.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-4.png width=50% heigth=50%/>
</div>

3. 在**抽取设置****数据加载设置**窗口,设置选择抽取和插入的数量,为了触发 MatrixOne 的直接写 S3 策略,建议填写 5000

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-5.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-5.png width=50% heigth=50%/>
</div>

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-6.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-6.png width=50% heigth=50%/>
</div>

4. 完成设置后,DBeaver 开始对数据进行迁移,在完成后 DBeaver 将会提示迁移成功。

Expand Down Expand Up @@ -93,9 +93,9 @@ MatrixOne 中,数据类型在保持与 MySQL 命名一致的情况下,在精

1. 打开 DBeaver,从 MySQL 中选择待迁移的表,鼠标右键点击后选择**生成 SQL > DDL > 复制**,先将这段 SQL 复制到一个文本编辑器内,给文本编辑器命名为 *tpch_ddl.sql*,保存到跳板机本地。

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-1.png width=50% heigth=50%/>
</div>
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-1.png width=50% heigth=50%/>
</div>

2. 连接到 MatrixOne,在 MatrixOne 中创建新的数据库和表:

Expand Down
2 changes: 1 addition & 1 deletion docs/MatrixOne/Migrate/migrate-from-oracle-to-matrixone.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ MatrixOne 与 Oracle 自带的数据类型存在着较多的差异,这些差
1. 打开 DBeaver,中选择待迁移的表,鼠标右键点击后选择**导出数据**,将 Oracle 数据表导出为 CSV 格式文件:
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-2.png width=50% heigth=50%/>
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-2.png width=40% heigth=40%/>
</div>
2. 在**转化目标 > 导出目标**窗口选择 **CSV**,点击**下一步**:
Expand Down
4 changes: 2 additions & 2 deletions docs/MatrixOne/Migrate/migrate-from-sqlserver-to-matrixone.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ MatrixOne 与 SQL Server 存在着诸多类型上的差异,这些差异有些
1. 打开 DBeaver,从 SQL Server 中选择待迁移的表,鼠标右键点击**导出数据**

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-2.png width=50% heigth=50%/>
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-2.png width=40% heigth=40%/>
</div>

2.**转化目标 > 导出目标**窗口选择**数据库**,点击**下一步**;在**表映射**窗口选择**目标容器**,目标容器选择 MatrixOne 的数据库 *tpch*
Expand Down Expand Up @@ -118,7 +118,7 @@ MatrixOne 与 SQL Server 存在着诸多类型上的差异,这些差异有些
1. 打开 DBeaver,中选择待迁移的表,鼠标右键点击后选择**导出数据**,将 SQL Server 数据表导出为 CSV 格式文件:

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-2.png width=50% heigth=50%/>
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/migrate/mysql-2.png width=40% heigth=40%/>
</div>

2. 在**转化目标 > 导出目标**窗口选择 **CSV**,点击**下一步**
Expand Down
18 changes: 13 additions & 5 deletions docs/MatrixOne/Overview/architecture/architecture-logservice.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ Logservice 使用基于 Raft 协议的 dragonboat 库(multi-raft group 的 Go

Logservice 的架构由客户端和服务端两部分组成。服务端包括 handler、dragonboat 和 RSM(Replicated State Machine)等模块,而客户端则包含多个关键接口。它们之间的协作关系如下图所示:

![](https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/overview/logservice/logserviece-arch.png)
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/overview/logservice/logserviece-arch.png width=80% heigth=80%/>
</div>

### 客户端

Expand Down Expand Up @@ -90,7 +92,9 @@ Logservice 和 HAKeeper 的状态机都是基于内存的状态机模型,所
5. 当状态机操作完成后,将结果返回给客户端。
6. Follower 节点在接收到来自 leader 的提交索引后,各自执行自己的状态机操作。

![](https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/overview/logservice/write.png)
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/overview/logservice/write.png width=80% heigth=80%/>
</div>

#### 读流程

Expand All @@ -103,7 +107,9 @@ Logservice 和 HAKeeper 的状态机都是基于内存的状态机模型,所
- 等待应用索引(apply index)大于或等于提交索引(commit index)。
- 一旦满足条件,可以读取状态机中的数据,并返回给客户端。

![](https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/overview/logservice/read.png)
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/overview/logservice/read.png width=80% heigth=80%/>
</div>

- 从日志数据库(log db)中读取日志条目。

Expand All @@ -125,7 +131,7 @@ logservice 使用基于内存的状态机,状态机中只记录了一些元数
在这种状态机分离的设计下,简单的快照(snapshot)机制会导致问题:

<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/overview/logservice/truncation-1.png width=50% heigth=50%/>
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/overview/logservice/truncation-1.png width=60% heigth=60%/>
</div>

1. 当 TN 发送一个截断请求时,截断索引(truncate index)设为 100,但此时 logservice 状态机的应用索引(applied index)是 200,即会删除 200 之前的日志,并在该位置生成快照。注意:截断索引不等于应用索引。
Expand All @@ -135,7 +141,9 @@ logservice 使用基于内存的状态机,状态机中只记录了一些元数

为解决上述问题,当前的截断工作流程如下:

![](https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/overview/logservice/truncation-2.png)
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/overview/logservice/truncation-2.png width=70% heigth=70%/>
</div>

1. TN 发送截断请求,更新 logservice 状态机中的截断 LSN(truncateLsn),此时仅更新该值,不执行快照/截断操作。
2. 每个 logservice 服务器内部启动一个截断工作器(truncation worker),定期发送截断请求(Truncate Request)。需要注意的是,该请求中的参数 Exported 设置为 true,表示该快照对系统不可见,仅将快照导出到指定目录下。
Expand Down
4 changes: 3 additions & 1 deletion docs/MatrixOne/Overview/feature/cost-effective.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ MatrixOne 是一款全新设计的数据库,其架构设计理念强调高性

MatrixOne 是专门设计用来解决混合负载问题的数据库。MatrixOne 能够在同一个集群中同时支持 OLTP 和 OLAP,真正实现混合事务/分析处理(Hybrid Transaction and Analytical Processing,HTAP)。用户不再需要分别搭建 OLTP 和 OLAP 两个数据库系统,只需一个数据库就能支持混合负载。这样一来,不仅避免了建设和维护两套系统的成本,还避免了将数据从 OLTP 系统同步到 OLAP 系统的 ETL 过程。用户能够在同一个集群中方便地处理业务和分析。

![](https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/overview/high-cost-performance/HTAP.png)
<div align="center">
<img src=https://community-shared-data-1308875761.cos.ap-beijing.myqcloud.com/artwork/docs/overview/high-cost-performance/HTAP.png width=80% heigth=80%/>
</div>

## 单一存储引擎实现 HTAP

Expand Down
Loading

0 comments on commit 2b9fcfc

Please sign in to comment.