Skip to content

Commit

Permalink
opts on resource requirements (#2595)
Browse files Browse the repository at this point in the history
  • Loading branch information
abby-cyber authored Dec 19, 2024
1 parent 155cc10 commit d792289
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 8 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ Users can estimate the memory, disk space, and partition number needed for a Neb
|:--- |:---|:--- |:---|
| Disk space for a cluster |Bytes| `the_sum_of_edge_number_and_vertex_number` * `average_bytes_of_properties` * 7.5 * 120% |For more information, see [Edge partitioning and storage amplification](../1.introduction/3.nebula-graph-architecture/4.storage-service.md).|
| Memory for a cluster |Bytes| [`the_sum_of_edge_number_and_vertex_number` * 16 + `the_number_of_RocksDB_instances` * (`write_buffer_size` * `max_write_buffer_number`) + `rocksdb_block_cache`] * 120% |`write_buffer_size` and `max_write_buffer_number` are RocksDB parameters. For more information, see [MemTable](https://github.com/facebook/rocksdb/wiki/MemTable). For details about `rocksdb_block_cache`, see [Memory usage in RocksDB](https://github.com/facebook/rocksdb/wiki/Memory-usage-in-RocksDB#block-cache).|
| Number of partitions for a graph space |-| `the_number_of_disks_in_the_cluster` * `disk_partition_num_multiplier` |`disk_partition_num_multiplier` is an integer between 2 and 20 (both including). Its value depends on the disk performance. Use 20 for SSD and 2 for HDD.|
| Number of partitions for a graph space |-| `the_number_of_disks_in_the_cluster` * `multiplier` |`multiplier` is an integer between 2 and 20 (both including). Its value depends on the disk performance. Use 20 for SSD and 2 for HDD.|


* Question 1: Why do I need to multiply by 7.5 in the disk space estimation formula?
Expand All @@ -246,8 +246,7 @@ Users can estimate the memory, disk space, and partition number needed for a Neb

* Question 3: How to get the number of RocksDB instances?

Answer: Each graph space corresponds to one RocksDB instance and each directory in the `--data_path` item in the `etc/nebula-storaged.conf` file corresponds to one RocksDB instance.
That is, the number of RocksDB instances = the number of directories * the number of graph spaces.
Answer: The number of RocksDB instances = the number of graph spaces * the total number of directories specified by the `--data_path` parameter across all Storage services. Each graph space corresponds to one RocksDB instance and each directory in the `--data_path` parameter in the `etc/nebula-storaged.conf` file corresponds to one RocksDB instance.

!!! note

Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@

# 准备编译、安装和运行 {{nebula.name}} 的环境




本文介绍编译、安装 {{nebula.name}} 的要求和建议,以及如何预估集群运行所需的资源。

## 关于存储硬件
Expand Down Expand Up @@ -222,7 +219,7 @@ storaged 进程的数量不会影响图空间副本的数量。
|:--- |:---|:--- |:---|
| 硬盘空间 |Bytes| `点和边的总数` * `属性的平均字节大小` * 7.5 * 120% |由于边存在存储放大现象,所以需要`点和边的总数` * `属性的平均字节大小` * 7.5 的空间,详情请参见[切边与存储放大](../1.introduction/3.nebula-graph-architecture/4.storage-service.md)。|
| 内存 |Bytes| [`点和边的总数` * 16 + `RocksDB 实例数量` * (`write_buffer_size` * `max_write_buffer_number`) + `块缓存大小`] * 120% |`点和边的总数` * 16 是 [BloomFilter](https://zh.wikipedia.org/wiki/%E5%B8%83%E9%9A%86%E8%BF%87%E6%BB%A4%E5%99%A8) 需要占用的内存空间,`write_buffer_size``max_write_buffer_number`是 RocksDB 内存相关参数,详情请参见 [MemTable](https://github.com/facebook/rocksdb/wiki/MemTable)。块缓存大小请参见 [Memory usage in RocksDB](https://github.com/facebook/rocksdb/wiki/Memory-usage-in-RocksDB#block-cache)。|
| 分区数量 |-| `集群硬盘数量` * `disk_partition_num_multiplier` |`disk_partition_num_multiplier`是一个用于衡量硬盘性能的整数,取值范围 2~20。建议在计算 SSD 硬盘的分区数量时使用 20 做参数值,HDD 硬盘使用 2。|
| 图空间的分片数量 |-| `集群硬盘数量` * `乘数` |`乘数`是一个用于衡量硬盘性能的整数,取值范围 2~20。建议在计算 SSD 硬盘的分区数量时使用 20 做参数值,HDD 硬盘使用 2。|

- 问题 1:为什么硬盘空间的预估公式中需要乘以 7.5?

Expand All @@ -234,7 +231,7 @@ storaged 进程的数量不会影响图空间副本的数量。

- 问题 3:如何获取 RocksDB 实例数量?

答:对于社区版 {{nebula.name}},每个图空间对应一个 RocksDB 实例,并且`--data_path`选项`etc`目录下的`nebula-storaged.conf`文件中)中的每个目录对应一个 RocksDB 实例。即,RocksDB 实例数量 = 图空间总数 * 目录总数
答:RocksDB 实例数量 = 图空间总数 * 所有 Storage 服务`--data_path`参数指定的目录数之和。对于社区版 {{nebula.name}},每个图空间对应一个 RocksDB 实例,并且`--data_path`参数`etc`目录下的`nebula-storaged.conf`文件中)中的每个目录对应一个 RocksDB 实例。

!!! note

Expand Down

0 comments on commit d792289

Please sign in to comment.