Skip to content

Commit

Permalink
Merge 3.0.0 (#1135)
Browse files Browse the repository at this point in the history
* Rename doc dir (#1032)

* Revert "Rename doc dir (#1032)" (#1033)

This reverts commit 51b6035.

* Update 3.0.0 core comlilation (#1034)

* update balance (#1028) (#1035)

* update balance

* Update docs-2.0/3.ngql-guide/18.operation-and-maintenance-statements/2.balance-syntax.md

Co-authored-by: abby.huang <[email protected]>

* Update docs-2.0/3.ngql-guide/18.operation-and-maintenance-statements/4.job-statements.md

Co-authored-by: abby.huang <[email protected]>

* Update docs-2.0/3.ngql-guide/18.operation-and-maintenance-statements/4.job-statements.md

Co-authored-by: abby.huang <[email protected]>

Co-authored-by: abby.huang <[email protected]>

Co-authored-by: abby.huang <[email protected]>

* Update deploy.yml

* dashboardent1.1.0-alerting-enw (#1012)

* alert&tasks

* new and updates

* Update 10.tasks.md

* updates

* Update 9.alerts.md

* Update 6.global-config.md

* license&dashboard-tasks

* Update 10.tasks.md

* Create 10.tasks.md

* Update 4.manage.md

* comments update

* Update deploy.yml

* Update deploy.yml

* Update deploy.yml

* Update deploy.yml

* Update mkdocs.yml

* Fix edit link

* Update 3.graph-config.md

* Update mkdocs.yml

* Update 2.balance-syntax.md (#1037)

* update cheatsheet (#1036)

* Update mkdocs.yml

* space-level-metrics (#1038)

* Update README.md (#1041)

* remove nebula-http-gateway from dashboard-ent (#1042)

* Update cloud formate (#1043)

* Delete 5.zone.md

* Delete zone1.png

* fix links (#1044)

* Update 3.how-to-set-solution.md

* Fix links2 (#1045)

* fix links

* fix links

* fix typo (#1046)

* update

* Update 3.go.md (#1040)

* update

* version upgrade (#1048)

* Update 4.delete-edge.md

* Update sample.md

* update algorithm (#1049)

* Add ranking config in exchange examples (#1050)

* add license check (#1051)

* Update 2.match.md

* update start

* change start step

* Create nebula-plato.md (#1055)

* Create nebula-plato.md

* plato upgrade

* Update 6.eco-tool-version.md

* Update nebula-plato.md

* Update docs-2.0/nebula-plato.md

Co-authored-by: cooper-lzy <[email protected]>

* add repartitionWithNebula for Exchange (#1054)

* timestamp_in_logfile_name and logrotate faq (#1047)

* Fix exchange conf links (#1056)

* disable balance data (#1058)

* update conifg shell

* update step

* fix schema rule (#1059)

* Update 2.deploy-dashboard.md

* operator docs excluded in 3.0.0 (#1060)

* Update 1.what-is-dashboard-ent.md

* Dashboard ent 220 config file updates  (#1063)

* Dashboard-ent-220-config_file updates

* Update 2.deploy-connect-dashboard-ent.md

* dashboard-ent 110-custom-installation-path (#1065)

* Update 2.deploy-dashboard.md

* delete zone (#1066)

* Update upgrade docs for 3.0.0 (#1069)

* Add caution for date_time_zonespec csv (#1070)

* manage storage hosts (#1068)

* manage storage hosts

* Update docs-2.0/4.deployment-and-installation/manage-storage-host.md

Co-authored-by: randomJoe211 <[email protected]>

* Update docs-2.0/4.deployment-and-installation/manage-storage-host.md

Co-authored-by: randomJoe211 <[email protected]>

* Update docs-2.0/4.deployment-and-installation/manage-storage-host.md

Co-authored-by: randomJoe211 <[email protected]>

* Update docs-2.0/4.deployment-and-installation/manage-storage-host.md

Co-authored-by: randomJoe211 <[email protected]>

* Update manage-storage-host.md

Co-authored-by: randomJoe211 <[email protected]>

* add faq for job type change (#1071)

* Remove zone (#1072)

* Exchange add PostgreSQL data source (#1073)

* show hosts (#1075)

The `show hosts` command displays the version.

* Update 6.eco-tool-version.md

* Update 4.storage-service.md

* Update comments.md

* Update 9.type-conversion.md

* Update 2.boolean.md

* Create 2.boolean.md

* Update 5.case-expressions.md

* Update 1.show-charset.md

* Update 8.show-indexes.md

* Update return.md

* Update ttl-options.md

* Update 1.create-space.md

* Update 3.upsert-vertex.md

* Update 3.upsert-edge.md

* Update 2.show-native-indexes.md

* Update 4.rebuild-native-index.md

* Update 1.resource-preparations.md

* Update 3.deploy-nebula-graph-with-docker-compose.md

* Update 1.resource-preparations.md

* Update 1.authentication.md

* Update compaction.md

* Update compaction.md

* update clients (#1076)

* Update 3.system-design.md

* Update 0.FAQ.md

* Update 0.FAQ.md

* Update how-to-contribute.md

* Update 0.FAQ.md

* Update 0.FAQ.md

* Update 0.FAQ.md

* update docs (#1077)

* Update mkdocs.yml

* Update source_connect-to-nebula-graph.md

* Update source_connect-to-nebula-graph.md

* Update 5.nebula-python-client.md

* update-studio (#1078)

* update-studio

* Update mkdocs.yml

* Update st-ug-deploy.md

* Explorer 220 content&updates (#1084)

* explorer 220 intro & pic updates

* Update mkdocs.yml

* Update 6.show-hosts.md

* add gifs for dashboard 110 (#1089)

* explorer220-graph exploration (#1088)

* visualization mode (#1081)

* explorer 220 shortcuts (#1079)

* Create explorer-console.md (#1080)

* gif for exploration (#1090)

* canvas-overview (#1087)

* explorer 220-pageoverview (#1086)

* explorer220-start query (#1085)

* explorer220 filtering (#1083)

* canvas-snapshot (#1082)

* Update 4.storage-service.md

* macro change (#1092)

* Update 2.deploy-dashboard.md (#1093)

* Update 2.deploy-dashboard.md

* dashboard community updates (#1095)

* update release notes (#1096)

* update release notes

* Update releasenote.md

* Update docs-2.0/20.appendix/releasenote.md

* Update docs-2.0/20.appendix/releasenote.md

* Update docs-2.0/20.appendix/releasenote.md

* Update docs-2.0/20.appendix/releasenote.md

* Update docs-2.0/20.appendix/releasenote.md

* Update docs-2.0/20.appendix/releasenote.md

* Update docs-2.0/20.appendix/releasenote.md

* Update docs-2.0/20.appendix/releasenote.md

* Update docs-2.0/20.appendix/releasenote.md

* Update docs-2.0/20.appendix/releasenote.md

* Update docs-2.0/20.appendix/releasenote.md

* Update releasenote.md

* Update docs-2.0/20.appendix/releasenote.md

* Update releasenote.md

* Update releasenote.md

Co-authored-by: abby.huang <[email protected]>

* Update visualization-mode.md

* Update deploy.yml

* Update releasenote.md

* Update 2.deploy-connect-dashboard-ent.md

* fixed some issues (#1098)

* Update releasenote.md

* Update releasenote.md

* Update mkdocs.yml

* Update 2.deploy-connect-dashboard-ent.md

* Create 2.deploy-connect-dashboard-ent.md

* Update releasenote.md

* Update mkdocs.yml

* update compatibility and branch (#1099)

* Update nebula-plato.md

* Update nebula-plato.md

* Update st-ug-deploy.md

* Update st-ug-release-note.md

* update br (#1101)

* Update ex-ug-limitations.md

* Update mkdocs.yml

* Update use-importer.md

* Update nebula-algorithm.md

* Update config-with-header.md

* Update config-without-header.md

* Update releasenote.md

* Update st-ug-what-is-graph-studio.md

* Update nebula-algorithm.md

* Update nebula-spark-connector.md

* update exchange 3.0.0 (#1103)

* Update studio and format (#1104)

* Update 2.deploy-connect-dashboard-ent.md

* change plato name (#1102)

* add backup&restore (#1100)

* add backup&restore

* Update 2.compile-br.md

* Update docs-2.0/7.data-security/2.backup-restore/4.br-restore-data.md

Co-authored-by: abby.huang <[email protected]>

* Update docs-2.0/7.data-security/2.backup-restore/4.br-restore-data.md

Co-authored-by: abby.huang <[email protected]>

* Update docs-2.0/7.data-security/2.backup-restore/4.br-restore-data.md

Co-authored-by: abby.huang <[email protected]>

* Update docs-2.0/7.data-security/2.backup-restore/4.br-restore-data.md

Co-authored-by: abby.huang <[email protected]>

* Update docs-2.0/7.data-security/2.backup-restore/4.br-restore-data.md

Co-authored-by: abby.huang <[email protected]>

* Update 4.br-restore-data.md

Co-authored-by: abby.huang <[email protected]>

* Update nebula-analytics.md

* Update nebula-analytics.md

* Update 3.br-backup-data.md

* Update 4.br-restore-data.md

* Update nebula-analytics.md

* Update releasenote.md

* remove br

* Update releasenote.md

* Update releasenote.md

* Update releasenote.md

* Update releasenote.md

* Update releasenote.md

* Create releasenote.md

* Update learning-path.md (#1105)

* update studio 3.2.1 (#1106)

* link and webpage logo error fix (#1107)

* update alter tag (#1108)

* update quick start (#1111)

* Update 0.FAQ.md

* Update 1.overview.md (#1112)

* Update 5.lookup.md

* Update 1.create-native-index.md

* Update 0.FAQ.md

* Note for basketallplayer (#1113)

* note for basketballplayer

* Update 1.overview.md

* No auto backup (#1114)

* No auto backup

* Update upgrade-nebula-graph-to-latest.md

* Update 0.FAQ.md (#1115)

* Update 1.what-is-br.md

* Update 8.faq.md (#1116)

* Update manage-storage-host.md (#1117)

* Update 3.graph-service.md

* Update 6.eco-tool-version.md (#1118)

* Update deploy.yml

* Explain data structure change in 3.0 (#1119)

* Explain data structure change in 3.0

* Update 4.storage-service.md

* Update 4.storage-service.md

* Update mkdocs.yml

* Update nebula-analytics.md

* Update requirements.txt

* Update mkdocs.yml

* Update requirements.txt

* add nebula-graph-history (#1120)

* Update 1.create-native-index.md

* fix analytics doc (#1121)

* Update mkdocs.yml

* Update nebula-analytics.md

* Update mkdocs.yml

* Update nebula-analytics.md

* Update 2.match.md

* Update visualization-mode.md (#1122)

* update full-text index (#1123)

* Fix exchange config files (#1124)

* Update 3.role-list.md

* Create 0-0-graph.md (#1125)

* Update keywords-and-reserved-words.md

* Update 3.deploy-nebula-graph-with-docker-compose.md (#1127)

* create -related technologies (#1126)

* Update 1.query-performance-metrics.md

* Update source_install-nebula-graph-by-rpm-or-deb.md

* Update mkdocs.yml (#1130)

* Create 0-1-graph-database.md (#1129)

* Update studio (#1131)

* Update Nebula Graph birdview (#1133)

Co-authored-by: randomJoe211 <[email protected]>
Co-authored-by: abby.huang <[email protected]>
Co-authored-by: min.wu <[email protected]>
Co-authored-by: foesa <[email protected]>
Co-authored-by: li Nico <[email protected]>
Co-authored-by: Nut He <[email protected]>
  • Loading branch information
7 people authored Mar 8, 2022
1 parent d113b3b commit 5b84c21
Show file tree
Hide file tree
Showing 46 changed files with 987 additions and 216 deletions.
210 changes: 210 additions & 0 deletions docs-2.0/1.introduction/0-0-graph.md

Large diffs are not rendered by default.

241 changes: 241 additions & 0 deletions docs-2.0/1.introduction/0-1-graph-database.md

Large diffs are not rendered by default.

253 changes: 253 additions & 0 deletions docs-2.0/1.introduction/0-2.relates.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs-2.0/1.introduction/1.what-is-nebula-graph.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Nebula Graph is an open-source, distributed, easily scalable, and native graph database. It is capable of hosting graphs with hundreds of billions of vertices and trillions of edges, and serving queries with millisecond-latency.

![Nebula Graph birdview](nebula-birdview.png)
![Nebula Graph birdview](nebula-graph-birdview-3.0.png)

## What is a graph database

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,10 +84,6 @@ In the `nebula-graphd.conf` file, when `enable_optimizer` is set to be `true`, P

As shown in the preceding figure, when the `Filter` node is explored, it is found that its children node is `GetNeighbors`, which matches successfully with the pre-defined rules, so a transformation is initiated to integrate the `Filter` node into the `GetNeighbors` node, the `Filter` node is removed, and then the process continues to the next rule. Therefore, when the `GetNeighbor` operator calls interfaces of the Storage layer to get the neighboring edges of a vertex during the execution stage, the Storage layer will directly filter out the unqualified edges internally. Such optimization greatly reduces the amount of data transfer, which is commonly known as filter pushdown.

!!! Note

Nebula Graph {{ nebula.release }} will not run optimization by default.

## Executor

The Executor module consists of Scheduler and Executor. The Scheduler generates the corresponding execution operators against the execution plan, starting from the leaf nodes and ending at the root node. The structure is as follows.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,37 +70,37 @@ Therefore, Nebula Graph develops its own KVStore with RocksDB as the local stora

- One Nebula Graph KVStore cluster supports multiple graph spaces, and each graph space has its own partition number and replica copies. Different graph spaces are isolated physically from each other in the same cluster.

<!--
## Data storage formats
## Data storage structure

Nebula Graph stores vertices and edges. Efficient property filtering is critical for a Graph Database. So, Nebula Graph uses keys to store vertices and edges, while uses values to store the related properties.
Graphs consist of vertices and edges. Nebula Graph uses key-value pairs to store vertices, edges, and their properties. Vertices and edges are stored in keys and their properties are stored in values. Such structure enables efficient property filtering.

Nebula Graph {{ nebula.base20 }} has changed a lot over its releases. The following will introduce the old and new data storage formats and cover their differences.
- The storage structure of vertices

- Vertex format
Different from Nebula Graph version 2.x, version 3.x added a new key for each vertex. Compared to the old key that still exists, the new key has no `TagID` field and no value. Vertices in Nebula Graph can now live without tags owing to the new key.

![The vertex format of storage service](https://docs-cdn.nebula-graph.com.cn/docs-2.0/1.introduction/2.nebula-graph-architecture/storage-vertex-format.png)
![The vertex structure of Nebula Graph](https://github.com/vesoft-inc/nebula-docs-cn/blob/{{nebula.branch}}/docs-2.0/1.introduction/3.nebula-graph-architecture/3.0-vertex-key.png?raw=true)

|Field|Description|
|:---|:---|
|`Type`|One byte, used to indicate the key type.|
|`PartID`|Three bytes, used to indicate the sharding partition and to scan the partition data based on the prefix when re-balancing the partition.|
|`VertexID`|Used to indicate vertex ID. For an integer VertexID, it occupies eight bytes. However, for a string VertexID, it is changed to `fixed_string` of a fixed length which needs to be specified by users when they create the space.|
|`VertexID`|The vertex ID. For an integer VertexID, it occupies eight bytes. However, for a string VertexID, it is changed to `fixed_string` of a fixed length which needs to be specified by users when they create the space.|
|`TagID`|Four bytes, used to indicate the tags that vertex relate with.|
|`SerializedValue`|The serialized value of the key. It stores the property information of the vertex.|

- Edge Format
- The storage structure of edges

![The edge format of storage service](https://docs-cdn.nebula-graph.com.cn/docs-2.0/1.introduction/2.nebula-graph-architecture/storage-edge-format.png)
![The edge structure of Nebula Graph](https://github.com/vesoft-inc/nebula-docs-cn/blob/{{nebula.branch}}/docs-2.0/1.introduction/3.nebula-graph-architecture/3.0-edge-key.png?raw=true)

|Field|Description|
|:---|:---|
|`Type`|One byte, used to indicate the key type.|
|`PartID`|Three bytes, used to indicate the sharding partition. This field can be used to scan the partition data based on the prefix when re-balancing the partition.|
|`VertexID`|Used to indicate vertex ID. The former VID refers to source VID in out-edge and dest VID in in-edge, while the latter VID refers to dest VID in out-edge and source VID in in-edge.|
|`Edge Type`|Four bytes, used to indicate edge type. Greater than zero means out-edge, less than zero means in-edge.|
|`PartID`|Three bytes, used to indicate the partition ID. This field can be used to scan the partition data based on the prefix when re-balancing the partition.|
|`VertexID`|Used to indicate vertex ID. The former VID refers to the source VID in the outgoing edge and the dest VID in the incoming edge, while the latter VID refers to the dest VID in the outgoing edge and the source VID in the incoming edge.|
|`Edge Type`|Four bytes, used to indicate the edge type. Greater than zero indicates out-edge, less than zero means in-edge.|
|`Rank`|Eight bytes, used to indicate multiple edges in one edge type. Users can set the field based on needs and store weight, such as transaction time and transaction number.|
|`PlaceHolder`|One byte. Reserved.|
-->
|`SerializedValue`|The serialized value of the key. It stores the property information of the edge.|

### Property descriptions

Expand All @@ -114,12 +114,11 @@ Since in an ultra-large-scale relational network, vertices can be as many as ten

![data partitioning](https://www-cdn.nebula-graph.com.cn/nebula-blog/DataModel02.png)

<!--
### Edge and storage amplification
### Edge partitioning and storage amplification

In Nebula Graph, an edge corresponds to two key-value pairs on the hard disk. When there are lots of edges and each has many properties, storage amplification will be obvious. The storage format of edges is shown in the figure below.

![edge storage](https://docs-cdn.nebula-graph.com.cn/docs-2.0/1.introduction/2.nebula-graph-architecture/two-edge-format.png)
![partitioning by edge](https://github.com/vesoft-inc/nebula-docs-cn/blob/{{nebula.branch}}/docs-2.0/1.introduction/3.nebula-graph-architecture/edge-division.png?raw=true)

In this example, ScrVertex connects DstVertex via EdgeA, forming the path of `(SrcVertex)-[EdgeA]->(DstVertex)`. ScrVertex, DstVertex, and EdgeA will all be stored in Partition x and Partition y as four key-value pairs in the storage layer. Details are as follows:

Expand Down
Binary file removed docs-2.0/1.introduction/nebula-birdview.png
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 5b84c21

Please sign in to comment.