diff --git a/docs-2.0/3.ngql-guide/15.full-text-index-statements/1.search-with-text-based-index.md b/docs-2.0/3.ngql-guide/15.full-text-index-statements/1.search-with-text-based-index.md index 4be371d4e7f..87368d9daaf 100644 --- a/docs-2.0/3.ngql-guide/15.full-text-index-statements/1.search-with-text-based-index.md +++ b/docs-2.0/3.ngql-guide/15.full-text-index-statements/1.search-with-text-based-index.md @@ -1,235 +1,416 @@ # Full-text indexes -Full-text indexes are used to do prefix, wildcard, regexp, and fuzzy search on a string property. +=== "NebulaGraph Community" -You can use the `WHERE` clause to specify the search strings in `LOOKUP` statements. + Full-text indexes are used to do prefix, wildcard, regexp, and fuzzy search on a string property. -## Prerequisite + You can use the `WHERE` clause to specify the search strings in `LOOKUP` statements. -Before using the full-text index, make sure that you have deployed a Elasticsearch cluster and a Listener cluster. For more information, see [Deploy Elasticsearch](../../4.deployment-and-installation/6.deploy-text-based-index/2.deploy-es.md) and [Deploy Listener](../../4.deployment-and-installation/6.deploy-text-based-index/3.deploy-listener.md). + ## Prerequisite -## Precaution + Before using the full-text index, make sure that you have deployed a Elasticsearch cluster and a Listener cluster. For more information, see [Deploy Elasticsearch](../../4.deployment-and-installation/6.deploy-text-based-index/2.deploy-es.md) and [Deploy Listener](../../4.deployment-and-installation/6.deploy-text-based-index/3.deploy-listener.md). -Before using the full-text index, make sure that you know the [restrictions](../../4.deployment-and-installation/6.deploy-text-based-index/1.text-based-index-restrictions.md). + ## Precaution -## Full Text Queries + Before using the full-text index, make sure that you know the [restrictions](../../4.deployment-and-installation/6.deploy-text-based-index/1.text-based-index-restrictions.md). -Full-text queries enable you to search for parsed text fields, using a parser with strict syntax to return content based on the query string provided. For details, see [Query string query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#query-dsl-query-string-query). + ## Natural language full-text search -## Syntax + A natural language search interprets the search string as a phrase in natural human language. The search is case-sensitive and by default prefixes the string with a match. For example, there are three vertices with the tag `player`. The tag `player` contains the property `name`. The `name` of these three vertices are `Kevin Durant`, `Tim Duncan`, and `David Beckham`. Now that the full-text index of `player.name` is established, only `David Beckham` will be queried when using the prefix search statement `LOOKUP ON player WHERE PREFIX(player.name,"D");`. -### Create full-text indexes + ## Syntax -```ngql -CREATE FULLTEXT {TAG | EDGE} INDEX ON { | } ( [,]...) [ANALYZER=""]; -``` + ### Create full-text indexes -- Composite indexes with multiple properties are supported when creating full-text indexes. -- `` is the name of the analyzer. The default value is `standard`. To use other analyzers (e.g. [IK Analysis](https://github.com/medcl/elasticsearch-analysis-ik)), you need to make sure that the corresponding analyzer is installed in Elasticsearch in advance. + ```ngql + CREATE FULLTEXT {TAG | EDGE} INDEX ON { | } ([]); + ``` -### Show full-text indexes + ### Show full-text indexes -```ngql -SHOW FULLTEXT INDEXES; -``` + ```ngql + SHOW FULLTEXT INDEXES; + ``` -### Rebuild full-text indexes + ### Rebuild full-text indexes -```ngql -REBUILD FULLTEXT INDEX; -``` + ```ngql + REBUILD FULLTEXT INDEX; + ``` -!!! caution - When there is a large amount of data, rebuilding full-text index is slow, you can modify `snapshot_send_files=false` in the configuration file of Storage service(`nebula-storaged.conf`). + !!! caution + When there is a large amount of data, rebuilding full-text index is slow, you can modify `snapshot_send_files=false` in the configuration file of Storage service(`nebula-storaged.conf`). -### Drop full-text indexes + ### Drop full-text indexes -```ngql -DROP FULLTEXT INDEX ; -``` + ```ngql + DROP FULLTEXT INDEX ; + ``` -### Use query options + ### Use query options -```ngql -LOOKUP ON { | } WHERE ES_QUERY(, "") YIELD [| LIMIT [,] ]; + ```ngql + LOOKUP ON { | } WHERE [YIELD ]; - - [AS ] [, [AS ] ...] [, id(vertex) [AS ]] [, score() AS ] -``` + ::= + PREFIX | WILDCARD | REGEXP | FUZZY -- `index_name`: The name of the full-text index. + + [AS ] [, [AS ] ...] + ``` -- `text`: Search conditions. For supported syntax, see [Query string syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#query-string-syntax). + - PREFIX(schema_name.prop_name, prefix_string, row_limit, timeout) -- `score()`: The score calculated by doing N degree expansion for the eligible vertices. The default value is `1.0`. The higher the score, the higher the degree of match. The return value is sorted by default from highest to lowest score. For details, see [Search and Scoring in Lucene](https://lucene.apache.org/core/9_6_0/core/org/apache/lucene/search/package-summary.html#package.description). + - WILDCARD(schema_name.prop_name, wildcard_string, row_limit, timeout) -## Examples + - REGEXP(schema_name.prop_name, regexp_string, row_limit, timeout) -```ngql -// This example creates the graph space. -nebula> CREATE SPACE IF NOT EXISTS basketballplayer (partition_num=3,replica_factor=1, vid_type=fixed_string(30)); + - FUZZY(schema_name.prop_name, fuzzy_string, fuzziness, operator, row_limit, timeout) -// This example signs in the text service. -nebula> SIGN IN TEXT SERVICE (192.168.8.100:9200, HTTP); - -// This example checks the text service status. -nebula> SHOW TEXT SEARCH CLIENTS; -+-----------------+-----------------+------+ -| Type | Host | Port | -+-----------------+-----------------+------+ -| "ELASTICSEARCH" | "192.168.8.100" | 9200 | -+-----------------+-----------------+------+ - -// This example switches the graph space. -nebula> USE basketballplayer; - -// This example adds the listener to the NebulaGraph cluster. -nebula> ADD LISTENER ELASTICSEARCH 192.168.8.100:9789; - -// This example checks the listener status. When the status is `Online`, the listener is ready. -nebula> SHOW LISTENER; -+--------+-----------------+------------------------+-------------+ -| PartId | Type | Host | Host Status | -+--------+-----------------+------------------------+-------------+ -| 1 | "ELASTICSEARCH" | ""192.168.8.100":9789" | "ONLINE" | -| 2 | "ELASTICSEARCH" | ""192.168.8.100":9789" | "ONLINE" | -| 3 | "ELASTICSEARCH" | ""192.168.8.100":9789" | "ONLINE" | -+--------+-----------------+------------------------+-------------+ - -// This example creates the tag. -nebula> CREATE TAG IF NOT EXISTS player(name string, city string); - -// This example creates a single-attribute full-text index. -nebula> CREATE FULLTEXT TAG INDEX fulltext_index_1 ON player(name) ANALYZER="standard"; - -// This example creates a multi-attribute full-text indexe. -nebula> CREATE FULLTEXT TAG INDEX fulltext_index_2 ON player(name,city) ANALYZER="standard"; - -// This example rebuilds the full-text index. -nebula> REBUILD FULLTEXT INDEX; - -// This example shows the full-text index. -nebula> SHOW FULLTEXT INDEXES; -+--------------------+-------------+-------------+--------------+------------+ -| Name | Schema Type | Schema Name | Fields | Analyzer | -+--------------------+-------------+-------------+--------------+------------+ -| "fulltext_index_1" | "Tag" | "player" | "name" | "standard" | -| "fulltext_index_2" | "Tag" | "player" | "name, city" | "standard" | -+--------------------+-------------+-------------+--------------+------------+ - -// This example inserts the test data. -nebula> INSERT VERTEX player(name, city) VALUES \ - "Russell Westbrook": ("Russell Westbrook", "Los Angeles"), \ - "Chris Paul": ("Chris Paul", "Houston"),\ - "Boris Diaw": ("Boris Diaw", "Houston"),\ - "David West": ("David West", "Philadelphia"),\ - "Danny Green": ("Danny Green", "Philadelphia"),\ - "Tim Duncan": ("Tim Duncan", "New York"),\ - "James Harden": ("James Harden", "New York"),\ - "Tony Parker": ("Tony Parker", "Chicago"),\ - "Aron Baynes": ("Aron Baynes", "Chicago"),\ - "Ben Simmons": ("Ben Simmons", "Phoenix"),\ - "Blake Griffin": ("Blake Griffin", "Phoenix"); - -// These examples run test queries. -nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"Chris") YIELD id(vertex); -+--------------+ -| id(VERTEX) | -+--------------+ -| "Chris Paul" | -+--------------+ - -nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"Harden") YIELD properties(vertex); -+----------------------------------------------------------------+ -| properties(VERTEX) | -+----------------------------------------------------------------+ -| {_vid: "James Harden", city: "New York", name: "James Harden"} | -+----------------------------------------------------------------+ - -nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"Da*") YIELD properties(vertex); -+------------------------------------------------------------------+ -| properties(VERTEX) | -+------------------------------------------------------------------+ -| {_vid: "David West", city: "Philadelphia", name: "David West"} | -| {_vid: "Danny Green", city: "Philadelphia", name: "Danny Green"} | -+------------------------------------------------------------------+ - -nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"*b*") YIELD id(vertex); -+---------------------+ -| id(VERTEX) | -+---------------------+ -| "Russell Westbrook" | -| "Boris Diaw" | -| "Aron Baynes" | -| "Ben Simmons" | -| "Blake Griffin" | -+---------------------+ - -nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"*b*") YIELD id(vertex) | LIMIT 2,3; -+-----------------+ -| id(VERTEX) | -+-----------------+ -| "Aron Baynes" | -| "Ben Simmons" | -| "Blake Griffin" | -+-----------------+ - -nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"*b*") YIELD id(vertex) | YIELD count(*); -+----------+ -| count(*) | -+----------+ -| 5 | -+----------+ - -nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"*b*") YIELD id(vertex), score() AS score; -+---------------------+-------+ -| id(VERTEX) | score | -+---------------------+-------+ -| "Russell Westbrook" | 1.0 | -| "Boris Diaw" | 1.0 | -| "Aron Baynes" | 1.0 | -| "Ben Simmons" | 1.0 | -| "Blake Griffin" | 1.0 | -+---------------------+-------+ - -// For documents containing a word `b`, its score will be multiplied by a weighting factor of 4, while for documents containing a word `c`, the default weighting factor of 1 is used. -nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"*b*^4 OR *c*") YIELD id(vertex), score() AS score; -+---------------------+-------+ -| id(VERTEX) | score | -+---------------------+-------+ -| "Russell Westbrook" | 4.0 | -| "Boris Diaw" | 4.0 | -| "Aron Baynes" | 4.0 | -| "Ben Simmons" | 4.0 | -| "Blake Griffin" | 4.0 | -| "Chris Paul" | 1.0 | -| "Tim Duncan" | 1.0 | -+---------------------+-------+ - -// When using a multi-attribute full-text index query, the conditions are matched within all properties of the index. -nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_2,"*h*") YIELD properties(vertex); -+------------------------------------------------------------------+ -| properties(VERTEX) | -+------------------------------------------------------------------+ -| {_vid: "Chris Paul", city: "Houston", name: "Chris Paul"} | -| {_vid: "Boris Diaw", city: "Houston", name: "Boris Diaw"} | -| {_vid: "David West", city: "Philadelphia", name: "David West"} | -| {_vid: "James Harden", city: "New York", name: "James Harden"} | -| {_vid: "Tony Parker", city: "Chicago", name: "Tony Parker"} | -| {_vid: "Aron Baynes", city: "Chicago", name: "Aron Baynes"} | -| {_vid: "Ben Simmons", city: "Phoenix", name: "Ben Simmons"} | -| {_vid: "Blake Griffin", city: "Phoenix", name: "Blake Griffin"} | -| {_vid: "Danny Green", city: "Philadelphia", name: "Danny Green"} | -+------------------------------------------------------------------+ - -// When using multi-attribute full-text index queries, you can specify different text for different properties for the query. -nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_2,"name:*b* AND city:Houston") YIELD properties(vertex); -+-----------------------------------------------------------+ -| properties(VERTEX) | -+-----------------------------------------------------------+ -| {_vid: "Boris Diaw", city: "Houston", name: "Boris Diaw"} | -+-----------------------------------------------------------+ - -// Delete single-attribute full-text index. -nebula> DROP FULLTEXT INDEX fulltext_index_1; -``` + - `fuzziness` (optional): Maximum edit distance allowed for matching. The default value is `AUTO`. For other valid values and more information, see [Elasticsearch document](https://www.elastic.co/guide/en/elasticsearch/reference/6.8/common-options.html#fuzziness). + + - `operator` (optional): Boolean logic used to interpret the text. Valid values are `OR` (default) and `AND`. + + - `row_limit` (optional): Specifies the number of rows to return. The default value is `100`. + + - `timeout` (optional): Specifies the timeout time. The default value is `200ms`. + + ## Examples + + ```ngql + // This example creates the graph space. + nebula> CREATE SPACE IF NOT EXISTS basketballplayer (partition_num=3,replica_factor=1, vid_type=fixed_string(30)); + + // This example signs in the text service. + nebula> SIGN IN TEXT SERVICE (127.0.0.1:9200, HTTP); + + // This example checks the text service status. + nebula> SHOW TEXT SEARCH CLIENTS; + + // This example switches the graph space. + nebula> USE basketballplayer; + + // This example adds the listener to the NebulaGraph cluster. + nebula> ADD LISTENER ELASTICSEARCH 192.168.8.5:9789; + + // This example checks the listener status. When the status is `Online`, the listener is ready. + nebula> SHOW LISTENER; + + // This example creates the tag. + nebula> CREATE TAG IF NOT EXISTS player(name string, age int); + + // This example creates the full-text index. The index name starts with "nebula_". + nebula> CREATE FULLTEXT TAG INDEX nebula_index_1 ON player(name); + + // This example rebuilds the full-text index. + nebula> REBUILD FULLTEXT INDEX; + + // This example shows the full-text index. + nebula> SHOW FULLTEXT INDEXES; + +------------------+-------------+-------------+--------+ + | Name | Schema Type | Schema Name | Fields | + +------------------+-------------+-------------+--------+ + | "nebula_index_1" | "Tag" | "player" | "name" | + +------------------+-------------+-------------+--------+ + + // This example inserts the test data. + nebula> INSERT VERTEX player(name, age) VALUES \ + "Russell Westbrook": ("Russell Westbrook", 30), \ + "Chris Paul": ("Chris Paul", 33),\ + "Boris Diaw": ("Boris Diaw", 36),\ + "David West": ("David West", 38),\ + "Danny Green": ("Danny Green", 31),\ + "Tim Duncan": ("Tim Duncan", 42),\ + "James Harden": ("James Harden", 29),\ + "Tony Parker": ("Tony Parker", 36),\ + "Aron Baynes": ("Aron Baynes", 32),\ + "Ben Simmons": ("Ben Simmons", 22),\ + "Blake Griffin": ("Blake Griffin", 30); + + // These examples run test queries. + nebula> LOOKUP ON player WHERE PREFIX(player.name, "B") YIELD id(vertex); + +-----------------+ + | id(VERTEX) | + +-----------------+ + | "Boris Diaw" | + | "Ben Simmons" | + | "Blake Griffin" | + +-----------------+ + + nebula> LOOKUP ON player WHERE WILDCARD(player.name, "*ri*") YIELD player.name, player.age; + +-----------------+-----+ + | name | age | + +-----------------+-----+ + | "Chris Paul" | 33 | + | "Boris Diaw" | 36 | + | "Blake Griffin" | 30 | + +-----------------+-----+ + + nebula> LOOKUP ON player WHERE WILDCARD(player.name, "*ri*") YIELD player.name, player.age | YIELD count(*); + +----------+ + | count(*) | + +----------+ + | 3 | + +----------+ + + nebula> LOOKUP ON player WHERE REGEXP(player.name, "R.*") YIELD player.name, player.age; + +---------------------+-----+ + | name | age | + +---------------------+-----+ + | "Russell Westbrook" | 30 | + +---------------------+-----+ + + nebula> LOOKUP ON player WHERE REGEXP(player.name, ".*") YIELD id(vertex); + +---------------------+ + | id(VERTEX) | + +---------------------+ + | "Danny Green" | + | "David West" | + ... + + nebula> LOOKUP ON player WHERE FUZZY(player.name, "Tim Dunncan", AUTO, OR) YIELD player.name; + +--------------+ + | name | + +--------------+ + | "Tim Duncan" | + +--------------+ + + // This example drops the full-text index. + nebula> DROP FULLTEXT INDEX nebula_index_1; + ``` + +=== "NebulaGraph Enterprise" + + Full-text indexes are used to do prefix, wildcard, regexp, and fuzzy search on a string property. + + You can use the `WHERE` clause to specify the search strings in `LOOKUP` statements. + + ## Prerequisite + + Before using the full-text index, make sure that you have deployed a Elasticsearch cluster and a Listener cluster. For more information, see [Deploy Elasticsearch](../../4.deployment-and-installation/6.deploy-text-based-index/2.deploy-es.md) and [Deploy Listener](../../4.deployment-and-installation/6.deploy-text-based-index/3.deploy-listener.md). + + ## Precaution + + Before using the full-text index, make sure that you know the [restrictions](../../4.deployment-and-installation/6.deploy-text-based-index/1.text-based-index-restrictions.md). + + ## Full Text Queries + + Full-text queries enable you to search for parsed text fields, using a parser with strict syntax to return content based on the query string provided. For details, see [Query string query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#query-dsl-query-string-query). + + ## Syntax + + ### Create full-text indexes + + ```ngql + CREATE FULLTEXT {TAG | EDGE} INDEX ON { | } ( [,]...) [ANALYZER=""]; + ``` + + - Composite indexes with multiple properties are supported when creating full-text indexes. + - `` is the name of the analyzer. The default value is `standard`. To use other analyzers (e.g. [IK Analysis](https://github.com/medcl/elasticsearch-analysis-ik)), you need to make sure that the corresponding analyzer is installed in Elasticsearch in advance. + + ### Show full-text indexes + + ```ngql + SHOW FULLTEXT INDEXES; + ``` + + ### Rebuild full-text indexes + + ```ngql + REBUILD FULLTEXT INDEX; + ``` + + !!! caution + When there is a large amount of data, rebuilding full-text index is slow, you can modify `snapshot_send_files=false` in the configuration file of Storage service(`nebula-storaged.conf`). + + ### Drop full-text indexes + + ```ngql + DROP FULLTEXT INDEX ; + ``` + + ### Use query options + + ```ngql + LOOKUP ON { | } WHERE ES_QUERY(, "") YIELD [| LIMIT [,] ]; + + + [AS ] [, [AS ] ...] [, id(vertex) [AS ]] [, score() AS ] + ``` + + - `index_name`: The name of the full-text index. + + - `text`: Search conditions. For supported syntax, see [Query string syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#query-string-syntax). + + - `score()`: The score calculated by doing N degree expansion for the eligible vertices. The default value is `1.0`. The higher the score, the higher the degree of match. The return value is sorted by default from highest to lowest score. For details, see [Search and Scoring in Lucene](https://lucene.apache.org/core/9_6_0/core/org/apache/lucene/search/package-summary.html#package.description). + + ## Examples + + ```ngql + // This example creates the graph space. + nebula> CREATE SPACE IF NOT EXISTS basketballplayer (partition_num=3,replica_factor=1, vid_type=fixed_string(30)); + + // This example signs in the text service. + nebula> SIGN IN TEXT SERVICE (192.168.8.100:9200, HTTP); + + // This example checks the text service status. + nebula> SHOW TEXT SEARCH CLIENTS; + +-----------------+-----------------+------+ + | Type | Host | Port | + +-----------------+-----------------+------+ + | "ELASTICSEARCH" | "192.168.8.100" | 9200 | + +-----------------+-----------------+------+ + + // This example switches the graph space. + nebula> USE basketballplayer; + + // This example adds the listener to the NebulaGraph cluster. + nebula> ADD LISTENER ELASTICSEARCH 192.168.8.100:9789; + + // This example checks the listener status. When the status is `Online`, the listener is ready. + nebula> SHOW LISTENER; + +--------+-----------------+------------------------+-------------+ + | PartId | Type | Host | Host Status | + +--------+-----------------+------------------------+-------------+ + | 1 | "ELASTICSEARCH" | ""192.168.8.100":9789" | "ONLINE" | + | 2 | "ELASTICSEARCH" | ""192.168.8.100":9789" | "ONLINE" | + | 3 | "ELASTICSEARCH" | ""192.168.8.100":9789" | "ONLINE" | + +--------+-----------------+------------------------+-------------+ + + // This example creates the tag. + nebula> CREATE TAG IF NOT EXISTS player(name string, city string); + + // This example creates a single-attribute full-text index. + nebula> CREATE FULLTEXT TAG INDEX fulltext_index_1 ON player(name) ANALYZER="standard"; + + // This example creates a multi-attribute full-text indexe. + nebula> CREATE FULLTEXT TAG INDEX fulltext_index_2 ON player(name,city) ANALYZER="standard"; + + // This example rebuilds the full-text index. + nebula> REBUILD FULLTEXT INDEX; + + // This example shows the full-text index. + nebula> SHOW FULLTEXT INDEXES; + +--------------------+-------------+-------------+--------------+------------+ + | Name | Schema Type | Schema Name | Fields | Analyzer | + +--------------------+-------------+-------------+--------------+------------+ + | "fulltext_index_1" | "Tag" | "player" | "name" | "standard" | + | "fulltext_index_2" | "Tag" | "player" | "name, city" | "standard" | + +--------------------+-------------+-------------+--------------+------------+ + + // This example inserts the test data. + nebula> INSERT VERTEX player(name, city) VALUES \ + "Russell Westbrook": ("Russell Westbrook", "Los Angeles"), \ + "Chris Paul": ("Chris Paul", "Houston"),\ + "Boris Diaw": ("Boris Diaw", "Houston"),\ + "David West": ("David West", "Philadelphia"),\ + "Danny Green": ("Danny Green", "Philadelphia"),\ + "Tim Duncan": ("Tim Duncan", "New York"),\ + "James Harden": ("James Harden", "New York"),\ + "Tony Parker": ("Tony Parker", "Chicago"),\ + "Aron Baynes": ("Aron Baynes", "Chicago"),\ + "Ben Simmons": ("Ben Simmons", "Phoenix"),\ + "Blake Griffin": ("Blake Griffin", "Phoenix"); + + // These examples run test queries. + nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"Chris") YIELD id(vertex); + +--------------+ + | id(VERTEX) | + +--------------+ + | "Chris Paul" | + +--------------+ + + nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"Harden") YIELD properties(vertex); + +----------------------------------------------------------------+ + | properties(VERTEX) | + +----------------------------------------------------------------+ + | {_vid: "James Harden", city: "New York", name: "James Harden"} | + +----------------------------------------------------------------+ + + nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"Da*") YIELD properties(vertex); + +------------------------------------------------------------------+ + | properties(VERTEX) | + +------------------------------------------------------------------+ + | {_vid: "David West", city: "Philadelphia", name: "David West"} | + | {_vid: "Danny Green", city: "Philadelphia", name: "Danny Green"} | + +------------------------------------------------------------------+ + + nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"*b*") YIELD id(vertex); + +---------------------+ + | id(VERTEX) | + +---------------------+ + | "Russell Westbrook" | + | "Boris Diaw" | + | "Aron Baynes" | + | "Ben Simmons" | + | "Blake Griffin" | + +---------------------+ + + nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"*b*") YIELD id(vertex) | LIMIT 2,3; + +-----------------+ + | id(VERTEX) | + +-----------------+ + | "Aron Baynes" | + | "Ben Simmons" | + | "Blake Griffin" | + +-----------------+ + + nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"*b*") YIELD id(vertex) | YIELD count(*); + +----------+ + | count(*) | + +----------+ + | 5 | + +----------+ + + nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"*b*") YIELD id(vertex), score() AS score; + +---------------------+-------+ + | id(VERTEX) | score | + +---------------------+-------+ + | "Russell Westbrook" | 1.0 | + | "Boris Diaw" | 1.0 | + | "Aron Baynes" | 1.0 | + | "Ben Simmons" | 1.0 | + | "Blake Griffin" | 1.0 | + +---------------------+-------+ + + // For documents containing a word `b`, its score will be multiplied by a weighting factor of 4, while for documents containing a word `c`, the default weighting factor of 1 is used. + nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_1,"*b*^4 OR *c*") YIELD id(vertex), score() AS score; + +---------------------+-------+ + | id(VERTEX) | score | + +---------------------+-------+ + | "Russell Westbrook" | 4.0 | + | "Boris Diaw" | 4.0 | + | "Aron Baynes" | 4.0 | + | "Ben Simmons" | 4.0 | + | "Blake Griffin" | 4.0 | + | "Chris Paul" | 1.0 | + | "Tim Duncan" | 1.0 | + +---------------------+-------+ + + // When using a multi-attribute full-text index query, the conditions are matched within all properties of the index. + nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_2,"*h*") YIELD properties(vertex); + +------------------------------------------------------------------+ + | properties(VERTEX) | + +------------------------------------------------------------------+ + | {_vid: "Chris Paul", city: "Houston", name: "Chris Paul"} | + | {_vid: "Boris Diaw", city: "Houston", name: "Boris Diaw"} | + | {_vid: "David West", city: "Philadelphia", name: "David West"} | + | {_vid: "James Harden", city: "New York", name: "James Harden"} | + | {_vid: "Tony Parker", city: "Chicago", name: "Tony Parker"} | + | {_vid: "Aron Baynes", city: "Chicago", name: "Aron Baynes"} | + | {_vid: "Ben Simmons", city: "Phoenix", name: "Ben Simmons"} | + | {_vid: "Blake Griffin", city: "Phoenix", name: "Blake Griffin"} | + | {_vid: "Danny Green", city: "Philadelphia", name: "Danny Green"} | + +------------------------------------------------------------------+ + + // When using multi-attribute full-text index queries, you can specify different text for different properties for the query. + nebula> LOOKUP ON player WHERE ES_QUERY(fulltext_index_2,"name:*b* AND city:Houston") YIELD properties(vertex); + +-----------------------------------------------------------+ + | properties(VERTEX) | + +-----------------------------------------------------------+ + | {_vid: "Boris Diaw", city: "Houston", name: "Boris Diaw"} | + +-----------------------------------------------------------+ + + // Delete single-attribute full-text index. + nebula> DROP FULLTEXT INDEX fulltext_index_1; + ``` diff --git a/docs-2.0/4.deployment-and-installation/6.deploy-text-based-index/1.text-based-index-restrictions.md b/docs-2.0/4.deployment-and-installation/6.deploy-text-based-index/1.text-based-index-restrictions.md index 6b961918fb2..12b64364f87 100644 --- a/docs-2.0/4.deployment-and-installation/6.deploy-text-based-index/1.text-based-index-restrictions.md +++ b/docs-2.0/4.deployment-and-installation/6.deploy-text-based-index/1.text-based-index-restrictions.md @@ -1,34 +1,72 @@ # Full-text index restrictions -!!! caution +=== "Restrictions for NebulaGraph Community" - - This topic introduces the restrictions for full-text indexes. Please read the restrictions very carefully before using the full-text indexes. - - Version 3.5.0 redoes the full-text index function, which is not compatible with the previous versions, and requires deleting the previous index data and rebuilding the index. + !!! caution -For now, full-text search has the following limitations: + - This topic introduces the restrictions for full-text indexes. Please read the restrictions very carefully before using the full-text indexes. + - Version 3.5.0 redoes the full-text index function, which is not compatible with the previous versions, and requires deleting the previous index data and rebuilding the index. -- Currently, full-text search supports `LOOKUP` statements only. + For now, full-text search has the following limitations: -- The full-text index name can contain only numbers, lowercase letters, and underscores. + * Currently, full-text search supports `LOOKUP` statements only. -- The query returns 10 records by default. You can use the `LIMIT` clause to return more records, up to 10,000. You can modify the ElasticSearch parameters to adjust the maximum number of records returned. + * The full-text index name can contain only numbers, lowercase letters, and underscores. -- If there is a full-text index on the tag/edge type, the tag/edge type cannot be deleted or modified. + * The query returns 10 records by default. You can use the `LIMIT` clause to return more records, up to 10,000. You can modify the ElasticSearch parameters to adjust the maximum number of records returned. -- The type of properties must be `STRING` or `FIXED_STRING`. + * If there is a full-text index on the tag/edge type, the tag/edge type cannot be deleted or modified. -- Full-text index can not be applied to search multiple tags/edge types. + * The type of properties must be `STRING` or `FIXED_STRING`. -- Full-text index can not search properties with value `NULL`. + * Full-text index can not be applied to search multiple tags/edge types. -- Altering Elasticsearch indexes is not supported at this time. + * Full-text index can not search properties with value `NULL`. -- Modifying the analyzer is not supported. You have to delete the index data and then specify the analyzer when you rebuild the index. + * Altering Elasticsearch indexes is not supported at this time. -- `WHERE` clauses supports full-text search only working on single terms. + * Modifying the analyzer is not supported. You have to delete the index data and then specify the analyzer when you rebuild the index. -- Make sure that you start the Elasticsearch cluster and Nebula Graph at the same time. If not, the data writing on the Elasticsearch cluster can be incomplete. + * `WHERE` clauses supports full-text search only working on single terms. + + * Make sure that you start the Elasticsearch cluster and Nebula Graph at the same time. If not, the data writing on the Elasticsearch cluster can be incomplete. + + * It may take a while for Elasticsearch to create indexes. If Nebula Graph warns no index is found, you can check the status of the indexing task. + + - NebulaGraph clusters deployed with K8s do not have native support for the full-text search feature. However, you can manually deploy the feature yourself. + +=== "Restrictions for NebulaGraph Enterprise" + + !!! caution + + This topic introduces the restrictions for full-text indexes. Please read the restrictions very carefully before using the full-text indexes. + + For now, full-text search has the following limitations: + + * Currently, full-text search supports `LOOKUP` statements only. + + * The full-text index name can contain only numbers, lowercase letters, and underscores. + + * If there is a full-text index on the tag/edge type, the tag/edge type cannot be deleted or modified. + + * The type of properties must be `STRING` or `FIXED_STRING`. + + * Full-text index can not be applied to search multiple tags/edge types. + + * Sorting for the returned results of the full-text search is not supported. Data is returned in the order of data insertion. + + * Full-text index can not search properties with value `NULL`. + + * Altering Elasticsearch indexes is not supported at this time. + + * The pipe operator is not supported. + + * `WHERE` clauses supports full-text search only working on single terms. + + * Make sure that you start the Elasticsearch cluster and Nebula Graph at the same time. If not, the data writing on the Elasticsearch cluster can be incomplete. + + * It may take a while for Elasticsearch to create indexes. If Nebula Graph warns no index is found, wait for the index to take effect (however, the waiting time is unknown and there is no code to check). + + * NebulaGraph clusters deployed with K8s do not support the full-text search feature. -- It may take a while for Elasticsearch to create indexes. If Nebula Graph warns no index is found, you can check the status of the indexing task. -- NebulaGraph clusters deployed with K8s do not have native support for the full-text search feature. However, you can manually deploy the feature yourself. diff --git a/docs-2.0/4.deployment-and-installation/6.deploy-text-based-index/2.deploy-es.md b/docs-2.0/4.deployment-and-installation/6.deploy-text-based-index/2.deploy-es.md index 17f3c390803..b707299e748 100644 --- a/docs-2.0/4.deployment-and-installation/6.deploy-text-based-index/2.deploy-es.md +++ b/docs-2.0/4.deployment-and-installation/6.deploy-text-based-index/2.deploy-es.md @@ -1,78 +1,161 @@ # Deploy full-text index -Nebula Graph full-text indexes are powered by [Elasticsearch](https://en.wikipedia.org/wiki/Elasticsearch). This means that you can use Elasticsearch full-text query language to retrieve what you want. Full-text indexes are managed through built-in procedures. They can be created only for variable `STRING` and `FIXED_STRING` properties when the listener cluster and the Elasticsearch cluster are deployed. +=== "NebulaGraph Community" -## Precaution + Nebula Graph full-text indexes are powered by [Elasticsearch](https://en.wikipedia.org/wiki/Elasticsearch). This means that you can use Elasticsearch full-text query language to retrieve what you want. Full-text indexes are managed through built-in procedures. They can be created only for variable `STRING` and `FIXED_STRING` properties when the listener cluster and the Elasticsearch cluster are deployed. -Before you start using the full-text index, please make sure that you know the [restrictions](../../4.deployment-and-installation/6.deploy-text-based-index/1.text-based-index-restrictions.md). + ## Precaution -## Deploy Elasticsearch cluster + Before you start using the full-text index, please make sure that you know the [restrictions](../../4.deployment-and-installation/6.deploy-text-based-index/1.text-based-index-restrictions.md). -To deploy an Elasticsearch cluster, see [Kubernetes Elasticsearch deployment](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-deploy-elasticsearch.html) or [Elasticsearch installation](https://www.elastic.co/guide/en/elasticsearch/reference/7.15/targz.html). + ## Deploy Elasticsearch cluster -!!! note + To deploy an Elasticsearch cluster, see [Kubernetes Elasticsearch deployment](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-deploy-elasticsearch.html) or [Elasticsearch installation](https://www.elastic.co/guide/en/elasticsearch/reference/7.15/targz.html). Currently only `7.x` versions of `Elasticsearch` are supported. - To support external network access to Elasticsearch, set `network.host` to `0.0.0.0` in `config/elasticsearch.yml`. + !!! compatibility -You can configure the Elasticsearch to meet your business needs. To customize the Elasticsearch, see [Elasticsearch Document](https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html). + For NebulaGraph 3.4 and later versions, no additional templates need to be created. -## Sign in to the text search clients + You can configure the Elasticsearch to meet your business needs. To customize the Elasticsearch, see [Elasticsearch Document](https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html). -When the Elasticsearch cluster is deployed, use the `SIGN IN` statement to sign in to the Elasticsearch clients. Multiple `elastic_ip:port` pairs are separated with commas. You must use the IPs and the port number in the configuration file for the Elasticsearch. + ## Sign in to the text search clients -### Syntax + When the Elasticsearch cluster is deployed, use the `SIGN IN` statement to sign in to the Elasticsearch clients. Multiple `elastic_ip:port` pairs are separated with commas. You must use the IPs and the port number in the configuration file for the Elasticsearch. -```ngql -SIGN IN TEXT SERVICE (, {HTTP | HTTPS} [,"", ""]) [, (, ...)]; -``` + ### Syntax -### Example + ```ngql + SIGN IN TEXT SERVICE (, {HTTP | HTTPS} [,"", ""]) [, (, ...)]; + ``` -```ngql -nebula> SIGN IN TEXT SERVICE (192.168.8.100:9200, HTTP); -``` + ### Example -!!! Note + ```ngql + nebula> SIGN IN TEXT SERVICE (127.0.0.1:9200, HTTP); + ``` - Elasticsearch does not have a username or password by default. If you configured a username and password, you need to specify them in the `SIGN IN` statement. + !!! Note -!!! caution + Elasticsearch does not have a username or password by default. If you configured a username and password, you need to specify them in the `SIGN IN` statement. - The Elasticsearch client can only be logged in once, and if there are changes, you need to `SIGN OUT` and then `SIGN IN` again, and the client takes effect globally, and multiple graph spaces share the same Elasticsearch client. + !!! caution -## Show text search clients + The Elasticsearch client can only be logged in once, and if there are changes, you need to `SIGN OUT` and then `SIGN IN` again, and the client takes effect globally, and multiple graph spaces share the same Elasticsearch client. -The `SHOW TEXT SEARCH CLIENTS` statement can list the text search clients. + ## Show text search clients -### Syntax + The `SHOW TEXT SEARCH CLIENTS` statement can list the text search clients. -```ngql -SHOW TEXT SEARCH CLIENTS; -``` + ### Syntax -### Example + ```ngql + SHOW TEXT SEARCH CLIENTS; + ``` -```ngql -nebula> SHOW TEXT SEARCH CLIENTS; -+-----------------+-----------------+------+ -| Type | Host | Port | -+-----------------+-----------------+------+ -| "ELASTICSEARCH" | "192.168.8.100" | 9200 | -+-----------------+-----------------+------+ -``` + ### Example -## Sign out to the text search clients + ```ngql + nebula> SHOW TEXT SEARCH CLIENTS; + +-------------+------+ + | Host | Port | + +-------------+------+ + | "127.0.0.1" | 9200 | + | "127.0.0.1" | 9200 | + | "127.0.0.1" | 9200 | + +-------------+------+ + ``` -The `SIGN OUT TEXT SERVICE` statement can sign out all the text search clients. + ## Sign out to the text search clients -### Syntax + The `SIGN OUT TEXT SERVICE` statement can sign out all the text search clients. -```ngql -SIGN OUT TEXT SERVICE; -``` + ### Syntax -### Example + ```ngql + SIGN OUT TEXT SERVICE; + ``` -```ngql -nebula> SIGN OUT TEXT SERVICE; -``` + ### Example + + ```ngql + nebula> SIGN OUT TEXT SERVICE; + ``` + +=== "NebulaGraph Enterprise" + + Nebula Graph full-text indexes are powered by [Elasticsearch](https://en.wikipedia.org/wiki/Elasticsearch). This means that you can use Elasticsearch full-text query language to retrieve what you want. Full-text indexes are managed through built-in procedures. They can be created only for variable `STRING` and `FIXED_STRING` properties when the listener cluster and the Elasticsearch cluster are deployed. + + ## Precaution + + Before you start using the full-text index, please make sure that you know the [restrictions](../../4.deployment-and-installation/6.deploy-text-based-index/1.text-based-index-restrictions.md). + + ## Deploy Elasticsearch cluster + + To deploy an Elasticsearch cluster, see [Kubernetes Elasticsearch deployment](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-deploy-elasticsearch.html) or [Elasticsearch installation](https://www.elastic.co/guide/en/elasticsearch/reference/7.15/targz.html). + + !!! note + + To support external network access to Elasticsearch, set `network.host` to `0.0.0.0` in `config/elasticsearch.yml`. + + You can configure the Elasticsearch to meet your business needs. To customize the Elasticsearch, see [Elasticsearch Document](https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html). + + ## Sign in to the text search clients + + When the Elasticsearch cluster is deployed, use the `SIGN IN` statement to sign in to the Elasticsearch clients. Multiple `elastic_ip:port` pairs are separated with commas. You must use the IPs and the port number in the configuration file for the Elasticsearch. + + ### Syntax + + ```ngql + SIGN IN TEXT SERVICE (, {HTTP | HTTPS} [,"", ""]) [, (, ...)]; + ``` + + ### Example + + ```ngql + nebula> SIGN IN TEXT SERVICE (192.168.8.100:9200, HTTP); + ``` + + !!! Note + + Elasticsearch does not have a username or password by default. If you configured a username and password, you need to specify them in the `SIGN IN` statement. + + !!! caution + + The Elasticsearch client can only be logged in once, and if there are changes, you need to `SIGN OUT` and then `SIGN IN` again, and the client takes effect globally, and multiple graph spaces share the same Elasticsearch client. + + ## Show text search clients + + The `SHOW TEXT SEARCH CLIENTS` statement can list the text search clients. + + ### Syntax + + ```ngql + SHOW TEXT SEARCH CLIENTS; + ``` + + ### Example + + ```ngql + nebula> SHOW TEXT SEARCH CLIENTS; + +-----------------+-----------------+------+ + | Type | Host | Port | + +-----------------+-----------------+------+ + | "ELASTICSEARCH" | "192.168.8.100" | 9200 | + +-----------------+-----------------+------+ + ``` + + ## Sign out to the text search clients + + The `SIGN OUT TEXT SERVICE` statement can sign out all the text search clients. + + ### Syntax + + ```ngql + SIGN OUT TEXT SERVICE; + ``` + + ### Example + + ```ngql + nebula> SIGN OUT TEXT SERVICE; + ``` diff --git a/docs-2.0/4.deployment-and-installation/6.deploy-text-based-index/3.deploy-listener.md b/docs-2.0/4.deployment-and-installation/6.deploy-text-based-index/3.deploy-listener.md index a8854c52392..639800c6b30 100644 --- a/docs-2.0/4.deployment-and-installation/6.deploy-text-based-index/3.deploy-listener.md +++ b/docs-2.0/4.deployment-and-installation/6.deploy-text-based-index/3.deploy-listener.md @@ -1,100 +1,209 @@ # Deploy Raft Listener for NebulaGraph Storage service -Full-text index data is written to the Elasticsearch cluster asynchronously. The Raft Listener (Listener for short) is a separate process that fetches data from the Storage Service and writes them into the Elasticsearch cluster. +=== "NebulaGraph Community" -## Prerequisites + Full-text index data is written to the Elasticsearch cluster asynchronously. The Raft Listener (Listener for short) is a separate process that fetches data from the Storage Service and writes them into the Elasticsearch cluster. -* You have read and fully understood the [restrictions](../../4.deployment-and-installation/6.deploy-text-based-index/1.text-based-index-restrictions.md) for using full-text indexes. + ## Prerequisites -* You have [deployed a NebulaGraph cluster](../2.compile-and-install-nebula-graph/deploy-nebula-graph-cluster.md). + * You have read and fully understood the [restrictions](../../4.deployment-and-installation/6.deploy-text-based-index/1.text-based-index-restrictions.md) for using full-text indexes. -* You have [deployed a Elasticsearch cluster](./2.deploy-es.md). + * You have [deployed a NebulaGraph cluster](../2.compile-and-install-nebula-graph/deploy-nebula-graph-cluster.md). -* You have prepared one or multiple servers to run one or multiple raft listeners. + * You have [deploy a Elasticsearch cluster](./2.deploy-es.md). -## Precautions + * You have prepared at least one extra Storage Server. To use the full-text search, you must run one or more Storage Server as the Raft Listener. -* The Storage Service that you want to run as the Listener must have the same or later release with all the other Nebula Graph services in the cluster. + ## Precautions -* For now, you can only add all Listeners to a graph space once and for all. Trying to add a new Listener to a graph space that already has a Listener will fail. To add all Listeners, set them [in one statement](#step_4_add_listeners_to_nebula_graph). + * The Storage Service that you want to run as the Listener must have the same or later release with all the other Nebula Graph services in the cluster. -## Deployment process + * For now, you can only add all Listeners to a graph space once and for all. Trying to add a new Listener to a graph space that already has a Listener will fail. To add all Listeners, set them [in one statement](#step_4_add_listeners_to_nebula_graph). -### Step 1: Install the Listener service + ## Deployment process -The Listener service uses the same binary as the storaged service. However, the configuration files are different and the processes use different ports. You can install NebulaGraph on all servers that need to deploy a Listener, but only the storaged service can be used. For details, see [Install NebulaGraph by RPM or DEB Package](../2.compile-and-install-nebula-graph/2.install-nebula-graph-by-rpm-or-deb.md). + ### Step 1: Install the Storage service -### Step 2: Prepare the configuration file for the Listener + The Listener process and the storaged process use the same binary file. However, their configuration files and using ports are different. You can install NebulaGraph on all servers that need to deploy a Listener, but only the Storage service can be used. For details, see [Install NebulaGraph by RPM or DEB Package](../2.compile-and-install-nebula-graph/2.install-nebula-graph-by-rpm-or-deb.md). -In the `etc` directory, remove the suffix from `nebula-storaged-listener.conf.default` or `nebula-storaged-listener.conf.production` to `nebula-storaged-listener.conf`, and then modify the configuration content. + ### Step 2: Prepare the configuration file for the Listener -Most configurations are the same as the configurations of [Storage Service](../../5.configurations-and-logs/1.configurations/4.storage-config.md). This topic only introduces the differences. + You have to prepare a corresponding configuration file on the machine that you want to deploy a Listener. The file must be named as `nebula-storaged-listener.conf` and stored in the `etc` directory. A [template](https://github.com/vesoft-inc/nebula-storage/blob/master/conf/nebula-storaged-listener.conf.production) is provided for your reference. Note that the file suffix `.production` should be removed. -| Name | Default value | Description | -| :----------- | :----------------------- | :------------------ | -| `daemonize` | `true` | When set to `true`, the process is a daemon process. | -| `pid_file` | `pids/nebula-metad-listener.pid` | The file that records the process ID. | -| `meta_server_addrs` | - | IP addresses and ports of all Meta services. Multiple Meta services are separated by commas. | -| `local_ip` | - | The local IP address of the Listener service. Use real IP addresses instead of domain names or loopback IP addresses such as `127.0.0.1`. | -| `port` | - | The listening port of the RPC daemon of the Listener service. | -| `heartbeat_interval_secs` | `10` | The heartbeat interval of the Meta service. The unit is second (s). | -| `listener_path` | `data/listener` | The WAL directory of the Listener. Only one directory is allowed. | -| `data_path` | `data` | For compatibility reasons, this parameter can be ignored. Fill in the default value `data`. | -| `part_man_type` | `memory` | The type of the part manager. Optional values ​​are `memory` and `meta`. | -| `rocksdb_batch_size` | `4096` | The default reserved bytes for batch operations. | -| `rocksdb_block_cache` | `4` | The default block cache size of BlockBasedTable. The unit is Megabyte (MB). | -| `engine_type` | `rocksdb` | The type of the Storage engine, such as `rocksdb`, `memory`, etc. | -| `part_type` | `simple` | The type of the part, such as `simple`, `consensus`, etc. | + Most configurations are the same as the configurations of [Storage Service](../../5.configurations-and-logs/1.configurations/4.storage-config.md). This topic only introduces the differences. -### Step 3: Start Listeners + | Name | Default value | Description | + | :----------- | :----------------------- | :------------------ | + | `daemonize` | `true` | When set to `true`, the process is a daemon process. | + | `pid_file` | `pids/nebula-metad-listener.pid` | The file that records the process ID. | + | `meta_server_addrs` | - | IP addresses and ports of all Meta services. Multiple Meta services are separated by commas. | + | `local_ip` | - | The local IP address of the Listener service. | + | `port` | - | The listening port of the RPC daemon of the Listener service. | + | `heartbeat_interval_secs` | `10` | The heartbeat interval of the Meta service. The unit is second (s). | + | `listener_path` | `data/listener` | The WAL directory of the Listener. Only one directory is allowed. | + | `data_path` | `data` | For compatibility reasons, this parameter can be ignored. Fill in the default value `data`. | + | `part_man_type` | `memory` | The type of the part manager. Optional values ​​are `memory` and `meta`. | + | `rocksdb_batch_size` | `4096` | The default reserved bytes for batch operations. | + | `rocksdb_block_cache` | `4` | The default block cache size of BlockBasedTable. The unit is Megabyte (MB). | + | `engine_type` | `rocksdb` | The type of the Storage engine, such as `rocksdb`, `memory`, etc. | + | `part_type` | `simple` | The type of the part, such as `simple`, `consensus`, etc. | -To initiate the Listener, navigate to the installation path of the desired cluster and execute the following command: + !!! note -```bash -./bin/nebula-storaged --flagfile etc/nebula-storaged-listener.conf -``` + Use real IP addresses in the configuration file instead of domain names or loopback IP addresses such as `127.0.0.1`. -### Step 4: Add Listeners to NebulaGraph + ### Step 3: Start Listeners -[Connect to NebulaGraph](../../2.quick-start/3.connect-to-nebula-graph.md) and run [`USE `](../../3.ngql-guide/9.space-statements/2.use-space.md) to enter the graph space that you want to create full-text indexes for. Then run the following statement to add a Listener into NebulaGraph. + Run the following command to start the Listener. -```ngql -ADD LISTENER ELASTICSEARCH [,, ...] -``` + ```bash + ./bin/nebula-storaged --flagfile /nebula-storaged-listener.conf + ``` + + `${listener_config_path}` is the path where you store the Listener configuration file. + + ### Step 4: Add Listeners to NebulaGraph + + [Connect to NebulaGraph](../../2.quick-start/3.connect-to-nebula-graph.md) and run [`USE `](../../3.ngql-guide/9.space-statements/2.use-space.md) to enter the graph space that you want to create full-text indexes for. Then run the following statement to add a Listener into NebulaGraph. + + ```ngql + ADD LISTENER ELASTICSEARCH [,, ...] + ``` + + !!! warning + + You must use real IPs for a Listener. + + Add all Listeners in one statement completely. + + ```ngql + nebula> ADD LISTENER ELASTICSEARCH 192.168.8.5:9789,192.168.8.6:9789; + ``` + + ## Show Listeners + + Run the `SHOW LISTENER` statement to list all Listeners. -!!! warning + ### Example - You must use real IPs for a Listener. + ```ngql + nebula> SHOW LISTENER; + +--------+-----------------+-----------------------+----------+ + | PartId | Type | Host | Status | + +--------+-----------------+-----------------------+----------+ + | 1 | "ELASTICSEARCH" | "[192.168.8.5:46780]" | "ONLINE" | + | 2 | "ELASTICSEARCH" | "[192.168.8.5:46780]" | "ONLINE" | + | 3 | "ELASTICSEARCH" | "[192.168.8.5:46780]" | "ONLINE" | + +--------+-----------------+-----------------------+----------+ + ``` + + ## Remove Listeners + + Run the `REMOVE LISTENER ELASTICSEARCH` statement to remove all Listeners in a graph space. - Add all Listeners in one statement completely. + ### Example ```ngql - nebula> ADD LISTENER ELASTICSEARCH 192.168.8.100:9789,192.168.8.101:9789; + nebula> REMOVE LISTENER ELASTICSEARCH; ``` -## Show Listeners +=== "NebulaGraph Enterprise" + + Full-text index data is written to the Elasticsearch cluster asynchronously. The Raft Listener (Listener for short) is a separate process that fetches data from the Storage Service and writes them into the Elasticsearch cluster. + + ## Prerequisites + + * You have read and fully understood the [restrictions](../../4.deployment-and-installation/6.deploy-text-based-index/1.text-based-index-restrictions.md) for using full-text indexes. + + * You have [deployed a NebulaGraph cluster](../2.compile-and-install-nebula-graph/deploy-nebula-graph-cluster.md). + + * You have [deployed a Elasticsearch cluster](./2.deploy-es.md). + + * You have prepared one or multiple servers to run one or multiple raft listeners. + + ## Precautions + + * The Storage Service that you want to run as the Listener must have the same or later release with all the other Nebula Graph services in the cluster. + + * For now, you can only add all Listeners to a graph space once and for all. Trying to add a new Listener to a graph space that already has a Listener will fail. To add all Listeners, set them [in one statement](#step_4_add_listeners_to_nebula_graph). -Run the `SHOW LISTENER` statement to list all Listeners. + ## Deployment process -### Example + ### Step 1: Install the Listener service -```ngql -nebula> SHOW LISTENER; -+--------+-----------------+------------------------+-------------+ -| PartId | Type | Host | Host Status | -+--------+-----------------+------------------------+-------------+ -| 1 | "ELASTICSEARCH" | ""192.168.8.100":9789" | "ONLINE" | -| 2 | "ELASTICSEARCH" | ""192.168.8.100":9789" | "ONLINE" | -| 3 | "ELASTICSEARCH" | ""192.168.8.100":9789" | "ONLINE" | -+--------+-----------------+------------------------+-------------+ -``` + The Listener service uses the same binary as the storaged service. However, the configuration files are different and the processes use different ports. You can install NebulaGraph on all servers that need to deploy a Listener, but only the storaged service can be used. For details, see [Install NebulaGraph by RPM or DEB Package](../2.compile-and-install-nebula-graph/2.install-nebula-graph-by-rpm-or-deb.md). -## Remove Listeners + ### Step 2: Prepare the configuration file for the Listener -Run the `REMOVE LISTENER ELASTICSEARCH` statement to remove all Listeners in a graph space. + In the `etc` directory, remove the suffix from `nebula-storaged-listener.conf.default` or `nebula-storaged-listener.conf.production` to `nebula-storaged-listener.conf`, and then modify the configuration content. -### Example + Most configurations are the same as the configurations of [Storage Service](../../5.configurations-and-logs/1.configurations/4.storage-config.md). This topic only introduces the differences. -```ngql -nebula> REMOVE LISTENER ELASTICSEARCH; -``` + | Name | Default value | Description | + | :----------- | :----------------------- | :------------------ | + | `daemonize` | `true` | When set to `true`, the process is a daemon process. | + | `pid_file` | `pids/nebula-metad-listener.pid` | The file that records the process ID. | + | `meta_server_addrs` | - | IP addresses and ports of all Meta services. Multiple Meta services are separated by commas. | + | `local_ip` | - | The local IP address of the Listener service. Use real IP addresses instead of domain names or loopback IP addresses such as `127.0.0.1`. | + | `port` | - | The listening port of the RPC daemon of the Listener service. | + | `heartbeat_interval_secs` | `10` | The heartbeat interval of the Meta service. The unit is second (s). | + | `listener_path` | `data/listener` | The WAL directory of the Listener. Only one directory is allowed. | + | `data_path` | `data` | For compatibility reasons, this parameter can be ignored. Fill in the default value `data`. | + | `part_man_type` | `memory` | The type of the part manager. Optional values ​​are `memory` and `meta`. | + | `rocksdb_batch_size` | `4096` | The default reserved bytes for batch operations. | + | `rocksdb_block_cache` | `4` | The default block cache size of BlockBasedTable. The unit is Megabyte (MB). | + | `engine_type` | `rocksdb` | The type of the Storage engine, such as `rocksdb`, `memory`, etc. | + | `part_type` | `simple` | The type of the part, such as `simple`, `consensus`, etc. | + + ### Step 3: Start Listeners + + To initiate the Listener, navigate to the installation path of the desired cluster and execute the following command: + + ```bash + ./bin/nebula-storaged --flagfile etc/nebula-storaged-listener.conf + ``` + + ### Step 4: Add Listeners to NebulaGraph + + [Connect to NebulaGraph](../../2.quick-start/3.connect-to-nebula-graph.md) and run [`USE `](../../3.ngql-guide/9.space-statements/2.use-space.md) to enter the graph space that you want to create full-text indexes for. Then run the following statement to add a Listener into NebulaGraph. + + ```ngql + ADD LISTENER ELASTICSEARCH [,, ...] + ``` + + !!! warning + + You must use real IPs for a Listener. + + Add all Listeners in one statement completely. + + ```ngql + nebula> ADD LISTENER ELASTICSEARCH 192.168.8.100:9789,192.168.8.101:9789; + ``` + + ## Show Listeners + + Run the `SHOW LISTENER` statement to list all Listeners. + + ### Example + + ```ngql + nebula> SHOW LISTENER; + +--------+-----------------+------------------------+-------------+ + | PartId | Type | Host | Host Status | + +--------+-----------------+------------------------+-------------+ + | 1 | "ELASTICSEARCH" | ""192.168.8.100":9789" | "ONLINE" | + | 2 | "ELASTICSEARCH" | ""192.168.8.100":9789" | "ONLINE" | + | 3 | "ELASTICSEARCH" | ""192.168.8.100":9789" | "ONLINE" | + +--------+-----------------+------------------------+-------------+ + ``` + + ## Remove Listeners + + Run the `REMOVE LISTENER ELASTICSEARCH` statement to remove all Listeners in a graph space. + + ### Example + + ```ngql + nebula> REMOVE LISTENER ELASTICSEARCH; + ``` diff --git a/mkdocs.yml b/mkdocs.yml index ee9038f7ecc..f5de3c16393 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -59,7 +59,8 @@ markdown_extensions: - pymdownx.caret - pymdownx.tilde - pymdownx.superfences - - pymdownx.tabbed + - pymdownx.tabbed: + alternate_style: true # Plugins plugins: