Support query_string function in SQL/PPL query engine

[chloe-zh]

Related design is presented in issue #182

Todo list

• Support the query_string functionality by pushing it down to the search engine
• Enable query_string function in SQL and PPL syntax and parser, including all the available parameters
• Make sure the function in new engine is compatible with the old engine support
• Add unit tests. (Unit tests are mandatory for all code changes.)
• Add integration test cases for query_string
• Update user manual

Function details

The query_string function maps to query string used in the search engine. This query returns documents based on a provided query string, using a parser with a strict syntax.

Syntax:
query_string([**field_expression** | **field_list**, ] **query_string_expression**[, option=<option_value>]*)

Query string expression syntax:
**query_expression** operator **query_expression**, where operator includes AND and OR
Example: "query_1 AND query_2"

Available options:

• analyzer
• default_field: (string) Default field you wish to search if no field is provided in the query string.
• fields: (array) Array of fields you wish to search. Set to "*" if wish to search all fields. Defaults to the index.query.default_field index setting, which has a default value of *.
• allow_leading_wildcard: (boolean) If true, the wildcard characters * and ? are allowed as the first character of the query string. Defaults to true.
• analyze_wildcard: (boolean) If true, the query attempts to analyze wildcard terms in the query string. Defaults to false.
• auto_generate_synonyms_phrase_query
• boost
• default_operator
• enable_position_increments: (boolean) If true, enable position increments in queries constructed from a query_string search. Defaults to true.
• fuzziness
• fuzzy_max_expansions
• fuzzy_prefix_length
• fuzzy_transpositions
• lenient
• max_determinized_states: (integer) Maximum number of automaton states required for the query. Default is 10000.
• minimum_should_match
• quote_analyzer: (string) Analyzer used to convert quoted text in the query string into tokens. Defaults to the search_quote_analyzer mapped for the default_field.
• phrase_slop: (integer) Maximum number of positions allowed between matching tokens for phrases. Defaults to 0. If 0, exact phrase matches are required. Transposed terms have a slop of 2.
• quote_field_suffix: (string) Suffix appended to quoted text in the query string.
• rewrite: (string) Method used to rewrite the query.
• time_zone

Sample query:

# Search query 1
GET my_index/_search
{
  "query": {
    "query_string": {
      "query": "(new york city) OR (big apple)",
      "default_field": "content"
    }
  }
}

# SQL
SELECT * FROM my_index WHERE query_string(content, "(new york city) OR (big apple)")

# PPL
source=my_index | where query_string(content, "(new york city) OR (big apple)")


# Search query 2
GET my_index/_search
{
  "query": {
    "query_string": {
      "fields": [ "content", "name" ],
      "query": "this AND that"
    }
  }
}

# SQL
SELECT * FROM my_index WHERE query_string([content, name], "this AND that")

# PPL
source=my_index | where query_string([content, name], "this AND that")

[forestmvey]

@joshuali925 Here is a quick demo of the query_string implementation usage in SQL and PPL for PR: 675

query_string_demo_video.mp4

[acarbonetto]

fixed by #675 and Bit-Quill#85

[dai-chen]

@forestmvey The video is really helpful to user. Do we have our channel for all these video? If not, maybe we can put them into doc in https://github.com/opensearch-project/sql/tree/main/docs/presentations

[forestmvey]

@forestmvey The video is really helpful to user. Do we have our channel for all these video? If not, maybe we can put them into doc in https://github.com/opensearch-project/sql/tree/main/docs/presentations

Updating the documentation with video links to the github issue may be nice to do alongside saving demo videos in this directory you've suggested. AFAIK demo videos have only been attached to the github issues so far.