Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support simple_query_string function in SQL/PPL query engine #192

Closed
5 tasks
chloe-zh opened this issue Aug 21, 2021 · 1 comment
Closed
5 tasks

Support simple_query_string function in SQL/PPL query engine #192

chloe-zh opened this issue Aug 21, 2021 · 1 comment

Comments

@chloe-zh
Copy link
Contributor

Related design is presented in issue #182

Todo list

  • Support the simple_query_string functionality by pushing it down to the search engine
  • Enable simple_query_string function in SQL and PPL syntax and parser, including all the available parameters
  • Add unit tests. (Unit tests are mandatory for all code changes.)
  • Add integration test cases for simple_query_string
  • Update user manual

Function details

The simple_query_string function maps to the simple query string used in the search engine. This query returns documents based on a provided query string, using a parser with a limited but fault-tolerant syntax. While its syntax is more limited than the query_string query, the simple_query_string query does not return errors for invalid syntax. Instead, it ignores any invalid parts of the query string.

Syntax:
simple_query_string([field_list, ] query_string_expression[, option=<option_value>]*)

Simple query string syntax:
The simple_query_string query supports the following operators:
+ signifies AND operation
| signifies OR operation
- negates a single token
" wraps a number of tokens to signify a phrase for searching
* at the end of a term signifies a prefix query
( and ) signify precedence
~N after a word signifies edit distance (fuzziness)
~N after a phrase signifies slop amount

FLAG valid values:
ALL (Default) Enables all optional operators.
AND Enables the + AND operator.
ESCAPE Enables \ as an escape character.
FUZZY Enables the ~N operator after a word, where N is an integer denoting the allowed edit distance for matching
NEAR Enables the ~N operator, after a phrase where N is the maximum number of positions allowed between matching tokens. Synonymous to SLOP.
NONE Disables all operators.
NOT Enables the - NOT operator.
OR Enables the | OR operator.
PHRASE Enables the " quotes operator used to search for phrases.
PRECEDENCE Enables the ( and ) operators to control operator precedence.
PREFIX Enables the * prefix operator.
SLOP Enables the ~N operator, after a phrase where N is maximum number of positions allowed between matching tokens. Synonymous to NEAR.
WHITESPACE Enables whitespace as split characters.

Available options:

  • default_operator
  • fields: (array) Array of fields you wish to search. Set to "*" if wish to search all fields. Defaults to the index.query.default_field index setting, which has a default value of *.
  • default_operator
  • analyze_wildcard
  • auto_generate_synonyms_phrase_query
  • flags: (string) List of enabled operators for the simple query string syntax. Defaults to ALL (all operators).
  • fuzzy_max_expansions
  • fuzzy_prefix_length
  • fuzzy_transpositions
  • lenient
  • minimum_should_match
  • quote_field_suffix

Sample query:

# Search query 1
GET my_index/_search
{
  "query": {
    "simple_query_string" : {
        "query": "\"fried eggs\" +(eggplant | potato) -frittata",
        "fields": ["title^5", "body"],
        "default_operator": "and"
    }
  }
}

# SQL
SELECT * FROM my_index WHERE simple_query_string(["title^5", "body"], "\"fried eggs\" +(eggplant | potato) -frittata", default_operator=and)

# PPL
source=my_index | where simple_query_string(["title^5", "body"], "\"fried eggs\" +(eggplant | potato) -frittata", default_operator=and)


# Search query 2
GET my_index/_search
{
  "query": {
    "simple_query_string": {
      "fields": [ "*" ],
      "query": "foo bar -baz"
    }
  }
}

# SQL
SELECT * FROM my_index WHERE simple_query_string(["*"], "foo bar -baz")

# PPL
source=my_index | where simple_query_string(["*"], "foo bar -baz")


# Search query 3
GET my_index/_search
{
  "query": {
    "simple_query_string": {
      "query": "foo | bar + baz*",
      "flags": "OR|AND|PREFIX"
    }
  }
}

# SQL
SELECT * FROM my_index WHERE simple_query_string("foo | bar + baz*", "OR|AND|PREFIX")

# PPL
source=my_index | where simple_query_string("foo | bar + baz*", "OR|AND|PREFIX")
MaxKsyunz referenced this issue in Bit-Quill/opensearch-project-sql May 14, 2022
MaxKsyunz referenced this issue in Bit-Quill/opensearch-project-sql May 14, 2022
…l' into dev-simple_query_string-#192-sql-simple-max_ast
Yury-Fridlyand referenced this issue in Bit-Quill/opensearch-project-sql Jun 1, 2022
…v-simple_query_string-#192-impl2

Signed-off-by: Yury Fridlyand <[email protected]>
Yury-Fridlyand referenced this issue in Bit-Quill/opensearch-project-sql Jun 2, 2022
#192-impl2`, since that branch was corrupted by incorrect merge.

Signed-off-by: Yury Fridlyand <[email protected]>
Yury-Fridlyand referenced this issue in Bit-Quill/opensearch-project-sql Jun 3, 2022
@acarbonetto
Copy link
Collaborator

fixed by #635

@dai-chen dai-chen closed this as completed Aug 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants