Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update query and setting API endpoints for SQL and PPL documentations #26

Merged
merged 2 commits into from
Jun 8, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 3 additions & 30 deletions docs/ppl/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ search source=accounts;
```

| account_number | firstname | address | balance | gender | city | employer | state | age | email | lastname |
:--- | :--- |
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
| 1 | Amber | 880 Holmes Lane | 39225 | M | Brogan | Pyrami | IL | 32 | [email protected] | Duke
| 6 | Hattie | 671 Bristol Street | 5686 | M | Dante | Netagy | TN | 36 | [email protected] | Bond
| 13 | Nanette | 789 Madison Street | 32838 | F | Nogal | Quility | VA | 28 | null | Bates
Expand All @@ -61,7 +61,7 @@ search source=accounts account_number=1 or gender="F";
```

| account_number | firstname | address | balance | gender | city | employer | state | age | email | lastname |
:--- | :--- |
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
| 1 | Amber | 880 Holmes Lane | 39225 | M | Brogan | Pyrami | IL | 32 | [email protected] | Duke |
| 13 | Nanette | 789 Madison Street | 32838 | F | Nogal | Quility | VA | 28 | null | Bates |

Expand Down Expand Up @@ -513,13 +513,11 @@ Use the `head` command to return the first N number of results in a specified se
### Syntax

```sql
head [keeplast = (true | false)] [while "("<boolean-expression>")"] [N]
head [N]
```

Field | Description | Required | Default
:--- | :--- |:---
`keeplast` | Use along with the `while` argument to check if the last result in the result set is retained. The last result is what caused the `while` condition to evaluate to false or NULL. Set `keeplast` to true to retain the last result and false to discard it. | No | True
`while` | An expression that evaluates to either true or false. You cannot use statistical functions in this expression. | No | False
`N` | Specify the number of results to return. | No | 10

*Example 1*: Get the first 10 results
Expand Down Expand Up @@ -549,31 +547,6 @@ search source=accounts | fields firstname, age | head 2;
| Amber | 32
| Hattie | 36

*Example 3*: Get the first N results that match a while condition

To get the first 3 results from all accounts with age less than 30:

```sql
search source=accounts | fields firstname, age | sort age | head while(age < 30) 3;
```

| firstname | age
:--- | :--- |
| Nanette | 28
| Amber | 32

*Example 4*: Get the first N results with a while condition with the last result that failed the condition

To get the first 3 results from all accounts with age less than 30 and include the last failed condition:

```sql
search source=accounts | fields firstname, age | sort age | head keeplast=false while(age < 30) 3;
```

| firstname | age
:--- | :--- |
| Nanette | 28

## rare

Use the `rare` command to find the least common values of all fields in a field list.
Expand Down
6 changes: 3 additions & 3 deletions docs/ppl/endpoint.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,13 @@ nav_order: 1
To send a query request to PPL plugin, use the HTTP POST request.
We recommend a POST request because it doesn't have any length limit and it allows you to pass other parameters to the plugin for other functionality.

Use the explain endpoint for query translation and troubleshooting.
Use the `_explain` endpoint for query translation and troubleshooting.

## Request Format

To use the PPL plugin with your own applications, send requests to `_opensearch/_ppl`, with your query in the request body:
To use the PPL plugin with your own applications, send requests to `_plugins/_ppl`, with your query in the request body:

```json
curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_ppl \
curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_ppl \
... -d '{"query" : "source=accounts | fields firstname, lastname"}'
```
12 changes: 6 additions & 6 deletions docs/ppl/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,11 @@ search source=accounts

#### Sample Response

| id | firstname | lastname |
:--- | :--- | :--- |
| 0 | Amber | Duke
| 1 | Hattie | Bond
| 2 | Nanette | Bates
| 3 | Dale | Adams
firstname | lastname |
:--- | :--- |
Amber | Duke
Hattie | Bond
Nanette | Bates
Dale | Adams

![PPL query workbench](../images/ppl.png)
4 changes: 2 additions & 2 deletions docs/ppl/protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The PPL plugin provides responses in JDBC format. The JDBC format is widely used
The body of HTTP POST request can take a few more additional fields with the PPL query:

```json
curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_ppl \
curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_ppl \
... -d '{"query" : "source=accounts | fields firstname, lastname"}'
```

Expand Down Expand Up @@ -58,7 +58,7 @@ The following example shows a normal response where the schema includes a field
If any error occurred, error message and the cause will be returned instead:

```json
curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_ppl \
curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_ppl \
... -d '{"query" : "source=unknown | fields firstname, lastname"}'
{
"error": {
Expand Down
24 changes: 19 additions & 5 deletions docs/ppl/settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ You can update these settings like any other cluster setting:
PUT _cluster/settings
{
"transient": {
"opensearch": {
"plugins": {
"ppl": {
"enabled": "false"
}
Expand All @@ -24,12 +24,26 @@ PUT _cluster/settings
}
```

Requests to `_opensearch/_ppl` include index names in the request body, so they have the same access policy considerations as the `bulk`, `mget`, and `msearch` operations. If you set the `rest.action.multi.allow_explicit_index` parameter to `false`, the PPL plugin is disabled.
Similarly, you can also update the settings by sending request to the plugin setting endpoint `_plugins/_query/settings` :
```json
PUT _plugins/_query/settings
{
"transient": {
"plugins": {
"ppl": {
"enabled": "false"
}
}
}
}
```

Requests to `_plugins/_ppl` include index names in the request body, so they have the same access policy considerations as the `bulk`, `mget`, and `msearch` operations. If you set the `rest.action.multi.allow_explicit_index` parameter to `false`, the PPL plugin is disabled.

You can specify the settings shown in the following table:

Setting | Description | Default
:--- | :--- | :---
`opensearch.ppl.enabled` | Change to `false` to disable the plugin. | True
`opensearch.ppl.query.memory_limit` | Set heap memory usage limit. If a query crosses this limit, it's terminated. | 85%
`opensearch.query.size_limit` | Set the maximum number of results that you want to see. This impacts the accuracy of aggregation operations. For example, if you have 1000 documents in an index, by default, only 200 documents are extracted from the index for aggregation. | 200
`plugins.ppl.enabled` | Change to `false` to disable the PPL component. | True
`plugins.query.memory_limit` | Set heap memory usage limit. If a query crosses this limit, it's terminated. | 85%
`plugins.query.size_limit` | Set the maximum number of results that you want to see. This impacts the accuracy of aggregation operations. For example, if you have 1000 documents in an index, by default, only 200 documents are extracted from the index for aggregation. | 200
122 changes: 61 additions & 61 deletions docs/sql/basic.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,12 +81,12 @@ SELECT *
FROM accounts
```

| id | account_number | firstname | gender | city | balance | employer | state | email | address | lastname | age
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
0 | 1 | Amber | M | Brogan | 39225 | Pyrami | IL | [email protected] | 880 Holmes Lane | Duke | 32
1 | 16 | Hattie | M | Dante | 5686 | Netagy | TN | [email protected] | 671 Bristol Street | Bond | 36
2 | 13 | Nanette | F | Nogal | 32838 | Quility | VA | [email protected] | 789 Madison Street | Bates | 28
3 | 18 | Dale | M | Orick | 4180 | | MD | [email protected] | 467 Hutchinson Court | Adams | 33
| account_number | firstname | gender | city | balance | employer | state | email | address | lastname | age
| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
| 1 | Amber | M | Brogan | 39225 | Pyrami | IL | [email protected] | 880 Holmes Lane | Duke | 32
| 16 | Hattie | M | Dante | 5686 | Netagy | TN | [email protected] | 671 Bristol Street | Bond | 36
| 13 | Nanette | F | Nogal | 32838 | Quility | VA | [email protected] | 789 Madison Street | Bates | 28
| 18 | Dale | M | Orick | 4180 | | MD | [email protected] | 467 Hutchinson Court | Adams | 33

*Example 2*: Use field name(s) to retrieve only specific fields:

Expand All @@ -95,12 +95,12 @@ SELECT firstname, lastname
FROM accounts
```

| id | firstname | lastname
:--- | :--- | :---
0 | Amber | Duke
1 | Hattie | Bond
2 | Nanette | Bates
3 | Dale | Adams
| firstname | lastname
| :--- | :---
| Amber | Duke
| Hattie | Bond
| Nanette | Bates
| Dale | Adams

*Example 3*: Use field aliases instead of field names. Field aliases are used to make field names more readable:

Expand All @@ -109,12 +109,12 @@ SELECT account_number AS num
FROM accounts
```

| id | num
:--- | :---
0 | 1
1 | 6
2 | 13
3 | 18
| num
:---
| 1
| 6
| 13
| 18

*Example 4*: Use the `DISTINCT` clause to get back only unique field values. You can specify one or more field names:

Expand All @@ -123,12 +123,12 @@ SELECT DISTINCT age
FROM accounts
```

| id | age
:--- | :---
0 | 28
1 | 32
2 | 33
3 | 36
| age
:---
| 28
| 32
| 33
| 36

## From

Expand Down Expand Up @@ -156,12 +156,12 @@ SELECT account_number, acc.age
FROM accounts acc
```

| id | account_number | age
:--- | :--- | :---
0 | 1 | 32
1 | 6 | 36
2 | 13 | 28
3 | 18 | 33
| account_number | age
| :--- | :---
| 1 | 32
| 6 | 36
| 13 | 28
| 18 | 33

*Example 2*: Use index patterns to query indices that match a specific pattern:

Expand All @@ -170,12 +170,12 @@ SELECT account_number
FROM account*
```

| id | account_number
:--- | :---
0 | 1
1 | 6
2 | 13
3 | 18
| account_number
:---
| 1
| 6
| 13
| 18

## Where

Expand Down Expand Up @@ -205,21 +205,21 @@ FROM accounts
WHERE account_number = 1
```

| id | account_number
:--- | :---
0 | 1
| account_number
| :---
| 1

*Example 2*: OpenSearch allows for flexible schema so documents in an index may have different fields. Use `IS NULL` or `IS NOT NULL` to retrieve only missing fields or existing fields. We do not differentiate between missing fields and fields explicitly set to `NULL`:
*Example 2*: OpenSearch allows for flexible schema so documents in an index may have different fields. Use `IS NULL` or `IS NOT NULL` to retrieve only missing fields or existing fields. We do not differentiate between missing fields and fields explicitly set to `NULL`:

```sql
SELECT account_number, employer
FROM accounts
WHERE employer IS NULL
```

| id | account_number | employer
:--- | :--- | :---
0 | 18 |
| account_number | employer
| :--- | :---
| 18 |

*Example 3*: Deletes a document that satisfies the predicates in the `WHERE` clause:

Expand Down Expand Up @@ -307,12 +307,12 @@ FROM accounts
ORDER BY account_number DESC
```

| id | account_number
:--- | :---
0 | 18
1 | 13
2 | 6
3 | 1
| account_number
| :---
| 18
| 13
| 6
| 1

*Example 2*: Specify if documents with missing fields are to be put at the beginning or at the end of the results. The default behavior of OpenSearch is to return nulls or missing fields at the end. To push them before non-nulls, use the `IS NOT NULL` operator:

Expand All @@ -322,12 +322,12 @@ FROM accounts
ORDER BY employer IS NOT NULL
```

| id | employer
:--- | :---
0 |
1 | Netagy
2 | Pyrami
3 | Quility
| employer
| :---
||
| Netagy
| Pyrami
| Quility

## Limit

Expand All @@ -341,9 +341,9 @@ FROM accounts
ORDER BY account_number LIMIT 1
```

| id | account_number
:--- | :---
0 | 1
| account_number
| :---
| 1

*Example 2*: If you pass in two arguments, the first is mapped to the `from` parameter and the second to the `size` parameter in OpenSearch. You can use this for simple pagination for small indices, as it's inefficient for large indices.
Use `ORDER BY` to ensure the same order between pages:
Expand All @@ -354,6 +354,6 @@ FROM accounts
ORDER BY account_number LIMIT 1, 1
```

| id | account_number
:--- | :---
0 | 6
| account_number
| :---
| 6
8 changes: 4 additions & 4 deletions docs/sql/complex.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ The `explain` output is complicated, because a `JOIN` clause is associated with
Result set:

| a.account_number | a.firstname | a.lastname | e.id | e.name
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
:--- | :--- | :--- | :--- | :---
6 | Hattie | Bond | 6 | Jane Smith

### Example 2: Cross join
Expand All @@ -167,7 +167,7 @@ JOIN employees_nested e
Result set:

| a.account_number | a.firstname | a.lastname | e.id | e.name
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
:--- | :--- | :--- | :--- | :---
1 | Amber | Duke | 3 | Bob Smith
1 | Amber | Duke | 4 | Susan Smith
1 | Amber | Duke | 6 | Jane Smith
Expand Down Expand Up @@ -199,7 +199,7 @@ LEFT JOIN employees_nested e
Result set:

| a.account_number | a.firstname | a.lastname | e.id | e.name
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
:--- | :--- | :--- | :--- | :---
1 | Amber | Duke | null | null
6 | Hattie | Bond | 6 | Jane Smith
13 | Nanette | Bates | null | null
Expand Down Expand Up @@ -350,7 +350,7 @@ Explain:
Result set:

| a1.firstname | a1.lastname | a1.balance
:--- | :--- | :--- | :--- | :--- | :---
:--- | :--- | :---
Amber | Duke | 39225
Nanette | Bates | 32838

Expand Down
Loading