Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ism rollup to documentation #1371

Merged
merged 6 commits into from
Oct 5, 2022
Merged

Add ism rollup to documentation #1371

Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
12936b0
add ism rollup to documentation
ariamarble Sep 30, 2022
ae3b71f
made suggested changes
ariamarble Oct 3, 2022
f233bd1
added field descriptions
ariamarble Oct 4, 2022
3a7b464
added further requested changes
ariamarble Oct 4, 2022
cd11096
tech review updates
ariamarble Oct 5, 2022
c6a5fc8
minor spelling changes and link update
ariamarble Oct 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 128 additions & 0 deletions _im-plugin/ism/policies.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ ISM supports the following operations:
- [snapshot](#snapshot)
- [index_priority](#index_priority)
- [allocation](#allocation)
- [rollup](#rollup)

### force_merge

Expand Down Expand Up @@ -438,6 +439,133 @@ Parameter | Description | Type | Required
]
```

### rollup

[Index rollup]({{site.url}}{{site.baseurl}}/im-plugin/index-rollups/index/) lets you periodically reduce data granularity by rolling up old data into summarized indices.

Only a single rollup job can be created with this method. The source index for the rollup job is inferred from the index the policy it is attached to.
ariamarble marked this conversation as resolved.
Show resolved Hide resolved
ariamarble marked this conversation as resolved.
Show resolved Hide resolved
{: .note }

#### Path and HTTP methods

````bash
POST _plugins/_ism/add/<index>
````

#### Sample ISM rollup policy

````json
{
ariamarble marked this conversation as resolved.
Show resolved Hide resolved
"policy": {
ariamarble marked this conversation as resolved.
Show resolved Hide resolved
"description": "Sample rollup" ,
"default_state": "rollup",
"states": [
{
"name": "rollup",
"actions": [
{
"rollup": {
"ism_rollup": {
"description": "Creating rollup through ISM",
"target_index": "target",
"page_size": 1000,
"dimensions": [
{
"date_histogram": {
"fixed_interval": "60m",
"source_field": "order_date",
"target_field": "order_date",
"timezone": "America/Los_Angeles"
}
},
{
"terms": {
"source_field": "customer_gender",
"target_field": "customer_gender"
}
},
{
"terms": {
"source_field": "day_of_week",
"target_field": "day_of_week"
}
}
],
"metrics": [
{
"source_field": "taxless_total_price",
"metrics": [
{
"sum": {}
}
]
},
{
"source_field": "total_quantity",
"metrics": [
{
"avg": {}
},
{
"max": {}
}
]
}
]
}
}
}
],
"transitions": []
}
]
}
}
````

#### Request Fields

The following request fields are required.

Field Parameter | Data Type | Description | Required
:--- | :--- |:--- |:---
description | String | The description name of the ISM policy. | No
default_state | String | The state that the ISM policy runs by default. Set the description to `rollup` for the ISM rollup policy. | Yes
states | List of objects | Define the states to be used in the ISM rollup policy. | Yes
states.name | List of objects | The description name of the state within the ISM policy. In this case, you will need to match the `default_state` description of "`rollup`." | Yes
states.actions | List of objects | Define the actions to be run by the state. | Yes
states.transition | List of objects | Define the conditions required to move on to the next state. When left blank, the policy completes its management of the index. | Yes
ariamarble marked this conversation as resolved.
Show resolved Hide resolved
states.actions.rollup | List of objects | Define the actions taken during index rollup. | Yes
states.actions.ism_rollup | List of objects | Define the actions taken during ISM rollup. | Yes
states.actions.rollup.ism_rollup.description | String | The description name of the `ism_rollup` field. | No
states.actions.rollup.ism_rollup.target_index | String | The index to be used by the ISM rollup profile. | Yes
states.actions.rollup.ism_rollup.page_size | Number | The number of documents per bucket to be returned per page. | Yes
states.actions.rollup.ism_rollup.dimensions.date_histogram.fixed_interval | Time | Define the interval period. Specify the fixed interval in milliseconds, seconds, minutes, hours, or days. For example, 60 minutes would be expressed as `60m`. | Yes
states.actions.rollup.ism_rollup.dimensions.date_histogram.source_field | String | The name of the field to transform. | Yes
states.actions.rollup.ism_rollup.dimensions.date_histogram.target_field | String | The target field that the newly transformed data is added to. | Yes
states.actions.rollup.ism_rollup.dimensions.date_histogram.timezone | String | The time zone that the rolled up documents are stored in. Default is UTC. | No
states.actions.rollup.ism_rollup.dimensions.terms.source_field | String | The name of the field to transform. | Yes
states.actions.rollup.ism_rollup.dimensions.terms.target_field | String | The target field that the newly transformed data is added to. | Yes
states.actions.rollup.ism_rollup.metrics | List of objects | Define the metrics to be collected during index rollup. | Yes
states.actions.rollup.ism_rollup.metrics.source_field | String | The name of the field to transform. | Yes
states.actions.rollup.ism_rollup.metrics.metrics | List of objects | The metrics to be collected during index rollup. | Yes
states.actions.rollup.ism_rollup.metrics.metrics.sum | Function | Outputs the sum of the aggregated documents tracked values. | No
states.actions.rollup.ism_rollup.metrics.metrics.avg | Function | Outputs the average value of the aggregated documents tracked values. | No
states.actions.rollup.ism_rollup.metrics.metrics.max | Function | Outputs the maximum value of the aggregated documents tracked values. | No
ariamarble marked this conversation as resolved.
Show resolved Hide resolved

#### Adding a rollup policy in Dashboards

To add a rollup policy in Dashboards, follow these steps:

- Select the menu button on the top-left of the Dashboards user interface.
- In the Dashboards menu, select `Index Management`.
- On the next screen select `Rollup jobs`.
- Select the `Create rollup` button.
- Follow the steps in the `Create rollup job` wizard.
- Add a name for the policy in the `Name` box.
- You can reference the above request fields to configure the rollup policy.
- Finally, select the `Create` button on the bottom-right of the Dashboards user interface.

ariamarble marked this conversation as resolved.
Show resolved Hide resolved
---

## Transitions
Expand Down