Skip to content

Commit

Permalink
Im plugin index rollup (opensearch-project#1813)
Browse files Browse the repository at this point in the history
* Added cluster permissions to cluster permissions list.

Signed-off-by: carolxob <[email protected]>

* Revert "Added cluster permissions to cluster permissions list."

This reverts commit 48a9fb5.

* Quick replace for some instances of rollup name.

Signed-off-by: carolxob <[email protected]>

* Fixes ISM bug

Signed-off-by: Fanit Kolchina <[email protected]>

Signed-off-by: carolxob <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Co-authored-by: Fanit Kolchina <[email protected]>
  • Loading branch information
carolxob and kolchfa-aws authored Nov 8, 2022
1 parent d8ff91b commit 333d138
Showing 1 changed file with 7 additions and 3 deletions.
10 changes: 7 additions & 3 deletions _im-plugin/index-rollups/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -520,14 +520,18 @@ The `doc_count` field in bucket aggregations contains the number of documents co

## Dynamic target index

In ISM rollup, the `target_index` field may contain a template that is compiled at the time of each rollup indexing. For example, if you specify the `target_index` field as `rollup_ndx-{{ctx.source_index}}`, the source index `log-000001` will roll up into a target index `rollup_ndx-log-000001`. This allows you to roll up data into multiple time-based indices, with one rollup job created for each source index.
<style>
.nobr { white-space: nowrap }
</style>

In ISM rollup, the `target_index` field may contain a template that is compiled at the time of each rollup indexing. For example, if you specify the `target_index` field as <span style="white-space: nowrap">`{% raw %}rollup_ndx-{{ctx.source_index}}{% endraw %}`,</span> the source index `log-000001` will roll up into a target index `rollup_ndx-log-000001`. This allows you to roll up data into multiple time-based indices, with one rollup job created for each source index.

The `source_index` parameter in {% raw %}`{{ctx.source_index}}`{% endraw %} cannot contain wildcards.
{: .note}

## Searching multiple rollup indices

When data is rolled up into multiple target indices, you can run one search across all of the rollup indices. To search multiple target indices that have the same rollup, specify the index names as a comma-separated list or a wildcard pattern. For example, with `target_index` as `rollup_ndx-{{ctx.source_index}}` and source indices that start with `log`, specify the `rollup_ndx-log*` pattern. Or, to search for rolled up log-000001 and log-000002 indices, specify the `rollup_ndx-log-000001,rollup_ndx-log-000002` list.
When data is rolled up into multiple target indices, you can run one search across all of the rollup indices. To search multiple target indices that have the same rollup, specify the index names as a comma-separated list or a wildcard pattern. For example, with `target_index` as <span style="white-space: nowrap">`{% raw %}rollup_ndx-{{ctx.source_index}}{% endraw %}`</span> and source indices that start with `log`, specify the `rollup_ndx-log*` pattern. Or, to search for rolled up log-000001 and log-000002 indices, specify the `rollup_ndx-log-000001,rollup_ndx-log-000002` list.

You cannot search a mix of rollup and non-rollup indices with the same query.
{: .note}
Expand Down Expand Up @@ -580,7 +584,7 @@ PUT _plugins/_ism/policies/rollover_policy
{
"rollup": {
"ism_rollup": {
"target_index": "rollup_ndx-{{ctx.source_index}}",
"target_index": {% raw %}"rollup_ndx-{{ctx.source_index}}"{% endraw %},
"description": "Example rollup job",
"page_size": 200,
"dimensions": [
Expand Down

0 comments on commit 333d138

Please sign in to comment.