Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.x] [DOCS] Add rollup V2 API docs (#65398) #65508

Merged
merged 1 commit into from
Nov 25, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,9 @@ testClusters.integTest {
if (singleNode().testDistribution == DEFAULT) {
setting 'xpack.license.self_generated.type', 'trial'
setting 'indices.lifecycle.history_index_enabled', 'false'
if (BuildParams.isSnapshotBuild() == true) {
systemProperty 'es.rollup_v2_feature_flag_enabled', 'true'
}
if (BuildParams.isSnapshotBuild() == false) {
systemProperty 'es.autoscaling_feature_flag_registered', 'true'
}
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/rest-api/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ include::{es-repo-dir}/ml/df-analytics/apis/index.asciidoc[]
include::{es-repo-dir}/migration/migration.asciidoc[]
include::{es-repo-dir}/indices/apis/reload-analyzers.asciidoc[]
include::{es-repo-dir}/repositories-metering-api/repositories-metering-apis.asciidoc[]
include::{es-repo-dir}/rollup/rollup-api.asciidoc[]
include::{es-repo-dir}/rollup/rollup-apis.asciidoc[]
include::{es-repo-dir}/search.asciidoc[]
include::{es-repo-dir}/searchable-snapshots/apis/searchable-snapshots-apis.asciidoc[]
include::{xes-repo-dir}/rest-api/security.asciidoc[]
Expand Down
187 changes: 187 additions & 0 deletions docs/reference/rollup/apis/rollup-api.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
[role="xpack"]
[testenv="basic"]
[[rollup-api]]
=== Rollup API
++++
<titleabbrev>Rollup</titleabbrev>
++++

Aggregates an index's time-series data and stores the results in another index.
For example, you can roll up hourly data into daily or weekly summaries.

[source,console]
----
POST /my-index-000001/_rollup
{
"rollup_index": "my-rollup-index",
"groups": {
"date_histogram": {
"field": "@timestamp",
"calendar_interval": "1d"
},
"terms": {
"fields": [
"my-keyword-field",
"my-other-keyword-field"
]
}
},
"metrics": [
{
"field": "my-numeric-field",
"metrics": [
"min",
"max",
"avg"
]
}
]
}
----
// TEST[setup:my_index]

[[rollup-api-request]]
==== {api-request-title}

`PUT /<index>/_rollup`

[[rollup-api-prereqs]]
==== {api-prereq-title}

If the {es} {security-features} are enabled, you must have the `manage` cluster
privilege to use this API. See <<security-privileges>>.

[[rollup-api-path-params]]
==== {api-path-parms-title}

`<index>`::
(Required, string)
Index to roll up. Cannot be a <<data-streams,data stream>> or
<<indices-aliases,index alias>>. Does not support <<multi-index,multi-target
syntax>> or wildcards (`*`).

[role="child_attributes"]
[[rollup-api-request-body]]
==== {api-request-body-title}

`rollup_index`::
(Required, string)
Index that stores the rollup results. Cannot be a <<data-streams,data stream>>
or <<indices-aliases,index alias>>.
+
If this index does not exist, the request creates it. If the `<index>` is a
backing index for a data stream, this new index is a backing index for the same
stream.

`groups`::
(Required, object)
Aggregates and stores fields in the rollup.
+
.Properties of `groups`
[%collapsible%open]
====
`date_histogram`::
(Required,
<<search-aggregations-bucket-datehistogram-aggregation,`date_histogram`
aggregation>> object)
Groups documents based on a provided time interval.
+
.Properties of `date_histogram`
[%collapsible%open]
=====
`field`::
(Required, string)
<<date,`date`>> or <<date_nanos,`date_nanos`>> field containing a timestamp. If
you're rolling up a backing index or using the {ecs-ref}[Elastic Common Schema
(ECS)], we recommend using `@timestamp`.

`calendar_interval` or `fixed_interval`::
(Required, <<time-units,time units>>)
Time interval used to group documents. For differences between
`calendar_interval` and `fixed_interval`, see <<calendar_and_fixed_intervals>>.
+
TIP: Choose this value carefully. You won't be able to use a smaller interval
later. For example, you can't aggregate daily rollups into hourly
summaries. However, smaller time intervals can greatly increase the size of your
`rollup_index`.

`time_zone`::
(Optional, string)
Time zone for the `field`. Valid values are ISO 8601 UTC offsets, such as
`+01:00` or `-08:00`, and IANA time zone IDs, such as `America/Los_Angeles`.
Defaults to `+00:00` (UTC).
=====

`histogram`::
(Optional, <<search-aggregations-bucket-histogram-aggregation,`histogram`
aggregation>> object)
Groups and stores <<number,numeric>> field values based on a provided interval.
+
.Properties of `histogram`
[%collapsible%open]
=====
`fields`::
(Required*, array of strings)
Array of <<number,numeric>> fields to group. If you specify a `histogram`
object, this property is required.

`interval`::
(Required*, integer)
Numeric interval used to group the `fields`. If you specify a `histogram`
object, this property is required.
=====

`terms`::
(Optional, <<search-aggregations-bucket-terms-aggregation,`terms`
aggregation>> object)
Stores values for <<keyword,keyword family>> and <<number,numeric>> fields.
+
.Properties of `terms`
[%collapsible%open]
=====
`fields`::
(Required*, array of strings)
Array of <<keyword,keyword family>> and <<number,numeric>> fields to store. If
you specify a `terms` object, this property is required.
+
TIP: Avoid storing high-cardinality fields. High-cardinality fields can greatly
increase the size of your `rollup_index`.
=====
====

`metrics`::
(Optional, array of objects)
Collects and stores metrics for <<number,numeric>> fields.
+
.Properties of `metrics` objects
[%collapsible%open]
====
`field`::
(Required*, string)
<<number,Numeric>> field to collect metrics for. If you specify a `metrics`
object, this property is required.

`metrics`::
(Required*, array)
Array of metrics to collect. Each value corresponds to a
<<search-aggregations-metrics,metric aggregation>>. Valid values are
<<search-aggregations-metrics-min-aggregation,`min`>>,
<<search-aggregations-metrics-max-aggregation,`max`>>,
<<search-aggregations-metrics-sum-aggregation,`sum`>>,
<<search-aggregations-metrics-avg-aggregation,`avg`>>, and
<<search-aggregations-metrics-valuecount-aggregation,`value_count`>>. You must
specify at least one value. If you specify a `metrics` object, this property is
required.
+
NOTE: The `avg` metric stores both the `sum` and `value_count` values. This lets
you accurately average rollups over larger time intervals. For example, you can
accurately roll up hourly averages into daily averages.
====

`page_size`::
(Optional, integer)
Maximum number of rollup results to process at once. Defaults to `1000`. Larger
values run faster but require more memory.
+
NOTE: This argument only affects the speed and memory usage of the rollup
operation. It does not affect the rollup results.
35 changes: 0 additions & 35 deletions docs/reference/rollup/rollup-api.asciidoc

This file was deleted.

88 changes: 88 additions & 0 deletions docs/reference/rollup/rollup-apis.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
[role="xpack"]
[testenv="basic"]
[[rollup-apis]]
== Rollup APIs

ifdef::permanently-unreleased-branch[]

A rollup aggregates an index's time-series data and stores the results in
another index, called a rollup index. For example, you can roll up hourly data
into daily or weekly summaries.

* <<rollup-api,Rollup API>>

[discrete]
[[legacy-rollup-apis]]
=== Legacy rollup APIs

Prior to {es} 7.x, you could only create rollups using periodic cron jobs.
Special APIs were required to manage the jobs and search the resulting rollup
indices. These rollup APIs are now deprecated and will be removed in a future
release.

[discrete]
[[rollup-jobs-endpoint]]
==== Jobs

* <<rollup-put-job,Create>> or <<rollup-delete-job,delete {rollup-jobs}>>
* <<rollup-start-job,Start>> or <<rollup-stop-job,stop {rollup-jobs}>>
* <<rollup-get-job,Get {rollup-jobs}>>

[discrete]
[[rollup-data-endpoint]]
==== Data

* <<rollup-get-rollup-caps,Get rollup capabilities>>
* <<rollup-get-rollup-index-caps,Get rollup index capabilities>>

[discrete]
[[rollup-search-endpoint]]
==== Search

* <<rollup-search,Rollup search>>

include::apis/rollup-api.asciidoc[]
include::apis/put-job.asciidoc[]
include::apis/delete-job.asciidoc[]
include::apis/get-job.asciidoc[]
include::apis/rollup-caps.asciidoc[]
include::apis/rollup-index-caps.asciidoc[]
include::apis/rollup-search.asciidoc[]
include::apis/start-job.asciidoc[]
include::apis/stop-job.asciidoc[]

endif::[]
ifndef::permanently-unreleased-branch[]

[discrete]
[[rollup-jobs-endpoint]]
=== Jobs

* <<rollup-put-job,Create>> or <<rollup-delete-job,delete {rollup-jobs}>>
* <<rollup-start-job,Start>> or <<rollup-stop-job,stop {rollup-jobs}>>
* <<rollup-get-job,Get {rollup-jobs}>>

[discrete]
[[rollup-data-endpoint]]
=== Data

* <<rollup-get-rollup-caps,Get rollup capabilities>>
* <<rollup-get-rollup-index-caps,Get rollup index capabilities>>

[discrete]
[[rollup-search-endpoint]]
=== Search

* <<rollup-search,Rollup search>>


include::apis/put-job.asciidoc[]
include::apis/delete-job.asciidoc[]
include::apis/get-job.asciidoc[]
include::apis/rollup-caps.asciidoc[]
include::apis/rollup-index-caps.asciidoc[]
include::apis/rollup-search.asciidoc[]
include::apis/start-job.asciidoc[]
include::apis/stop-job.asciidoc[]

endif::[]