[DOCS] Add ILM tutorial

docs/management/index-lifecycle-policies/example-index-lifecycle-policy.asciidoc

@@ -1,23 +1,167 @@
 [role="xpack"]
+
 [[example-using-index-lifecycle-policy]]
-=== Example of using an index lifecycle policy
+=== Tutorial:  Use ILM to manage Filebeat time-based indices
+
+With index lifecycle management (ILM), you can create policies to define actions
+that are automatically performed against indices as they age.  In this tutorial,
+you will use {kib}’s *Index Lifecycle Policies* to modify and create ILM policies.
+
+
+[float]
+==== Scenario
+
+You’ve been tasked with sending syslog files to an {es} cluster. This
+project has the following data retention guidelines for log data:
+
+* Keep logs on hot data nodes for 30 days
+* Rollover to a new index if the size reaches 50GB
+* After 30 days:
+** Move the logs to warm data nodes
+** Set replicas to 1
+** Force merge indices to a single segment
+* Delete logs after 90 days
+
+
+[float]
+==== Prerequisites
+
+To complete this tutorial, you'll need:
+
+* An {es} cluster with hot and warm nodes configured for shard allocation
+awareness. If you’re using https://www.elastic.co/guide/en/cloud/current/ec-getting-started-templates-hot-warm.html[Elasticsearch Service],
+choose the hot-warm architecture deployment template.
+
++
+For a self-managed cluster, use https://www.elastic.co/guide/en/elasticsearch/reference/current/allocation-awareness.html[shard allocation awareness]
+to label data nodes as hot or warm. This step is required to migrate shards between
+nodes using shard allocation awareness.
++
+For example, you can set this in `elasticsearch.yml` for each data node:
++
+[source,yaml]
+--------------------------------------------------------------------------------
+node.attr.data: "warm"
+--------------------------------------------------------------------------------
+
+* A server with Filebeat installed and configured to send logs to the `elasticsearch`
+output as described in https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-getting-started.html[Getting Started with Filebeat].
+
+[float]
+==== View the Filebeat ILM policy
+
+Filebeat includes a default ILM policy that enables rollover. Index lifecycle
+management is enabled automatically if you’re using the default `filebeat.yml`
+and index template.
+
+To view the default policy in {kib}, go to *Management > Index Lifecycle Policies*,
+search for _filebeat_, and choose the _filebeat-version_ policy.
+
+This policy initiates the rollover action when the index size reaches 50G or 30
+days old.
+
+[role="screenshot"]
+image::images/tutorial-ilm-hotphaserollover-default.png["Default policy"]
+
+
+[float]
+==== Modify the policy
+
+The default policy is enough to prevent the creation of many tiny daily indices.
+You can modify the policy to meet more complex requirements.
+
+. Activate the warm phase.
+
++
+. Set either of the following options to control when the index moves to the warm phase:
+
+** Provide a value for *Timing for warm phase*. Setting this to *15* will keep the
+indices on hot nodes for a range of  15-45 days, depending on when the initial
+rollover occurred.
+
+** Enable *Move to warm phase on rollover*. The index might move to the warm phase
+more quickly than intended if it reaches the *Maximum index size* before
+*Maximum age*.
+
+. In *Select a node attribute to control shard allocation*, select *data:warm(2)* in
+the dropdown to migrate shards to warm data nodes.
+
+. Change *Number of replicas* to *1*.
+
+. Enable *Force merge data* and set *Number of segments* to *1*.
++
+NOTE:  When rollover is enabled in the hot phase, action timing in the other phases
+is based on the rollover date.
+
++
+[role="screenshot"]
+image::images/tutorial-ilm-modify-default-warm-phase-rollover.png["Modify to add warm phase"]
+
+. Activate the delete phase and set *Timing for delete phase* to *90* days.
++
+[role="screenshot"]
+image::images/tutorial-ilm-delete-rollover.png["Add a delete phase"]
+
+[float]
+==== Create a custom policy
+
+If meeting a specific retention time period is your most important priority, you
+can create a custom policy.  For this option, you will use Filebeat daily indices
+without rollover.
+
+To create a custom policy in {kib}, go to *Management > Index Lifecycle Policies >
+Create Policy*.
+
+
+. Activate the warm phase and configure it as follows:
++
+|===
+|*Setting* |*Value*
+
+|Timing for warm phase
+|30 days from index creation
+
+|Node attribute
+|`data:warm`
+
+|Number of replicas
+|1
+
+|Force merge data
+|enable
+
+|Number of segments
+|1
+|===
+
++
+[role="screenshot"]
+image::images/tutorial-ilm-custom-policy.png["Modify the custom policy to add a warm phase"]
 
-A common use case for managing index lifecycle policies is when you’re using 
-{beats-ref}/beats-reference.html[Beats] to continually send time-series data, 
-such as metrics and log data, to {es}.  When you create the Beats packages, an 
-index template is installed.  The template includes a default policy to apply 
-when new indices are created.  
 
-You can edit the policy in {kib}'s *Index Lifecycle Policies*.  For example, you might:
++
+. Activate the delete phase and set the timing.
++
+|===
+|*Setting* |*Value*
+|Timing for delete phase
+|90
+|===
 
-* Rollover the index when it reaches 50 GB in size or is 30 days old.  These 
-settings are the default for the Beats lifecycle policy. This avoids 
-having 1000s of tiny indices. When a rollover occurs, a new “hot” index is 
-created and added to the index alias.
++
+[role="screenshot"]
+image::images/tutorial-ilm-delete-phase-creation.png["Delete phase"]
 
-* Move the index into the warm phase, shrink the index down to a single shard, 
-and force merge to a single segment.
+. Configure the index to use the new policy in *{kib} > Management > Index Lifecycle
+Policies > _Select your custom policy_ > Actions > Add policy to template > Select
+your Filebeat index template*
 
-* After 60 days, move the index into the cold phase and onto less expensive hardware.
++
+NOTE: If you initially used the default Filebeat ILM policy, you will see a notice that the
+template already has a policy associated with it. Confirm that you want to
+overwrite that configuration.
 
-* Delete the index after 90 days.
+TIP: When you change the policy associated with the index template, the active
+index will continue to use the policy it was associated with at index creation
+unless you manually update it. The next new index will use the updated policy.
+For more reasons that your ILM policy changes might be delayed, see https://www.elastic.co/guide/en/elasticsearch/reference/current/update-lifecycle-policy.html#update-lifecycle-policy[Update Lifecycle Policy].