[8.2] Re-add Customise integration data retention documentation (backport #1724)

docs/en/ingest-management/data-streams.asciidoc

@@ -98,17 +98,145 @@ These templates are loaded when the integration is installed, and are used to co
 [[data-streams-ilm]]
 == Configure an index lifecycle management (ILM) policy
 
-Use the {ref}/getting-started-index-lifecycle-management.html[index lifecycle
+Use the {ref}/index-lifecycle-management.html[index lifecycle
 management] (ILM) feature in {es} to manage your {agent} data stream indices as they age.
 For example, create a new index after a certain period of time,
 or delete stale indices to enforce data retention standards.
 
-{agent} uses ILM policies built-in to {es} to manage backing indices for its data streams.
-See the {ref}/example-using-index-lifecycle-policy.html[Customize built-in ILM policies] tutorial
-to learn how to customize these policies based on your performance, resilience, and retention requirements.
+Installed integrations may have one or many associated data streams--each with an associated ILM policy.
+By default, these data streams use an ILM policy that matches their data type.
+For example, the data stream `metrics-system.logs-*`,
+uses the metrics ILM policy as defined in the `metrics-system.logs` index template.
 
-To instead create a new ILM policy, in {kib},
-go to **Stack Management** > **Index Lifecycle Policies**. Click **Create policy**.
-Define data tiers for your data, and any associated actions,
-like a rollover, freeze, or shrink.
-See {ref}/set-up-lifecycle-policy.html[configure a lifecycle policy] for more information.
+[discrete]
+[[data-streams-ilm-tutorial]]
+== Tutorial: Customize data retention for integrations
+
+This tutorial explains how to apply a custom ILM policy to an integration's data stream.
+
+**Scenario:** You have {agent}s collecting system metrics with the System integration in two environments--one with the namespace `development`, and one with `production`.
+
+**Goal:** Customize the ILM policy for the `system.network` data stream in the `production` namespace.
+Specifically, apply the built-in `90-days-default` ILM policy so that data is deleted after 90 days.
+
+[discrete]
+[[data-streams-ilm-one]]
+=== Step 1: View data streams
+
+The **Data Streams** view in {kib} shows you the data streams,
+index templates, and ILM policies associated with a given integration.
+
+. Navigate to **Stack Management** > **Index Management** > **Data Streams**.
+. Search for `system` to see all data streams associated with the System integration.
+. Select the `metrics-system.network-{namespace}` data stream to view its associated index template and ILM policy.
+As you can see, the data stream follows the <<data-streams-naming-scheme>> and starts with its type, `metrics-`.
++
+[role="screenshot"]
+image::images/data-stream-info.png[Data streams info]
+
+[discrete]
+[[data-streams-ilm-two]]
+=== Step 2: Create a component template
+
+For your changes to continue to be applied in future versions,
+you must put all custom index settings into a component template.
+The component template must follow the data stream naming scheme,
+and end with `@custom`:
+
+[source,text]
+----
+<type>-<dataset>-<namespace>@custom
+----
+
+For example, to create custom index settings for the `system.network` data stream with a namespace of `production`,
+the component template name would be:
+
+[source,text]
+----
+metrics-system.network-production@custom
+----
+
+. Navigate to **Stack Management** > **Index Management** > **Component Templates**
+. Click **Create component template**.
+. Use the template above to set the name--in this case, `metrics-system.network-production@custom`. Click **Next**.
+. Under **Index settings**, set the ILM policy name under the `lifecycle.name` key:
++
+[source,json]
+----
+{
+  "lifecycle": {
+    "name": "90-days-default"
+  }
+}
+----
+. Continue to **Review** and ensure your request looks similar to the image below.
+If it does, click **Create component template**.
++
+[role="screenshot"]
+image::images/create-component-template.png[Create component template]
+
+[discrete]
+[[data-streams-ilm-three]]
+=== Step 3: Clone and modify the existing index template
+
+Now that you've created a component template,
+you need to create an index template to apply the changes to the correct data stream.
+The easiest way to do this is to duplicate and modify the integration's existing index template.
+
+WARNING: When duplicating the index template, do not change or remove any managed properties. This may result in problems when upgrading.
+
+. Navigate to **Stack Management** > **Index Management** > **Index Templates**.
+. Find the index template you want to clone. The index template will have the `<type>` and `<dataset>` in its name,
+but not the `<namespace>`. In this case, it's `metrics-system.network`.
+. Select **Actions** > **Clone**.
+. Set the name of the new index template to `metrics-system.network-production`.
+. Change the index pattern to include a namespace--in this case, `metrics-system.network-production*`.
+This ensures the previously created component template is only applied to the `production` namespace.
+. Set the priority to `250`. This ensures that the new index template takes precedence over other index templates that match the index pattern.
+. Under **Component templates**, search for and add the component template created in the previous step.
+To ensure your namespace-specific settings are applied over other custom settings,
+the new template should be added below the existing `@custom` template.
+. Create the index template.
+
+[role="screenshot"]
+image::images/create-index-template.png[Create index template]
+
+[discrete]
+[[data-streams-ilm-four]]
+=== Step 4: Roll over the data stream (optional)
+
+To confirm that the data stream is now using the new index template and ILM policy,
+you can either repeat <<data-streams-ilm-one,step one>>, or navigate to **Dev Tools ** and run the following:
+
+[source,bash]
+----
+GET /_data_stream/metrics-system.network-production <1>
+----
+<1> The name of the data stream we've been hacking on
+
+The result should include the following:
+
+[source,json]
+----
+{
+  "data_streams" : [
+    {
+      ...
+      "template" : "metrics-system.network-production", <1>
+      "ilm_policy" : "90-days-default", <2>
+      ...
+    }
+  ]
+}
+----
+<1> The name of the custom index template created in step three
+<2> The name of the ILM policy applied to the new component template in step two
+
+New ILM policies only take effect when new indices are created,
+so you either must wait for a rollover to occur (usually after 30 days or when the index size reaches 50GB),
+or force a rollover using the {ref}/indices-rollover-index.html[{es} rollover API]:
+
+[source,bash]
+----
+POST /metrics-system.network-production/_rollover/
+----