Merge branch 'main' into ai-assistant-apis

docs/AI-for-security/ai-security-assistant.asciidoc

@@ -68,7 +68,7 @@ You can also chat with AI Assistant from several particular pages in {elastic-se
 * <<data-quality-dash, Data Quality dashboard>>: Select the *Incompatible fields* tab, then click *Chat*. (This is only available for fields marked red, indicating they're incompatible).
 * <<timelines-ui, Timeline>>: Select the *Security Assistant* tab.
 
-NOTE: Each user's chat history and custom quick prompts are automatically saved, so you can leave ((elastic-sec)) and return to pick up a conversation later.
+NOTE: Each user's chat history and custom quick prompts are automatically saved, so you can leave {elastic-sec} and return to pick up a conversation later.
 
 [discrete]
 [[interact-with-assistant]]
@@ -96,8 +96,10 @@ Quick prompt availability varies based on context — for example, the **Alert s
 ** *Add to existing case* (image:images/icon-add-to-case.png[Add to case icon,19,16]): Add a comment to an existing case using the selected text.
 ** *Copy to clipboard* (image:images/icon-copy.png[Copy to clipboard icon,17,18]): Copy the text to clipboard to paste elsewhere. Also helpful for resubmitting a previous prompt.
 ** *Add to timeline* (image:images/icon-add-to-timeline.png[Add to timeline icon,17,18]): Add a filter or query to Timeline using the text. This button appears for particular queries in AI Assistant's responses.
-+
-TIP: Be sure to specify which language you'd like AI Assistant to use when writing a query. For example: "Can you generate an Event Query Language query to find four failed logins followed by a successful login?"
+
+Be sure to specify which language you'd like AI Assistant to use when writing a query. For example: "Can you generate an Event Query Language query to find four failed logins followed by a successful login?"
+
+TIP: AI Assistant can remember particular information you tell it to remember. For example, you could tell it: "When anwering any question about srv-win-s1-rsa or an alert that references it, mention that this host is in the New York data center". This will cause it to remember the detail you highlighted.
 
 [discrete]
 [[configure-ai-assistant]]
@@ -123,6 +125,12 @@ NOTE: To delete a custom prompt, open the *Name* drop-down menu, hover over the
 [[ai-assistant-anonymization]]
 === Anonymization
 
+.Requirements
+[sidebar]
+--
+To modify Anonymization settings, you need the **Elastic AI Assistant: All** privilege, with **Customize sub-feature privileges** enabled.
+--
+
 The **Anonymization** tab of the AI Assistant settings menu allows you to define default data anonymization behavior for events you send to AI Assistant. Fields with **Allowed** toggled on are included in events provided to AI Assistant. **Allowed** fields with **Anonymized** set to **Yes** are included, but with their values obfuscated.
 
 [role="screenshot"]
@@ -139,7 +147,7 @@ When you include a particular event as context, such as an alert from the Alerts
 === Knowledge base
 beta::[]
 
-The **Knowledge base** tab of the AI Assistant settings menu allows you to enable AI Assistant to answer questions about the Elastic Search Query Language ({esql}), and about alerts in your environment.
+The **Knowledge base** tab of the AI Assistant settings menu allows you to enable AI Assistant to answer questions about the Elastic Search Query Language ({esql}), and about alerts in your environment. To use knowledge base, you must <<ml-requirements, enable machine learning>>.
 
 [discrete]
 [[rag-for-esql]]
@@ -153,12 +161,10 @@ IMPORTANT: {esql} queries generated by AI Assistant might require additional val
 
 When this feature is enabled, AI Assistant can help you write an {esql} query for a particular use case, or answer general questions about {esql} syntax and usage. To enable AI Assistant to answer questions about {esql}:
 
-. Enable the Elastic Learned Sparse EncodeR (ELSER). This model provides additional context to the third-party LLM. To learn more, refer to {ml-docs}/ml-nlp-elser.html#download-deploy-elser[Configure ELSER].
-. Initialize the knowledge base by clicking *Initialize*.
-. Turn on the *Knowledge Base* option.
+. Turn on the knowledge base by clicking **Setup**. If the **Setup** button doesn't appear, knowledge base is already enabled.
 . Click *Save*. The knowledge base is now active. A quick prompt for {esql} queries becomes available, which provides a good starting point for your {esql} conversations and questions. 
 
-NOTE: To update AI Assistant so that it uses the most current {esql} documentation to answer your questions, click **Delete** next to **Knowledge Base**, and toggle the **Knowledge Base** slider off and then on. 
+NOTE: AI Assistant's knowledge base gets additional context from {ml-docs}/ml-nlp-elser.html#download-deploy-elser[Elastic Learned Sparse EncodeR (ELSER)].
 
 [discrete]
 [[rag-for-alerts]]
@@ -167,8 +173,8 @@ When this feature is enabled, AI Assistant will receive multiple alerts as conte
 
 To enable RAG for alerts:
 
-. Turn on the **Alerts** setting.
-. Use the slider to select the number of alerts to send to AI Assistant. 
+. Turn on the knowledge base by clicking **Setup**. If the **Setup** button doesn't appear, knowledge base is already enabled.
+. Use the slider to select the number of alerts to send to AI Assistant. Click **Save**.
 +
 [role="screenshot"]
 image::images/knowledge-base-settings.png["AI Assistant's settings menu open to the Knowledge Base tab",75%]

docs/AI-for-security/llm-performance-matrix.asciidoc

@@ -3,14 +3,13 @@
 
 This table describes the performance of various large language models (LLMs) for different use cases in {elastic-sec}, based on our internal testing. To learn more about these use cases, refer to <<attack-discovery, Attack discovery>> or <<security-assistant, AI Assistant>>.
 
-[cols="1,1,1,1,1,1", options="header"]
+[cols="1,1,1,1,1,1,1,1", options="header"]
 |===
-| *Feature*                     | *Model*               |                    |                   |         |                          
-|                               | *Claude 3: Opus*      | *Claude 3.5: Sonnet* | *Claude 3: Haiku* | *GPT-4o* | *GPT-4 Turbo* 
-
-| *Assistant - General*         | Excellent             | Excellent          | Excellent         | Excellent | Excellent     
-| *Assistant - {esql} generation*| Great                 | Great              | Poor              | Excellent | Poor          
-| *Assistant - Alert questions* | Excellent             | Excellent          | Excellent         | Excellent | Poor          
-| *Attack discovery*            | Excellent             | Excellent            | Poor              | Poor      | Good        
+| *Feature*                     | *Model*               |                    |                   |         |                 |                       |                     
+|                               | *Claude 3: Opus*      | *Claude 3.5: Sonnet* | *Claude 3: Haiku* | *GPT-4o* | *GPT-4 Turbo*  | **Gemini 1.5 Pro ** | **Gemini 1.5 Flash** 
+| *Assistant - General*         | Excellent             | Excellent          | Excellent         | Excellent | Excellent     | Excellent             | Excellent 
+| *Assistant - {esql} generation*| Great                 | Great              | Poor              | Excellent | Poor          | Good                 | Poor 
+| *Assistant - Alert questions* | Excellent             | Excellent          | Excellent         | Excellent | Poor          | Excellent             | Good 
+| *Attack discovery*            | Excellent             | Excellent            | Poor              | Poor      | Good        | Great                 | Poor 
 |===
 

docs/AI-for-security/usecase-attack-discovery-ai-assistant-incident-reporting.asciidoc

@@ -23,7 +23,7 @@ Attack discovery can detect a wide range of threats by finding relationships amo
 
 image::images/attck-disc-11-alerts-disc.png[An Attack discovery card showing an attack with 11 related alerts,90%]
 
-In the example above, Attack discovery found connections between eleven alerts, and used them to identify and describe an attack chain.
+In the example above, Attack discovery found connections between nine alerts, and used them to identify and describe an attack chain.
 
 After Attack discovery outlines your threat landscape, use Elastic AI Assistant to quickly analyze a threat in detail. 
 
@@ -33,6 +33,8 @@ After Attack discovery outlines your threat landscape, use Elastic AI Assistant
 
 From a discovery on the Attack discovery page, click **View in AI Assistant** to start a chat that includes the discovery as context. 
 
+image::images/attck-disc-remediate-sodinokibi.gif[A dialogue with AI Assistant that has the attack discovery as context,90%]
+
 AI Assistant can quickly compile essential data and provide suggestions to help you generate an incident report and plan an effective response. You can ask it to provide relevant data or answer questions, such as “How can I remediate this threat?” or “What {esql} query would isolate actions taken by this user?” 
 
 image::images/attck-disc-esql-query-gen-example.png[An AI Assistant dialogue in which the user asks for a purpose-built {esql} query,90%]
@@ -43,7 +45,7 @@ At any point in a conversation with AI Assistant, you can add data, narrative su
 
 [discrete]
 [[use-case-incident-reporting-create-a-case-using-ai-assistant]]
-== Create a case using AI Assistant
+== Generate reports
 
 From the AI Assistant dialog window, click **Add to case** (image:images/icon-add-to-case.png[Add to case icon,19,16]) next to a message to add the information in that message to a <<cases-overview,case>>. Cases help centralize relevant details in one place for easy sharing with stakeholders.
 
@@ -52,6 +54,9 @@ If you add a message that contains a discovery to a case, AI Assistant automatic
 [discrete]
 [[use-case-incident-reporting-translate]]
 == Translate incident information to a different human language using AI Assistant
+
+image::images/attck-disc-translate-japanese.png[An AI Assistant dialogue in which the assistant translates from English to Japanese,90%]
+
 AI Assistant can translate its findings into other human languages, helping to enable collaboration among global security teams, and making it easier to operate within multilingual organizations. 
 
 After AI Assistant provides information in one language, you can ask it to translate its responses. For example, if it provides remediation steps for an incident, you can instruct it to “Translate these remediation steps into Japanese.” You can then add the translated output to a case. This can help team members receive the same information and insights regardless of their primary language.

docs/advanced-entity-analytics/api/asset-criticality-api-bulk-upsert.asciidoc

@@ -0,0 +1,78 @@
+[[asset-criticality-api-bulk-upsert]]
+=== Bulk upsert (create or update) asset criticality records
+
+Create or update asset criticality records for multiple entities.
+
+If asset criticality records already exist for the entities specified in the request, this API overwrites those records with the specified values.
+
+If asset criticality records don't exist for the specified entities, new records are created.
+
+==== Request URL
+
+`POST <kibana host>:<port>/api/asset_criticality/bulk`
+
+==== Request body
+
+A JSON object defining the asset criticality records.
+
+[width="100%",options="header"]
+|==============================================
+|Name |Type |Description |Required
+|`records` |Array |Array of asset criticality records to be created or updated. The maximum number of records is 1000.
+|Yes
+|`records.id_field` |String |The field that contains the entity ID. This must be either `user.name` or `host.name`.
+|Yes
+|`records.id_value` |String |The ID (host name or user name) of the entity specified in the `records.id_field` field.
+|Yes
+|`records.criticality_level` |String a|The asset criticality level to assign, which must be one of the following:
+
+* `low_impact`
+* `medium_impact`
+* `high_impact`
+* `extreme_impact`
+
+For example, you can assign `extreme_impact` to business-critical entities, or `low_impact` to entities that pose minimal risk to your security posture.
+|Yes
+|==============================================
+
+===== Example requests
+
+[source,console]
+--------------------------------------------------
+POST /api/asset_criticality/bulk
+{
+  "records": [
+    {
+      "id_field": "host.name",
+      "id_value": "my_host",
+      "criticality_level": "medium_impact"
+    },
+    {
+      "id_field": "host.name",
+      "id_value": "my_other_host",
+      "criticality_level": "low_impact"
+    }
+  ]
+}
+--------------------------------------------------
+
+==== Response code
+
+`200`::
+    Indicates a successful call.
+
+==== Example response
+
+Successful response
+
+[source,json]
+--------------------------------------------------
+{
+  "errors": [],
+  "stats": {
+    "successful": 2,
+    "failed": 0,
+    "total": 2
+  }
+}
+--------------------------------------------------

docs/advanced-entity-analytics/api/asset-criticality-api-delete.asciidoc

@@ -0,0 +1,64 @@
+[[delete-criticality-api-delete]]
+=== Delete asset criticality record
+
+Delete a single asset criticality record by ID field and ID value.
+
+==== Request URL
+
+`DELETE <kibana host>:<port>/api/asset_criticality`
+
+==== URL query parameters
+
+[width="100%",options="header"]
+|==============================================
+|Name |Type |Description |Required
+
+|`id_field` |String |The field that contains the entity ID. This must be either `user.name` or `host.name`.
+|Yes
+|`id_value` |String |The ID (host name or user name) of the entity specified in the `id_field` field.
+|Yes
+
+|==============================================
+
+===== Example requests
+
+[source,console]
+--------------------------------------------------
+DELETE /api/asset_criticality?id_field=host.name&id_value=my_host
+
+--------------------------------------------------
+
+==== Response code
+
+`200`::
+    Indicates a successful call. Check the response body to see if the record was deleted.
+
+==== Example responses
+
+*Example 1*
+
+If the record was deleted.
+
+[source,json]
+--------------------------------------------------
+{
+  "deleted": true,
+  "record": {
+    "id_field": "host.name",
+    "id_value": "my_host",
+    "criticality_level": "medium_impact",
+    "@timestamp": "2024-08-05T09:42:11.240Z"
+  }
+}
+--------------------------------------------------
+
+*Example 2*
+
+If the record was not found and could not be deleted.
+
+[source,json]
+--------------------------------------------------
+{
+  "deleted": false
+}
+--------------------------------------------------

docs/advanced-entity-analytics/api/asset-criticality-api-get.asciidoc

@@ -0,0 +1,48 @@
+[[asset-criticality-api-get]]
+=== Get asset criticality record
+
+Retrieve a single asset criticality record by ID field and ID value.
+
+==== Request URL
+
+`GET <kibana host>:<port>/api/asset_criticality`
+
+==== URL query parameters
+
+[width="100%",options="header"]
+|==============================================
+|Name |Type |Description |Required
+
+|`id_field` |String |The field that contains the entity ID. This must be either `user.name` or `host.name`.
+|Yes
+|`id_value` |String |The ID (host name or user name) of the entity specified in the `id_field` field.
+|Yes
+
+|==============================================
+
+===== Example requests
+
+[source,console]
+--------------------------------------------------
+GET /api/asset_criticality?id_field=host.name&id_value=my_host
+
+--------------------------------------------------
+
+==== Response code
+
+`200`::
+    Indicates a successful call.
+`404`::
+    Indicates the criticality record was not found.
+
+==== Example response
+
+[source,json]
+--------------------------------------------------
+{
+  "id_field": "host.name",
+  "id_value": "my_host",
+  "criticality_level": "high_impact",
+  "@timestamp": "2024-08-02T11:15:34.290Z"
+}
+--------------------------------------------------

docs/advanced-entity-analytics/api/asset-criticality-api-index.asciidoc

@@ -0,0 +1,11 @@
+include::asset-criticality-api-overview.asciidoc[]
+
+include::asset-criticality-api-upsert.asciidoc[]
+
+include::asset-criticality-api-bulk-upsert.asciidoc[]
+
+include::asset-criticality-api-get.asciidoc[]
+
+include::asset-criticality-api-list.asciidoc[]
+
+include::asset-criticality-api-delete.asciidoc[]