Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AI Assistant docs for 8.9 #3549

Merged
merged 17 commits into from
Jul 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/assistant/images/add-alert-context.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assistant/images/assistant.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assistant/images/icon-settings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assistant/images/quick-prompts.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assistant/images/system-prompt.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
103 changes: 66 additions & 37 deletions docs/assistant/security-assistant.asciidoc
Original file line number Diff line number Diff line change
@@ -1,67 +1,65 @@
[[security-assistant]]
[chapter]
= Security Assistant
= AI Assistant

:frontmatter-description: The Elastic Security Assistant is a generative AI open-code chat assistant.
:frontmatter-description: The Elastic AI Assistant is a generative AI open-code chat assistant.
:frontmatter-tags-products: [security]
:frontmatter-tags-content-type: [overview]
:frontmatter-tags-user-goals: [get-started]

The Elastic Security Assistant utilizes generative AI to bolster your cybersecurity operations team. It allows users to interact with {elastic-sec} for tasks such as alert investigation, incident response, and query generation or conversion using natural language and much more.
The Elastic AI Assistant utilizes generative AI to bolster your cybersecurity operations team. It allows users to interact with {elastic-sec} for tasks such as alert investigation, incident response, and query generation or conversion using natural language and much more.

A connector for OpenAI and Azure OpenAI Service powers the Security Assistant.
A connector for OpenAI or Azure OpenAI Service powers AI Assistant.

[role="screenshot"]
image::images/assistant.gif[Animation of the Security Assistant chat window,90%]
image::images/assistant.gif[Animation of AI Assistant chat window,90%]

[IMPORTANT]
====
This is an initial release of the Elastic Security Assistant. While designed to enhance your analysis with smart dialogues, its capabilities are still developing. Users should leverage it sensibly as the reliability of its responses might vary. Your insights, patience, and feedback help us calibrate this feature for optimal use. Always cross-verify any returned advice for accurate threat detection and response, insights, and query generation.

Also, the data you provide to the Security Assistant is _not_ anonymized, and is stored and processed by the third-party AI provider. This includes any data used in conversations for analysis or context, such as alert or event data, detection rule configurations, and queries. Therefore, be careful about sharing any confidential or sensitive details while using this feature.
This is an initial release of the Elastic AI Assistant, designed to enhance your analysis with smart dialogues. Its capabilities are still developing. Users should exercise caution as the quality of its responses might vary. Your insights and feedback will help us improve this feature. Always cross-verify AI-generated advice for accuracy.
====

.Requirements
[sidebar]
--
* The Elastic Security Assistant and Generative AI connector are available in {stack} version 8.8.1 and later.
* The Elastic AI Assistant and Generative AI connector are available in {stack} version 8.8.1 and later.

* This feature requires an https://www.elastic.co/pricing[Enterprise subscription].

* You must have an account with a third-party generative AI provider, which the Security Assistant uses to generate responses. Supported providers are OpenAI (`gpt-3.5-turbo` model) and Azure OpenAI Service (any model).
* You must have an account with a third-party generative AI provider, which AI Assistant uses to generate responses. Supported providers are OpenAI and Azure OpenAI Service.
--

[discrete]
[[data-information]]
== Your data and AI Assistant

Elastic does not store or examine prompts or results used by AI Assistant, or use this data for model training. This includes anything you send the model, such as alert or event data, detection rule configurations, queries, and prompts. However, any data you provide to AI Assistant will be processed by the third-party provider you chose when setting up the Generative AI connector as part of the assistant setup.

Elastic does not control third-party tools, and assumes no responsibility or liability for their content, operation, or use, nor for any loss or damage that may arise from your using such tools. Please exercise caution when using AI tools with personal, sensitive, or confidential information. Any data you submit may be used by the provider for AI training or other purposes. There is no guarantee that the provider will keep any information you provide secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative AI tools prior to use.

NOTE: Elastic can automatically anonymize event data that you provide to AI Assistant as context. To learn more, refer to <<configure-ai-assistant, Configure AI Assistant>>.
benironside marked this conversation as resolved.
Show resolved Hide resolved


[discrete]
[[set-up-ai-assistant]]
== Set up the Security Assistant
== Set up AI Assistant

You must complete these steps before you can use the Security Assistant:
You must complete these steps before you can use AI Assistant:

. Create an API key with your AI provider to authenticate requests from the Security Assistant. You'll use this in a later step. Refer to the provider's documentation for generating API keys:
. Create an API key with your AI provider to authenticate requests from AI Assistant. You'll use this in the next step. Refer to the provider's documentation for generating API keys:
+
* https://platform.openai.com/docs/api-reference[OpenAI]
* https://learn.microsoft.com/en-us/azure/cognitive-services/openai/reference[Azure OpenAI Service]

. Add the following feature flag to {kib}'s configuration settings:
+
`xpack.securitySolution.enableExperimental: ['assistantEnabled']`
+
The configuration method depends on your deployment type:
+
* *Self-managed (on-premises) deployments*: Add the feature flag to the `kibana.yml` file, which is used to {kibana-ref}/settings.html[configure {kib}], then restart {kib}.
* *{ecloud} deployments*: Use the YAML editor in the {ecloud} console to add the feature flag to {cloud}/ec-manage-kibana-settings.html[{kib} user settings].

. Create a {kibana-ref}/gen-ai-action-type.html[Generative AI connector] using the AI provider's API key and URL to configure communication between {elastic-sec} and the provider. You can do this in {kib} from *Stack Management* -> *Connectors*, or from within the Security Assistant.
+
NOTE: The Generative AI connector type requires the `assistantEnabled` feature flag for use.
. Create a {kibana-ref}/gen-ai-action-type.html[Generative AI connector] using the AI provider's API key and URL to authenticate communication between {elastic-sec} and the provider. You can do this in {kib} from *Stack Management* -> *Connectors*, or from within AI Assistant.

[discrete]
[[start-chatting]]
== Start chatting

To open the Security Assistant, press *Cmd + ;* (or *Ctrl + ;* in Windows) from anywhere in the {security-app}. This opens the *Welcome* chat interface, where you can ask general questions about {elastic-sec}.
To open AI Assistant, press *Cmd + ;* (or *Ctrl + ;* on Windows) from anywhere in the {security-app}. This opens the *Welcome* chat interface, where you can ask general questions about {elastic-sec}.

You can also chat with the Security Assistant from several areas in {elastic-sec}, and context-specific data and prompts will populate your conversation.
You can also chat with AI Assistant from several particular pages in {elastic-sec} where you can easily send context-specific data and prompts to AI Assistant.

* <<view-alert-details, Alert details>> or Event details flyout: Click *Chat* while viewing the details of an alert or event.
* <<rules-ui-management, Rules page>>: Select one or more rules, then click the magic wand icon (🪄✨) at the top of the page next to the *Rules* title.
Expand All @@ -72,29 +70,60 @@ NOTE: All chat history and custom quick prompts persist in local browser storage

[discrete]
[[interact-with-assistant]]
== Interact with the Security Assistant
== Interact with AI Assistant

Use these features to adjust and act on your conversations with the Security Assistant:
Use these features to adjust and act on your conversations with AI Assistant:

* Select a _system prompt_ at the beginning of a conversation to establish how detailed and technical you want the Security Assistant's answers to be.
* Select a _system prompt_ at the beginning of a conversation to establish how detailed and technical you want AI Assistant's answers to be.
+
[role="screenshot"]
image::images/system-prompt.gif[The system prompt drop-down menu,90%]
+
NOTE: The system prompt is only configurable at the start of a conversation. To reconfigure it, clear the chat and start a new conversation.
System prompts provide context to the model, informing its response. To create a custom system prompt, open the system prompts dropdown menu and click *+ Add new system prompt...*.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find the + Add new system prompt... label, but I did see + Add quick prompt... in the BC5:
prompt

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
System prompts provide context to the model, informing its response. To create a custom system prompt, open the system prompts dropdown menu and click *+ Add new system prompt...*.
System prompts provide context to the model, informing its response. To create a custom system prompt, open the system prompts drop-down menu and click *+ Add new system prompt...*.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find the + Add new system prompt... label, but I did see + Add quick prompt... in the BC5

Add new system prompt is inside the system prompt drop-down. The instructions do say to "open the system prompts drop-down" to click this, but maybe it could be clearer? The animated GIF also shows this, but slightly cuts off the Add new system prompt option at the bottom.


* Select a _quick prompt_ at the bottom of the chat window to get help writing a prompt for a specific purpose, such as summarizing an alert or converting a query from a legacy SIEM to {elastic-sec}. Available quick prompts vary based on context. You can also add custom quick prompts for questions you frequently ask the Security Assistant.
* Select a _quick prompt_ at the bottom of the chat window to get help writing a prompt for a specific purpose, such as summarizing an alert or converting a query from a legacy SIEM to {elastic-sec}.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be helpful to let users know that they'll need to click the Submit message button after selecting a prompt. I know it's a little hand-holdy, but it could be a small detail that new users miss.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is a little less explicitly hand-holdy, and includes other things the user can do:

Suggested change
* Select a _quick prompt_ at the bottom of the chat window to get help writing a prompt for a specific purpose, such as summarizing an alert or converting a query from a legacy SIEM to {elastic-sec}.
* Select a _quick prompt_ at the bottom of the chat window to get help writing a prompt for a specific purpose, such as summarizing an alert or converting a query from a legacy SIEM to {elastic-sec}. This adds a suggested prompt to the entry area, which you can edit before submitting.

+
[role="screenshot"]
image::images/quick-prompts.png[Quick prompts highlighted below a conversation,90%]
+
Quick prompt availability varies based on context — for example, the **Alert summarization** quick prompt appears when you open AI Assistant while viewing an alert. To customize existing quick prompts and create new ones, click *Add Quick prompt*.

* Use these buttons to perform actions in the conversation history and prompt entry area:

** *Add note to timeline* (image:images/icon-add-note.png[Add note icon,16,16]): Create a note in Timeline using the selected text.
** *Add note to timeline* (image:images/icon-add-note.png[Add note icon,16,16]): Add the selected text to your currently active Timeline as a note.
** *Add to existing case* (image:images/icon-add-to-case.png[Add to case icon,19,16]): Add a comment to an existing case using the selected text.
** *Copy to clipboard* (image:images/icon-copy.png[Copy to clipboard icon,17,18]): Copy the text to clipboard to paste elsewhere. This is also helpful for resubmitting a previous prompt.
** *Add to timeline* (image:images/icon-add-to-timeline.png[Copy to clipboard icon,17,18]): Add a filter or query to Timeline using the text. This button appears for certain queries in the Security Assistant's responses.
** *Copy to clipboard* (image:images/icon-copy.png[Copy to clipboard icon,17,18]): Copy the text to clipboard to paste elsewhere. Also helpful for resubmitting a previous prompt.
** *Add to timeline* (image:images/icon-add-to-timeline.png[Copy to clipboard icon,17,18]): Add a filter or query to Timeline using the text. This button appears for particular queries in AI Assistant's responses.
+
TIP: Be sure to specify which language you'd like the Security Assistant to use for queries. For example: "Can you generate an Event Query Language query to find four failed logins followed by a successful login?"
TIP: Be sure to specify which language you'd like AI Assistant to use when writing a query. For example: "Can you generate an Event Query Language query to find four failed logins followed by a successful login?"
** *Clear chat* (image:images/icon-clear-red.png[Red X icon,16,16]): Delete the conversation history and start a new chat.
** *Conversation settings* (image:images/icon-settings.png[Settings icon,17,17]): Choose the Generative AI connector that the Security Assistant uses, or create a new connector.

[discrete]
[[configure-ai-assistant]]
== Configure AI Assistant
The *Settings* menu (image:images/icon-settings.png[Settings icon,17,17]) allows you to configure default conversations, quick prompts, system prompts, and data anonymization.

[role="screenshot"]
image::images/assistant-settings-menu.png[AI Assistant's settings menu, open to the Conversations tab]

The *Settings* menu has four tabs:

* **Conversations:** When you open AI Assistant from certain pages, such as Timeline or Alerts, it defaults to the relevant conversation type. Choose the default system prompt for each conversation type, the connector, and model (if applicable).
* **Quick Prompts:** Modify existing quick prompts or create new ones. To create a new quick prompt, type a unique name in the *Name* field, then press *enter*. Under *Prompt*, enter or update the quick prompt's text. Under *Contexts*, select where the quick prompt should appear.
* **System Prompts:** Edit existing system prompts or create new ones. To create a new system prompt, type a unique name in the *Name* field, then press *enter*. Under *Prompt*, enter or update the system prompt's text. Under *Contexts*, select where the system prompt should appear.
+
NOTE: To delete a custom prompt, open the *Name* drop-down menu, hover over the prompt you want to delete, and click the *X* that appears. You cannot delete the default prompts.

* **Anonymization:** When you provide an event to AI Assistant as context, you can select fields to include as plaintext, to obfuscate, and to not send. The **Anonymization** tab allows you to define default data anonymization behavior. You can update these settings for individual events when you include them in the chat.
+
[role="screenshot"]
image::images/assistant-anonymization-menu.png[AI Assistant's settings menu, open to the Anonymization tab]
+
The fields on this list are among those most likely to provide relevant context to AI Assistant. Fields with *Allowed* toggled on are included. *Allowed* fields with *Anonymized* set to *Yes* are included, but with their values obfuscated.
+
[role="screenshot"]
image::images/add-alert-context.gif[A video that shows an alert being added as context to an AI Assistant chat message]
+
When you include a particular event as context, you can use a similar interface to adjust anonymization behavior. Be sure the anonymization behavior meets your specifications before sending a message with the event attached.

The *Show anonymized* toggle controls whether you see the obfuscated or plaintext versions of the fields you sent to AI Assistant. It doesn't control what gets obfuscated — that's determined by the anonymization settings. It also doesn't affect how event fields appear _before_ being sent to AI Assistant. Instead, it controls how fields that were already sent and obfuscated appear to you.