Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.8] [Security Solution] Elastic Security Assistant (#156933) #158974

Merged
merged 2 commits into from
Jun 2, 2023

Conversation

spong
Copy link
Member

@spong spong commented Jun 2, 2023

Backport

This will backport the following commits from main to 8.8:

Questions ?

Please refer to the Backport tool documentation

## [Security Solution] Elastic Security Assistant

The _Elastic Security Assistant_ has entered the chat, integrating generative AI and large language models (LLMs) into the workflows of Elastic Security users.

Bring your alerts, events, rules, and data quality checks into the conversation.

<https://github.com/elastic/kibana/assets/2946766/31d65c78-5692-4817-b726-820c5df0801c>

This PR merges a feature branch developed by @spong and @andrew-goldstein , seeded by @jamesspi 's prototype of the assistant. Connectivity to LLMs is provided the [Generative AI Connector](<elastic#157228>) , developed by @stephmilovic . This PR includes:

- A new reusable Kibana package containing the assistant: `x-pack/packages/kbn-elastic-assistant`
  - See the `How to embed the Assistant in other parts of Kibana` for details
- Assistant integration into Elastic Security Solution workflows (e.g. alerts, cases, Timeline, rules, data quality)

### An assistant trained on the Elastic stack and Elastic Security

The [Generative AI Connector](<elastic#157228>) connects the assistant to OpenAI and Azure OpenAI models trained with knowledge of the Elastic stack and the Elastic Security solution, including:

- The Elastic open [Detection Rules](https://github.com/elastic/detection-rules)
- The [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/current/index.html)
- Elastic query languages, including [KQL](https://www.elastic.co/guide/en/kibana/current/kuery-query.html), [EQL](https://www.elastic.co/guide/en/elasticsearch/reference/current/eql-syntax.html), and the [Elastic Query DSL](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html)
- [Elasticsearch API documentation](https://www.elastic.co/guide/en/elasticsearch/reference/8.8/rest-apis.html)

This training enables the assistant to offer fully interactive chat experiences that include:

- alert summarization
- interactive query generation
- workflow suggestions
- generating ingestion configurations that conform to the Elastic Common Schema
- your imagination

using context from Elastic Security.

### Take action from your conversations

The Actions (from assistant response):

- Send KQL to Timeline
- Send EQL to Timeline
- Send Elasticsearch DSL to Timeline
- Send Note to timeline
- Create new case
- Add to existing case
- Copy to clipboard

### Components architecture diagram

![Untitled Diagram drawio (1)](https://github.com/elastic/kibana/assets/55110838/8f446313-629f-4646-b44d-0cb0ca74aeaf)

### How to embed the Assistant in other parts of Kibana

Follow the general instructions in `x-pack/packages/kbn-elastic-assistant/index.ts` to integrate the assistant into a Kibana app.

#### Step 1 - Wrap your Kibana app in the `AssistantProvider` component

```ts
// Step 1: Wrap your Kibana app in the `AssistantProvider` component. This typically
// happens in the root of your app. Optionally provide a custom title for the assistant:

/** provides context (from the app) to the assistant, and injects Kibana services, like `http` */
export { AssistantProvider } from './impl/assistant_context';
```

#### Step 2: Add the `AssistantOverlay` component to your app

```ts
// Step 2: Add the `AssistantOverlay` component to your app. This component displays the assistant
// overlay in a modal, bound to a shortcut key:

/** modal overlay for Elastic Assistant conversations */
export { AssistantOverlay } from './impl/assistant/assistant_overlay';

// In addition to the `AssistantOverlay`, or as an alternative, you may use the `Assistant` component
// to display the assistant without the modal overlay:

/** this component renders the Assistant without the modal overlay to, for example, render it in a Timeline tab */
export { Assistant } from './impl/assistant';
```

#### Step 3: Wherever you want to bring context into the assistant, use the any combination of the following

```ts
// Step 3: Wherever you want to bring context into the assistant, use the any combination of the following
// components and hooks:
// - `NewChat` component
// - `NewChatById` component
// - `useAssistantOverlay` hook

/**
 * `NewChat` displays a _New chat_ icon button, providing all the context
 * necessary to start a new chat. You may optionally style the button icon,
 * or override the default _New chat_ text with custom content, like `🪄✨`
 *
 * USE THIS WHEN: All the data necessary to start a new chat is available
 * in the same part of the React tree as the _New chat_ button.
 */
export { NewChat } from './impl/new_chat';

/**
 * `NewChatByID` displays a _New chat_ icon button by providing only the `promptContextId`
 * of a context that was (already) registered by the `useAssistantOverlay` hook. You may
 * optionally style the button icon, or override the default _New chat_ text with custom
 * content, like {'🪄✨'}
 *
 * USE THIS WHEN: all the data necessary to start a new chat is NOT available
 * in the same part of the React tree as the _New chat_ button. When paired
 * with the `useAssistantOverlay` hook, this option enables context to be be
 * registered where the data is available, and then the _New chat_ button can be displayed
 * in another part of the tree.
 */
export { NewChatById } from './impl/new_chat_by_id';

/**
 * `useAssistantOverlay` is a hook that registers context with the assistant overlay, and
 * returns an optional `showAssistantOverlay` function to display the assistant overlay.
 * As an alterative to using the `showAssistantOverlay` returned from this hook, you may
 * use the `NewChatById` component and pass it the `promptContextId` returned by this hook.
 *
 * USE THIS WHEN: You want to register context in one part of the tree, and then show
 * a _New chat_ button in another part of the tree without passing around the data, or when
 * you want to build a custom `New chat` button with features not not provided by the
 * `NewChat` component.
 */
export { useAssistantOverlay } from './impl/assistant/use_assistant_overlay';
```

Co-authored-by: Garrett Spong <[email protected]>
Co-authored-by: Andrew Macri <[email protected]>
(cherry picked from commit 4e38817)

# Conflicts:
#	.github/CODEOWNERS
#	x-pack/plugins/security_solution/public/common/components/event_details/translations.ts
#	x-pack/plugins/security_solution/public/timelines/components/timeline/tabs_content/index.tsx
@spong spong added the backport label Jun 2, 2023
@spong spong enabled auto-merge (squash) June 2, 2023 21:43
@spong spong disabled auto-merge June 2, 2023 21:59
@spong spong enabled auto-merge (squash) June 2, 2023 23:03
@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
securitySolution 3862 4053 +191

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/elastic-assistant - 55 +55

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id before after diff
@kbn/elastic-assistant - 4 +4

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 9.1MB 10.5MB ⚠️ +1.3MB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
@kbn/elastic-assistant - 3 +3
stackConnectors 0 1 +1
total +4

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
securitySolution 60.6KB 61.1KB +480.0B
Unknown metric groups

API count

id before after diff
@kbn/elastic-assistant - 75 +75

ESLint disabled line counts

id before after diff
@kbn/elastic-assistant - 15 +15
enterpriseSearch 17 19 +2
securitySolution 400 404 +4
total +21

Total ESLint disabled count

id before after diff
@kbn/elastic-assistant - 15 +15
enterpriseSearch 18 20 +2
securitySolution 480 484 +4
total +21

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@spong spong merged commit 0773c44 into elastic:8.8 Jun 2, 2023
@spong spong deleted the backport/8.8/pr-156933 branch June 3, 2023 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants