Skip to content

Commit

Permalink
[Security Solution] add defend insights elastic assistant tool (#198676)
Browse files Browse the repository at this point in the history
### Summary
Adds the new Defend Insights Elastic Assistant tool. This assistant tool
provides Elastic Defend configuration insights. For this initial PR,
only incompatible antivirus detection is supported. Telemetry is
collected for success and error events.

For incompatible antivirus detection, Defend Insights will review the
last 200 file events for the given endpoint and output suspected
antiviruses. Improvements such as customizable event count and date
range will come in the future.

This PR does not include any UI, that will come in a separate PR. 3
internal APIs for interacting with Defend Insights are provided here:
- `POST /defend_insights` for creating a new Defend Insight
- `GET /defend_insights/{id}` for getting a Defend Insight
- `GET /defend_insights` for getting multiple Defend Insights
	- available optional query params:
		- `size` - default 10
		- `ids`
		- `connector_id`
		- `type` - `incompatible_antivirus`
		- `status` - `running`, `completed`, `failed`, `canceled`
		- `endpoint_ids`

This initial implementation does not include the LangGraph/output
chunking upgrades seen in Attack Discovery due to time constraints.
We'll look to make this upgrade in a future PR.

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios


### For maintainers

- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)
  • Loading branch information
joeypoon authored Nov 19, 2024
1 parent 4f96b85 commit efc0568
Show file tree
Hide file tree
Showing 59 changed files with 4,574 additions and 10 deletions.
5 changes: 5 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -2158,6 +2158,7 @@ x-pack/test/security_solution_cypress/cypress/tasks/expandable_flyout @elastic/
/x-pack/plugins/security_solution/public/flyout/document_details/isolate_host/ @elastic/security-defend-workflows
/x-pack/plugins/security_solution/common/endpoint/ @elastic/security-defend-workflows
/x-pack/plugins/security_solution/common/api/endpoint/ @elastic/security-defend-workflows
x-pack/plugins/security_solution/server/assistant/tools/defend_insights @elastic/security-defend-workflows
/x-pack/plugins/security_solution/server/endpoint/ @elastic/security-defend-workflows
/x-pack/plugins/security_solution/server/lists_integration/endpoint/ @elastic/security-defend-workflows
/x-pack/plugins/security_solution/server/lib/license/ @elastic/security-defend-workflows
Expand All @@ -2169,6 +2170,10 @@ x-pack/test/security_solution_cypress/cypress/tasks/expandable_flyout @elastic/
/x-pack/plugins/security_solution_serverless/public/upselling/sections/endpoint_management @elastic/security-defend-workflows
/x-pack/plugins/security_solution_serverless/public/upselling/pages/endpoint_management @elastic/security-defend-workflows
/x-pack/plugins/security_solution_serverless/server/endpoint @elastic/security-defend-workflows
x-pack/packages/kbn-elastic-assistant-common/impl/schemas/defend_insights @elastic/security-defend-workflows
x-pack/plugins/elastic_assistant/server/__mocks__/defend_insights_schema.mock.ts @elastic/security-defend-workflows
x-pack/plugins/elastic_assistant/server/ai_assistant_data_clients/defend_insights @elastic/security-defend-workflows
x-pack/plugins/elastic_assistant/server/routes/defend_insights @elastic/security-defend-workflows

## Security Solution sub teams - security-telemetry (Data Engineering)
x-pack/plugins/security_solution/server/usage/ @elastic/security-data-analytics
Expand Down
5 changes: 5 additions & 0 deletions x-pack/packages/kbn-elastic-assistant-common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,8 @@ export const ELASTIC_AI_ASSISTANT_KNOWLEDGE_BASE_INDICES_URL =

export const ELASTIC_AI_ASSISTANT_EVALUATE_URL =
`${ELASTIC_AI_ASSISTANT_INTERNAL_URL}/evaluate` as const;

// Defend insights
export const DEFEND_INSIGHTS_TOOL_ID = 'defend-insights';
export const DEFEND_INSIGHTS = `${ELASTIC_AI_ASSISTANT_INTERNAL_URL}/defend_insights`;
export const DEFEND_INSIGHTS_BY_ID = `${DEFEND_INSIGHTS}/{id}`;
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,5 @@ export type AssistantFeatureKey = keyof AssistantFeatures;
*/
export const defaultAssistantFeatures = Object.freeze({
assistantModelEvaluation: false,
defendInsights: false,
});
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,5 @@ import { z } from '@kbn/zod';
export type GetCapabilitiesResponse = z.infer<typeof GetCapabilitiesResponse>;
export const GetCapabilitiesResponse = z.object({
assistantModelEvaluation: z.boolean(),
defendInsights: z.boolean(),
});
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,11 @@ paths:
properties:
assistantModelEvaluation:
type: boolean
defendInsights:
type: boolean
required:
- assistantModelEvaluation
- defendInsights
'400':
description: Generic Error
content:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

/*
* NOTICE: Do not edit this file manually.
* This file is automatically generated by the OpenAPI Generator, @kbn/openapi-generator.
*
* info:
* title: Common Defend Insights Attributes
* version: not applicable
*/

import { z } from '@kbn/zod';

import { NonEmptyString, User } from '../common_attributes.gen';
import { Replacements, ApiConfig } from '../conversations/common_attributes.gen';

/**
* A Defend insight event
*/
export type DefendInsightEvent = z.infer<typeof DefendInsightEvent>;
export const DefendInsightEvent = z.object({
/**
* The event's ID
*/
id: z.string(),
/**
* The endpoint's ID
*/
endpointId: z.string(),
/**
* The value of the event
*/
value: z.string(),
});

/**
* The insight type (ie. incompatible_antivirus)
*/
export type DefendInsightType = z.infer<typeof DefendInsightType>;
export const DefendInsightType = z.enum(['incompatible_antivirus', 'noisy_process_tree']);
export type DefendInsightTypeEnum = typeof DefendInsightType.enum;
export const DefendInsightTypeEnum = DefendInsightType.enum;

/**
* A Defend insight generated from endpoint events
*/
export type DefendInsight = z.infer<typeof DefendInsight>;
export const DefendInsight = z.object({
/**
* The group category of the events (ie. Windows Defender)
*/
group: z.string(),
/**
* An array of event objects
*/
events: z.array(DefendInsightEvent).optional(),
});

/**
* Array of Defend insights
*/
export type DefendInsights = z.infer<typeof DefendInsights>;
export const DefendInsights = z.array(DefendInsight);

/**
* The status of the Defend insight.
*/
export type DefendInsightStatus = z.infer<typeof DefendInsightStatus>;
export const DefendInsightStatus = z.enum(['running', 'succeeded', 'failed', 'canceled']);
export type DefendInsightStatusEnum = typeof DefendInsightStatus.enum;
export const DefendInsightStatusEnum = DefendInsightStatus.enum;

/**
* Run durations for the Defend insight
*/
export type DefendInsightGenerationInterval = z.infer<typeof DefendInsightGenerationInterval>;
export const DefendInsightGenerationInterval = z.object({
/**
* The time the Defend insight was generated
*/
date: z.string(),
/**
* The duration of the Defend insight generation
*/
durationMs: z.number().int(),
});

export type DefendInsightsResponse = z.infer<typeof DefendInsightsResponse>;
export const DefendInsightsResponse = z.object({
id: NonEmptyString,
timestamp: NonEmptyString.optional(),
/**
* The last time the Defend insight was updated.
*/
updatedAt: z.string(),
/**
* The last time the Defend insight was viewed in the browser.
*/
lastViewedAt: z.string(),
/**
* The number of events in the context.
*/
eventsContextCount: z.number().int().optional(),
/**
* The time the Defend insight was created.
*/
createdAt: z.string(),
replacements: Replacements.optional(),
users: z.array(User),
/**
* The status of the Defend insight.
*/
status: DefendInsightStatus,
endpointIds: z.array(NonEmptyString),
insightType: DefendInsightType,
/**
* The Defend insights.
*/
insights: DefendInsights,
/**
* LLM API configuration.
*/
apiConfig: ApiConfig,
/**
* Kibana space
*/
namespace: z.string(),
/**
* The backing index required for update requests.
*/
backingIndex: z.string(),
/**
* The most 5 recent generation intervals
*/
generationIntervals: z.array(DefendInsightGenerationInterval),
/**
* The average generation interval in milliseconds
*/
averageIntervalMs: z.number().int(),
/**
* The reason for a status of failed.
*/
failureReason: z.string().optional(),
});

export type DefendInsightUpdateProps = z.infer<typeof DefendInsightUpdateProps>;
export const DefendInsightUpdateProps = z.object({
id: NonEmptyString,
/**
* LLM API configuration.
*/
apiConfig: ApiConfig.optional(),
/**
* The number of events in the context.
*/
eventsContextCount: z.number().int().optional(),
/**
* The Defend insights.
*/
insights: DefendInsights.optional(),
/**
* The status of the Defend insight.
*/
status: DefendInsightStatus.optional(),
replacements: Replacements.optional(),
/**
* The most 5 recent generation intervals
*/
generationIntervals: z.array(DefendInsightGenerationInterval).optional(),
/**
* The backing index required for update requests.
*/
backingIndex: z.string(),
/**
* The reason for a status of failed.
*/
failureReason: z.string().optional(),
/**
* The last time the Defend insight was viewed in the browser.
*/
lastViewedAt: z.string().optional(),
});

export type DefendInsightsUpdateProps = z.infer<typeof DefendInsightsUpdateProps>;
export const DefendInsightsUpdateProps = z.array(DefendInsightUpdateProps);

export type DefendInsightCreateProps = z.infer<typeof DefendInsightCreateProps>;
export const DefendInsightCreateProps = z.object({
/**
* The Defend insight id.
*/
id: z.string().optional(),
/**
* The status of the Defend insight.
*/
status: DefendInsightStatus,
/**
* The number of events in the context.
*/
eventsContextCount: z.number().int().optional(),
endpointIds: z.array(NonEmptyString),
insightType: DefendInsightType,
/**
* The Defend insights.
*/
insights: DefendInsights,
/**
* LLM API configuration.
*/
apiConfig: ApiConfig,
replacements: Replacements.optional(),
});
Loading

0 comments on commit efc0568

Please sign in to comment.