[Security Solution] add defend insights elastic assistant tool #198676

joeypoon · 2024-11-01T13:43:14Z

Summary

Adds the new Defend Insights Elastic Assistant tool. This assistant tool provides Elastic Defend configuration insights. For this initial PR, only incompatible antivirus detection is supported. Telemetry is collected for success and error events.

For incompatible antivirus detection, Defend Insights will review the last 200 file events for the given endpoint and output suspected antiviruses. Improvements such as customizable event count and date range will come in the future.

This PR does not include any UI, that will come in a separate PR. 3 internal APIs for interacting with Defend Insights are provided here:

POST /defend_insights for creating a new Defend Insight
GET /defend_insights/{id} for getting a Defend Insight
GET /defend_insights for getting multiple Defend Insights
- available optional query params:
  - size - default 10
  - ids
  - connector_id
  - type - incompatible_antivirus
  - status - running, completed, failed, canceled
  - endpoint_ids

This initial implementation does not include the LangGraph/output chunking upgrades seen in Attack Discovery due to time constraints. We'll look to make this upgrade in a future PR.

Checklist

Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

elasticmachine · 2024-11-04T15:41:48Z

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

paul-tavares

Left some feedback that I would I would like to hear back on. Note that I focused only on files under security solution.

Also - why is there no feature flag for this code? With out one it just be available in serverless with the next release after it is merged to main - is that ok?

x-pack/plugins/security_solution/common/endpoint/constants.ts

x-pack/plugins/security_solution/server/assistant/tools/defend_insights/errors.ts

...security_solution/server/assistant/tools/defend_insights/get_events/get_file_events_query.ts

x-pack/plugins/security_solution/server/assistant/tools/defend_insights/get_events/index.ts

x-pack/plugins/security_solution/server/assistant/tools/defend_insights/index.ts

x-pack/plugins/security_solution/server/assistant/tools/defend_insights/errors.ts

...s/security_solution/server/assistant/tools/defend_insights/prompts/incompatible_antivirus.ts

x-pack/plugins/security_solution/server/assistant/tools/defend_insights/index.ts

joeypoon · 2024-11-12T13:38:35Z

Left some feedback that I would I would like to hear back on. Note that I focused only on files under security solution.

Thanks for the review @paul-tavares. Responded directly on each one. Will update code for all of the ones I reacted with 👍.

Also - why is there no feature flag for this code? With out one it just be available in serverless with the next release after it is merged to main - is that ok?

That's an oversight by me. It's technically an internal API but we should still add a feature flag. Will update.

x-pack/plugins/security_solution/server/assistant/tools/defend_insights/index.ts

.../packages/kbn-elastic-assistant-common/impl/schemas/defend_insights/common_attributes.gen.ts

x-pack/plugins/security_solution/server/assistant/tools/index.ts

andrew-goldstein

Thanks @joeypoon!
LGTM 🚀

tomsonpl · 2024-11-18T15:09:02Z

Hey @joeypoon 👋 so exciting to see the cool stuff here!

However, given this PR has +++4.5k lines, it's very difficult to actually get a solid understanding and therefore do a review.
Do you think it's possible to merge this functionality in smaller chunks?
edit:
Or maybe not this particular PR since it's been open for a while (sorry for a delay in reviewing), but at least for the future ones?

joeypoon · 2024-11-18T15:20:27Z

Hey @joeypoon 👋 so exciting to see the cool stuff here!

However, given this PR has +++4.5k lines, it's very difficult to actually get a solid understanding and therefore do a review. Do you think it's possible to merge this functionality in smaller chunks? edit: Or maybe not this particular PR since it's been open for a while (sorry for a delay in reviewing), but at least for the future ones?

I agree with you in concept but unfortunately this one is really hard to split up. In fact, this is already split up and there will be quite a few more PRs following this 😅. Happy to walk through it with you if it helps but breaking this one up any smaller was unfortunately not feasible.

paul-tavares · 2024-11-18T17:09:57Z

x-pack/plugins/security_solution/common/experimental_features.ts

+  /**
+   * Enables the Defend Insights feature
+   */
+  defendInsights: false,


Thanks for adding this. I searched around and I don't actually see it being used... Where is it checked?

Each of the 3 API handlers check for it like this. We don't actually use it in security_solution plugin yet but will for following PRs.

@joeypoon - Thats strange (to me). So somehow this elastic_assistant plugin has access to our internal features? The code you pointed to is a bit hard to follow. Do you know how where it is actually is checking the feature flag is enabled/disabled?

We're checking for it at the beginning of all 3 new API handlers (in elastic_assistant) and that definition for those is actually here. The definition in x-pack/plugins/security_solution/common/experimental_features.ts isn't actually being used right now and is just prep for next PRs to have less conflicts.

yeah. ok. I'm just a bit confused how the FF from security solution will be used in another plugin.

yeah. ok. I'm just a bit confused how the FF from security solution will be used in another plugin.

It's not, it will be used within the security solution plugin. It's only introduced here since it's the first PR being merged for the overall feature.

paul-tavares

Thanks for the changes. I left a question regarding the feature flag, but am approving and you can just answer it when you can

* Adds a new elastic AI assistant tool, Defend Insights * this tool provides Elastic Defend configuration insights * initially it will only support incompatible antivirus detection * additional Defend Insight types can be added by simply registering a type, prompt, and context query * this is to be internally used to provide actionable Endpoint Insights such as suggesting Trusted App entries for detected incompatible antiviruses

tomsonpl

LGTM 👍 Left a few questions, but just out of curiosity. Thanks!

tomsonpl · 2024-11-18T15:17:47Z

x-pack/plugins/elastic_assistant/server/ai_assistant_data_clients/defend_insights/helpers.ts

+  response: estypes.SearchResponse<EsDefendInsightSchema>
+): DefendInsightsResponse[] => {
+  return response.hits.hits
+    .filter((hit) => hit._source !== undefined)


Just a thought:
Since _source: true is included in the request, wouldn’t every document already have a _source field?
I’m wondering if this filter is still necessary in that case.

tomsonpl · 2024-11-18T15:19:39Z

x-pack/plugins/elastic_assistant/server/ai_assistant_data_clients/defend_insights/helpers.ts

+    .filter((hit) => hit._source !== undefined)
+    .map((hit) => {
+      // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
+      const insightSchema = hit._source!;


Ah, now I see - you’re ensuring _source is always present. Just to clarify, does it actually go missing sometimes, or is this primarily to satisfy TypeScript?

tomsonpl · 2024-11-18T15:20:43Z

x-pack/plugins/elastic_assistant/server/ai_assistant_data_clients/defend_insights/helpers.ts

+      const defendInsight: DefendInsightsResponse = {
+        timestamp: insightSchema['@timestamp'],
+        // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
+        id: hit._id!,


I’m not sure if using the bang (!) operator in these places is the best approach. Wouldn’t it be clearer and safer to define the response type more explicitly instead?

SearchResponse is defined at ES level and both _id and _source are defined as optional. To your point though, we can do a null check instead.

tomsonpl · 2024-11-18T15:23:13Z

x-pack/plugins/elastic_assistant/server/ai_assistant_data_clients/defend_insights/helpers.ts

+        createdAt: insightSchema.created_at,
+        updatedAt: insightSchema.updated_at,
+        lastViewedAt: insightSchema.last_viewed_at,
+        users:


Why not just:

users: insightSchema.users ?? [];

?

Honestly, I copied this over from attack discovery and just changed the necessary keys 😅. My impression though is that we want to ensure we don't potentially leak additional fields? and maybe also just because new objects.

kibanamachine · 2024-11-19T10:08:01Z

Starting backport for target branches: 7.17, 8.15, 8.16, 8.18, 8.x

https://github.com/elastic/kibana/actions/runs/11910742167

elasticmachine · 2024-11-19T10:08:37Z

💚 Build Succeeded

Buildkite Build
Commit: 0059617

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`integrationAssistant`	466	471	+5
`securitySolution`	6213	6218	+5
total			+10

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/elastic-assistant-common`	369	405	+36

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	13.4MB	13.4MB	+97.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`securitySolution`	86.1KB	86.1KB	+18.0B

Unknown metric groups

API count

id	before	after	diff
`@kbn/elastic-assistant-common`	400	442	+42

ESLint disabled in files

id	before	after	diff
`elasticAssistant`	0	5	+5

ESLint disabled line counts

id	before	after	diff
`elasticAssistant`	32	34	+2

Total ESLint disabled count

id	before	after	diff
`elasticAssistant`	32	39	+7

History

💚 Build #251855 succeeded eb9ab83
💔 Build #251785 failed 6ba79a6
💚 Build #250868 succeeded 86812fa
💔 Build #250736 failed 9e0a42f
💛 Build #248060 was flaky e67cdf4
💔 Build #247986 failed 68d1c85

kibanamachine · 2024-11-19T10:13:55Z

💔 All backports failed

Status	Branch	Result
❌	7.17	Backport failed because of merge conflicts
❌	8.15	Backport failed because of merge conflicts
❌	8.16	Backport failed because of merge conflicts
❌	8.18	The branch "8.18" does not exist
❌	8.x	Backport failed because of merge conflicts You might need to backport the following PRs to 8.x: - [Cloud Security] Fixed an issue with Host.name Alerts contextual flyout (#200626) - Surface Kibana security route deprecations in Upgrade Assistant (#199656) - [Infra] Use callback for logger.trace calls (#199805) - [Security Solution][Bidirectional Integrations Banner][Crowdstrike][SentinelOne] Banner for bidirectional integrations (#200625) - [Fleet] Use metering API in serverless (#200063) - Migrate `/test/apm_api_integration/tests/suggestions` to be deployment agnostic api tests (#200556) - [Inventory] Remove open in Discover button (#200574) - [Entity Analytics] API changes for right placement of deleting the old component template (#199734)

Manual backport

To create the backport manually run:

node scripts/backport --pr 198676

Questions ?

Please refer to the Backport tool documentation

joeypoon · 2024-11-21T10:26:13Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…ic#198676) ### Summary Adds the new Defend Insights Elastic Assistant tool. This assistant tool provides Elastic Defend configuration insights. For this initial PR, only incompatible antivirus detection is supported. Telemetry is collected for success and error events. For incompatible antivirus detection, Defend Insights will review the last 200 file events for the given endpoint and output suspected antiviruses. Improvements such as customizable event count and date range will come in the future. This PR does not include any UI, that will come in a separate PR. 3 internal APIs for interacting with Defend Insights are provided here: - `POST /defend_insights` for creating a new Defend Insight - `GET /defend_insights/{id}` for getting a Defend Insight - `GET /defend_insights` for getting multiple Defend Insights - available optional query params: - `size` - default 10 - `ids` - `connector_id` - `type` - `incompatible_antivirus` - `status` - `running`, `completed`, `failed`, `canceled` - `endpoint_ids` This initial implementation does not include the LangGraph/output chunking upgrades seen in Attack Discovery due to time constraints. We'll look to make this upgrade in a future PR. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels) (cherry picked from commit efc0568) # Conflicts: # .github/CODEOWNERS

kibanamachine · 2024-11-21T10:48:41Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

…198676) (#201104) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] add defend insights elastic assistant tool (#198676)](#198676)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)

…ic#198676) ### Summary Adds the new Defend Insights Elastic Assistant tool. This assistant tool provides Elastic Defend configuration insights. For this initial PR, only incompatible antivirus detection is supported. Telemetry is collected for success and error events. For incompatible antivirus detection, Defend Insights will review the last 200 file events for the given endpoint and output suspected antiviruses. Improvements such as customizable event count and date range will come in the future. This PR does not include any UI, that will come in a separate PR. 3 internal APIs for interacting with Defend Insights are provided here: - `POST /defend_insights` for creating a new Defend Insight - `GET /defend_insights/{id}` for getting a Defend Insight - `GET /defend_insights` for getting multiple Defend Insights - available optional query params: - `size` - default 10 - `ids` - `connector_id` - `type` - `incompatible_antivirus` - `status` - `running`, `completed`, `failed`, `canceled` - `endpoint_ids` This initial implementation does not include the LangGraph/output chunking upgrades seen in Attack Discovery due to time constraints. We'll look to make this upgrade in a future PR. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)

joeypoon added release_note:skip Skip the PR/issue when compiling release notes Team:Defend Workflows “EDR Workflows” sub-team of Security Solution labels Nov 1, 2024

joeypoon changed the title ~~[Security Solution] add defend insights ai assistant tool~~ [Security Solution] add defend insights elastic assistant tool Nov 1, 2024

joeypoon force-pushed the feature/endpoint-ai-insights branch 3 times, most recently from 1b32df0 to e29f793 Compare November 4, 2024 10:18

joeypoon marked this pull request as ready for review November 4, 2024 15:41

joeypoon requested review from a team as code owners November 4, 2024 15:41

joeypoon requested review from tomsonpl and paul-tavares November 4, 2024 15:41

joeypoon added the v8.18.0 label Nov 4, 2024

joeypoon requested a review from szwarckonrad November 5, 2024 15:35

paul-tavares requested changes Nov 11, 2024

View reviewed changes

szwarckonrad reviewed Nov 12, 2024

View reviewed changes

x-pack/plugins/security_solution/server/assistant/tools/defend_insights/index.ts Show resolved Hide resolved

andrew-goldstein reviewed Nov 12, 2024

View reviewed changes

.../packages/kbn-elastic-assistant-common/impl/schemas/defend_insights/common_attributes.gen.ts Show resolved Hide resolved

andrew-goldstein reviewed Nov 12, 2024

View reviewed changes

x-pack/plugins/security_solution/server/assistant/tools/index.ts Outdated Show resolved Hide resolved

andrew-goldstein approved these changes Nov 12, 2024

View reviewed changes

joeypoon added the backport:all-open Backport to all branches that could still receive a release label Nov 13, 2024

joeypoon force-pushed the feature/endpoint-ai-insights branch 2 times, most recently from 105700e to c789fa3 Compare November 13, 2024 10:13

joeypoon requested a review from paul-tavares November 13, 2024 10:16

joeypoon force-pushed the feature/endpoint-ai-insights branch 2 times, most recently from e3bceca to 86812fa Compare November 13, 2024 13:59

joeypoon added the v9.0.0 label Nov 18, 2024

paul-tavares reviewed Nov 18, 2024

View reviewed changes

paul-tavares approved these changes Nov 18, 2024

View reviewed changes

joeypoon force-pushed the feature/endpoint-ai-insights branch from eb9ab83 to 0059617 Compare November 19, 2024 08:21

joeypoon enabled auto-merge (squash) November 19, 2024 08:21

tomsonpl approved these changes Nov 19, 2024

View reviewed changes

joeypoon merged commit efc0568 into elastic:main Nov 19, 2024
44 checks passed

joeypoon mentioned this pull request Nov 21, 2024

[8.x] [Security Solution] add defend insights elastic assistant tool (#198676) #201104

Merged

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Nov 21, 2024

kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Nov 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution] add defend insights elastic assistant tool #198676

[Security Solution] add defend insights elastic assistant tool #198676

joeypoon commented Nov 1, 2024 •

edited by kibanamachine

Loading

elasticmachine commented Nov 4, 2024

paul-tavares left a comment

joeypoon commented Nov 12, 2024

andrew-goldstein left a comment

tomsonpl commented Nov 18, 2024 •

edited

Loading

joeypoon commented Nov 18, 2024

paul-tavares Nov 18, 2024

joeypoon Nov 19, 2024

paul-tavares Nov 19, 2024

joeypoon Nov 19, 2024

paul-tavares Nov 19, 2024

joeypoon Nov 19, 2024

paul-tavares left a comment

tomsonpl left a comment

tomsonpl Nov 18, 2024

tomsonpl Nov 18, 2024

tomsonpl Nov 18, 2024

joeypoon Nov 19, 2024

tomsonpl Nov 18, 2024

joeypoon Nov 19, 2024

kibanamachine commented Nov 19, 2024

elasticmachine commented Nov 19, 2024

API count

ESLint disabled in files

ESLint disabled line counts

Total ESLint disabled count

kibanamachine commented Nov 19, 2024

joeypoon commented Nov 21, 2024

kibanamachine commented Nov 21, 2024

[Security Solution] add defend insights elastic assistant tool #198676

[Security Solution] add defend insights elastic assistant tool #198676

Conversation

joeypoon commented Nov 1, 2024 • edited by kibanamachine Loading

Summary

Checklist

For maintainers

elasticmachine commented Nov 4, 2024

paul-tavares left a comment

Choose a reason for hiding this comment

joeypoon commented Nov 12, 2024

andrew-goldstein left a comment

Choose a reason for hiding this comment

tomsonpl commented Nov 18, 2024 • edited Loading

joeypoon commented Nov 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paul-tavares left a comment

Choose a reason for hiding this comment

tomsonpl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibanamachine commented Nov 19, 2024

elasticmachine commented Nov 19, 2024

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Page load bundle

API count

ESLint disabled in files

ESLint disabled line counts

Total ESLint disabled count

History

kibanamachine commented Nov 19, 2024

💔 All backports failed

Manual backport

Questions ?

joeypoon commented Nov 21, 2024

💚 All backports created successfully

Questions ?

kibanamachine commented Nov 21, 2024

joeypoon commented Nov 1, 2024 •

edited by kibanamachine

Loading

tomsonpl commented Nov 18, 2024 •

edited

Loading