Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Investigation guide - insights in markdown #145240

Merged

Conversation

kqualters-elastic
Copy link
Contributor

@kqualters-elastic kqualters-elastic commented Nov 15, 2022

Summary

This pr adds a new parsing plugin to the EuiMarkdownEditor used in security solution that enables users to create run time queries that can be parameterized from alert data, or hard coded literal values. A count of the matching events is displayed in a button that when clicked will open the same event set in timeline. Markdown is expected to be in the following format:

!{insight{"description":"2 top level OR providers, 1 nested AND","label":"test insight", "providers": [[{ "field": "event.id", "value": "kibana.alert.original_event.id", "type": "parameter" }], [{ "field": "event.category", "value": "network", "type": "literal" }, {"field": "process.pid", "value": "process.pid", "type": "parameter"}]]}}

The 2d array is used to allow nested queries, the top level arrays are OR'ed together, and the inner array AND'ed together:
image

Following a prefix of !insight, the configuration object takes optional description and label strings, along with a 2 dimensional array called "providers". This value corresponds to what are called data providers in the timeline view,
image

and are arrays of filters with 3 fields, "field" which is the field name for that part of the query clause, "value" which is the value to be used, and "type" which is either "parameter" or "literal". Filters of type parameter expect value to be the name of a field present in an alert document, and will use the value in the underlying document if found. If the field is not present for some reason, a wildcard is used. If the markdown is rendered in a context not tied to a specific alert, parameter fields are treated as a timeline template field.
image

Checklist

Delete any items that are not applicable to this PR.

@kqualters-elastic kqualters-elastic added release_note:feature Makes this part of the condensed release notes Team:Threat Hunting:Investigations Security Solution Investigations Team labels Nov 15, 2022
timeRange: {
kind: 'absolute',
from: timeRange,
to: new Date().toISOString(),
Copy link
Contributor

@michaelolo24 michaelolo24 Nov 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it always to the current time? and should timerange be from instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ya agree this was weirdly done by me, forgot to go back and change it. updated now so that timerange is passed as a prop or not at all. For the now, I think oldest timestamp -> now is as inclusive as we can be, since a value has to be supplied, not sure what else we could use.

const { addError } = useAppToasts();
let parsedProviders = [];
try {
if (providers !== undefined) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

providers can never be null?

Copy link
Contributor Author

@kqualters-elastic kqualters-elastic Nov 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't be, but if they are, we just pass an empty array for data providers, and the button will render but show 0

@kqualters-elastic kqualters-elastic changed the title [WIP][Security Solution] Investigation guide - insights in markdown [Security Solution] Investigation guide - insights in markdown Nov 15, 2022
@kqualters-elastic kqualters-elastic marked this pull request as ready for review November 15, 2022 20:18
@kqualters-elastic kqualters-elastic requested review from a team as code owners November 15, 2022 20:18
@@ -360,17 +360,20 @@ export const useTimelineEventsHandler = ({
...deStructureEqlOptions(prevEqlRequest),
};

const requestTimeRange =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

const timerange = startDate && endDate ? { timerange: {....} } : {};

searchParamStuff = {
  ...timerange
}

dataViewId,
fields: ['*'],
filterQuery: combinedQueries?.filterQuery,
id: 'timeline-1',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use TimelineId.active?

@@ -0,0 +1,132 @@
/*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding these tests!

export { InsightComponent as renderer };

const InsightEditorComponent = ({
node,
Copy link
Contributor

@michaelolo24 michaelolo24 Nov 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These props aren't used

Copy link
Contributor

@michaelolo24 michaelolo24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just had a few nits. Only thing is that it's not enabled for the cases markdown plugin, but we can probably add that to the implementation plan for 8.7?

@kqualters-elastic kqualters-elastic enabled auto-merge (squash) November 15, 2022 22:44
@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
securitySolution 3302 3305 +3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 9.7MB 9.7MB +11.8KB
Unknown metric groups

ESLint disabled in files

id before after diff
osquery 1 2 +1

ESLint disabled line counts

id before after diff
enterpriseSearch 19 21 +2
fleet 59 65 +6
osquery 108 113 +5
securitySolution 441 447 +6
total +19

Total ESLint disabled count

id before after diff
enterpriseSearch 20 22 +2
fleet 67 73 +6
osquery 109 115 +6
securitySolution 518 524 +6
total +20

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@kqualters-elastic kqualters-elastic merged commit 072c70d into elastic:main Nov 16, 2022
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Nov 16, 2022
kqualters-elastic added a commit that referenced this pull request Feb 7, 2023
…150363)

## Summary

This pr expands upon the work done in
#145240 to make use of the filters
builder form from unified_search to serialize filters into a markdown
compatible string, so that investigation guides, timeline notes or any
other place where text is parsed as markdown can make use of standard
kibana filters and view a count of the matching documents at a glance,
and open the entire set in timeline as well. These are generally
converted to timeline data providers to enable drag and drop query
building, however this is not supported for filters of range type, so
regular kibana filters are used in that case for now.

![Screenshot 2023-02-06 at 3 46 15
PM](https://user-images.githubusercontent.com/56408403/217081398-7e0d263f-cdb5-48eb-9328-f01a63af768e.png)
![Screenshot 2023-02-06 at 3 49 46
PM](https://user-images.githubusercontent.com/56408403/217082554-389edad5-89ff-4d86-bd31-c2085073b39a.png)
![Screenshot 2023-02-06 at 3 50 15
PM](https://user-images.githubusercontent.com/56408403/217082658-7ef8af2b-ba7f-4676-a775-e8c550adeee6.png)
![Screenshot 2023-02-06 at 3 50 54
PM](https://user-images.githubusercontent.com/56408403/217082770-9bacbd2a-fbee-4d1f-b6f5-b7d97ed2e3ca.png)
![Screenshot 2023-02-06 at 3 51 16
PM](https://user-images.githubusercontent.com/56408403/217082842-7494b1ac-6687-426e-8e85-6fec0afcc70e.png)
![Screenshot 2023-02-06 at 3 53 48
PM](https://user-images.githubusercontent.com/56408403/217083273-f9acfa30-a156-4146-86a2-5ebb84f4ecd0.png)
![Screenshot 2023-02-06 at 3 54 30
PM](https://user-images.githubusercontent.com/56408403/217083407-1a8af419-6c09-4558-9c18-11604cb7e796.png)




### Checklist

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting release_note:feature Makes this part of the condensed release notes Team:Threat Hunting:Investigations Security Solution Investigations Team v8.6.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants