Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Investigation guide - insights in markdown #145240

Merged
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
843821a
WIP starting to work
kqualters-elastic Oct 19, 2022
a6fb6fa
WIP starting to work
kqualters-elastic Oct 19, 2022
6963564
Merge remote-tracking branch 'upstream/main' into investigation-guide…
kqualters-elastic Oct 31, 2022
dafe3ee
Merge remote-tracking branch 'upstream/main' into investigation-guide…
kqualters-elastic Nov 1, 2022
5c75894
WIP query working minus timerange
kqualters-elastic Nov 2, 2022
e3a0fa9
Merge remote-tracking branch 'upstream/main' into investigation-guide…
kqualters-elastic Nov 2, 2022
0096785
Mostly working
kqualters-elastic Nov 2, 2022
2957a12
Merge branch 'main' into investigation-guide-insights
kqualters-elastic Nov 2, 2022
95b177f
WIP
kqualters-elastic Nov 8, 2022
e85e7d1
Merge branch 'main' into investigation-guide-insights
kqualters-elastic Nov 9, 2022
8f0c80a
Merge branch 'investigation-guide-insights' of github.com:kqualters-e…
kqualters-elastic Nov 13, 2022
3d065c3
Fix types, code working
kqualters-elastic Nov 15, 2022
119e094
Merge branch 'main' into investigation-guide-insights
kqualters-elastic Nov 15, 2022
7f55bb7
Remove unused variable
kqualters-elastic Nov 15, 2022
5fd36e7
Remove unused prop
kqualters-elastic Nov 15, 2022
3a6b4f6
Add useDataProvider tests
kqualters-elastic Nov 15, 2022
9b32d75
Add locator
kqualters-elastic Nov 15, 2022
89eb612
Add sanity test for query hook
kqualters-elastic Nov 15, 2022
cf46b22
Add sanity cypress test for insight markdown
kqualters-elastic Nov 15, 2022
f716b8f
Update cypress selector, add error message for invalid field
kqualters-elastic Nov 15, 2022
7ca6049
Disable button if combine queries fails
kqualters-elastic Nov 15, 2022
7cac44c
Remove delay when using markdown editor, use better prop names
kqualters-elastic Nov 15, 2022
28f00d6
Merge remote-tracking branch 'upstream/main' into investigation-guide…
kqualters-elastic Nov 15, 2022
386ac1b
Fix types
kqualters-elastic Nov 15, 2022
3ad85e3
Fix types without breaking pagination
kqualters-elastic Nov 15, 2022
710f476
PR feedback
kqualters-elastic Nov 15, 2022
4a2b16e
Merge remote-tracking branch 'upstream/main' into investigation-guide…
kqualters-elastic Nov 15, 2022
be6cf77
Remove unused variable
kqualters-elastic Nov 15, 2022
b5fa33d
Merge remote-tracking branch 'upstream/main' into investigation-guide…
kqualters-elastic Nov 15, 2022
89bdd49
Merge remote-tracking branch 'upstream/main' into investigation-guide…
kqualters-elastic Nov 16, 2022
5558d77
Remove noop
kqualters-elastic Nov 16, 2022
b845243
Merge remote-tracking branch 'upstream/main' into investigation-guide…
kqualters-elastic Nov 16, 2022
8273e2f
Prevent infinite loop with invalid markup in provider
kqualters-elastic Nov 16, 2022
06c3c18
Merge remote-tracking branch 'upstream/main' into investigation-guide…
kqualters-elastic Nov 16, 2022
997bcf5
Merge branch 'main' into investigation-guide-insights
jamster10 Nov 16, 2022
012863c
Merge branch 'main' into investigation-guide-insights
michaelolo24 Nov 16, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import {
NOTES_LINK,
NOTES_TEXT,
NOTES_TEXT_AREA,
MARKDOWN_INVESTIGATE_BUTTON,
} from '../../screens/timeline';
import { createTimeline } from '../../tasks/api_calls/timelines';

Expand Down Expand Up @@ -84,4 +85,11 @@ describe('Timeline notes tab', () => {
cy.get(NOTES_LINK).last().should('have.text', `${text}(opens in a new tab or window)`);
cy.get(NOTES_LINK).last().click();
});

it('should render insight query from markdown', () => {
addNotesToTimeline(
`!{insight{"description":"2 top level OR providers, 1 nested AND","label":"test insight", "providers": [[{ "field": "event.id", "value": "kibana.alert.original_event.id", "type": "parameter" }], [{ "field": "event.category", "value": "network", "type": "literal" }, {"field": "process.pid", "value": "process.pid", "type": "parameter"}]]}}`
);
cy.get(MARKDOWN_INVESTIGATE_BUTTON).should('exist');
});
});
2 changes: 2 additions & 0 deletions x-pack/plugins/security_solution/cypress/screens/timeline.ts
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,8 @@ export const NOTES_AUTHOR = '.euiCommentEvent__headerUsername';

export const NOTES_LINK = '[data-test-subj="markdown-link"]';

export const MARKDOWN_INVESTIGATE_BUTTON = '[data-test-subj="investigate-in-timeline-button"]';

export const OPEN_TIMELINE_ICON = '[data-test-subj="open-timeline-button"]';

export const OPEN_TIMELINE_MODAL = '[data-test-subj="open-timeline-modal"]';
Expand Down
2 changes: 1 addition & 1 deletion x-pack/plugins/security_solution/cypress/tasks/timeline.ts
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ export const addNotesToTimeline = (notes: string) => {
.then(($el) => {
const notesCount = parseInt($el.text(), 10);

cy.get(NOTES_TEXT_AREA).type(notes);
cy.get(NOTES_TEXT_AREA).type(notes, { parseSpecialCharSequences: false });
cy.get(ADD_NOTE_BUTTON).trigger('click');
cy.get(`${NOTES_TAB_BUTTON} .euiBadge`).should('have.text', `${notesCount + 1}`);
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
* 2.0.
*/

import React, { useMemo } from 'react';
import React, { useMemo, useCallback } from 'react';
import { EuiButton, EuiButtonEmpty } from '@elastic/eui';
import type { Filter } from '@kbn/es-query';
import { useDispatch } from 'react-redux';
Expand All @@ -26,7 +26,9 @@ export const InvestigateInTimelineButton: React.FunctionComponent<{
asEmptyButton: boolean;
dataProviders: DataProvider[] | null;
filters?: Filter[] | null;
}> = ({ asEmptyButton, children, dataProviders, filters, ...rest }) => {
timeRange?: string | null;
keepDataView?: boolean;
}> = ({ asEmptyButton, children, dataProviders, filters, timeRange, keepDataView, ...rest }) => {
const dispatch = useDispatch();

const getDataViewsSelector = useMemo(
Expand All @@ -37,15 +39,28 @@ export const InvestigateInTimelineButton: React.FunctionComponent<{
getDataViewsSelector(state)
);

const hasTemplateProviders =
dataProviders && dataProviders.find((provider) => provider.type === 'template');

const clearTimeline = useCreateTimeline({
timelineId: TimelineId.active,
timelineType: TimelineType.default,
timelineType: hasTemplateProviders ? TimelineType.template : TimelineType.default,
});

const configureAndOpenTimeline = React.useCallback(() => {
const configureAndOpenTimeline = useCallback(() => {
if (dataProviders || filters) {
// Reset the current timeline
clearTimeline();
if (timeRange) {
clearTimeline({
timeRange: {
kind: 'absolute',
from: timeRange,
to: new Date().toISOString(),
Copy link
Contributor

@michaelolo24 michaelolo24 Nov 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it always to the current time? and should timerange be from instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ya agree this was weirdly done by me, forgot to go back and change it. updated now so that timerange is passed as a prop or not at all. For the now, I think oldest timestamp -> now is as inclusive as we can be, since a value has to be supplied, not sure what else we could use.

},
});
} else {
clearTimeline();
}
if (dataProviders) {
// Update the timeline's providers to match the current prevalence field query
dispatch(
Expand All @@ -66,17 +81,28 @@ export const InvestigateInTimelineButton: React.FunctionComponent<{
}
// Only show detection alerts
// (This is required so the timeline event count matches the prevalence count)
dispatch(
sourcererActions.setSelectedDataView({
id: SourcererScopeName.timeline,
selectedDataViewId: defaultDataView.id,
selectedPatterns: [signalIndexName || ''],
})
);
if (!keepDataView) {
dispatch(
sourcererActions.setSelectedDataView({
id: SourcererScopeName.timeline,
selectedDataViewId: defaultDataView.id,
selectedPatterns: [signalIndexName || ''],
})
);
}
// Unlock the time range from the global time range
dispatch(inputsActions.removeLinkTo([InputsModelId.timeline, InputsModelId.global]));
}
}, [dataProviders, clearTimeline, dispatch, defaultDataView.id, signalIndexName, filters]);
}, [
dataProviders,
clearTimeline,
dispatch,
defaultDataView.id,
signalIndexName,
filters,
timeRange,
keepDataView,
]);

return asEmptyButton ? (
<EuiButtonEmpty
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import {

import * as timelineMarkdownPlugin from './timeline';
import * as osqueryMarkdownPlugin from './osquery';
import * as insightMarkdownPlugin from './insight';

export const { uiPlugins, parsingPlugins, processingPlugins } = {
uiPlugins: getDefaultEuiMarkdownUiPlugins(),
Expand All @@ -23,9 +24,11 @@ export const { uiPlugins, parsingPlugins, processingPlugins } = {
uiPlugins.push(timelineMarkdownPlugin.plugin);
uiPlugins.push(osqueryMarkdownPlugin.plugin);

parsingPlugins.push(insightMarkdownPlugin.parser);
parsingPlugins.push(timelineMarkdownPlugin.parser);
parsingPlugins.push(osqueryMarkdownPlugin.parser);

// This line of code is TS-compatible and it will break if [1][1] change in the future.
processingPlugins[1][1].components.insight = insightMarkdownPlugin.renderer;
processingPlugins[1][1].components.timeline = timelineMarkdownPlugin.renderer;
processingPlugins[1][1].components.osquery = osqueryMarkdownPlugin.renderer;
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import type { Plugin } from 'unified';
import React, { useContext } from 'react';
import type { RemarkTokenizer } from '@elastic/eui';
import { EuiSpacer, EuiCodeBlock, EuiLoadingSpinner, EuiIcon } from '@elastic/eui';
import type { EuiMarkdownEditorUiPluginEditorProps } from '@elastic/eui/src/components/markdown_editor/markdown_types';
import { i18n } from '@kbn/i18n';
import { useAppToasts } from '../../../../hooks/use_app_toasts';
import { useInsightQuery } from './use_insight_query';
import { useInsightDataProviders } from './use_insight_data_providers';
import { BasicAlertDataContext } from '../../../event_details/investigation_guide_view';
import { InvestigateInTimelineButton } from '../../../event_details/table/investigate_in_timeline_button';

interface InsightComponentProps {
label?: string;
description?: string;
providers?: string;
}

export const parser: Plugin = function () {
const Parser = this.Parser;
const tokenizers = Parser.prototype.inlineTokenizers;
const methods = Parser.prototype.inlineMethods;
const insightPrefix = '!{insight';

const tokenizeInsight: RemarkTokenizer = function (eat, value, silent) {
if (value.startsWith(insightPrefix) === false) {
return false;
}

const nextChar = value[insightPrefix.length];
if (nextChar !== '{' && nextChar !== '}') return false;
if (silent) {
return true;
}

// is there a configuration?
const hasConfiguration = nextChar === '{';

let configuration: InsightComponentProps = {};
if (hasConfiguration) {
let configurationString = '';
let openObjects = 0;

for (let i = insightPrefix.length; i < value.length; i++) {
const char = value[i];
if (char === '{') {
openObjects++;
configurationString += char;
} else if (char === '}') {
openObjects--;
if (openObjects === -1) {
break;
}
configurationString += char;
} else {
configurationString += char;
}
}

try {
configuration = JSON.parse(configurationString);
return eat(value)({
type: 'insight',
...configuration,
providers: JSON.stringify(configuration.providers),
});
} catch (err) {
const now = eat.now();
this.file.fail(
i18n.translate('xpack.securitySolution.markdownEditor.plugins.insightConfigError', {
values: { err },
defaultMessage: 'Unable to parse insight JSON configuration: {err}',
}),
{
line: now.line,
column: now.column + insightPrefix.length,
}
);
}
}
return false;
};
tokenizeInsight.locator = (value: string, fromIndex: number) => {
return value.indexOf(insightPrefix, fromIndex);
};
tokenizers.insight = tokenizeInsight;
methods.splice(methods.indexOf('text'), 0, 'insight');
};

// receives the configuration from the parser and renders
const InsightComponent = ({ label, description, providers }: InsightComponentProps) => {
const { addError } = useAppToasts();
let parsedProviders = [];
try {
if (providers !== undefined) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

providers can never be null?

Copy link
Contributor Author

@kqualters-elastic kqualters-elastic Nov 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't be, but if they are, we just pass an empty array for data providers, and the button will render but show 0

parsedProviders = JSON.parse(providers);
}
} catch (err) {
addError(err, {
title: i18n.translate('xpack.securitySolution.markdownEditor.plugins.insightProviderError', {
defaultMessage: 'Unable to parse insight provider configuration',
}),
});
}
const { data: alertData } = useContext(BasicAlertDataContext);
const dataProviders = useInsightDataProviders({
providers: parsedProviders,
alertData,
});
const { totalCount, isQueryLoading, oldestTimestamp } = useInsightQuery({
dataProviders,
});
if (isQueryLoading) {
return <EuiLoadingSpinner size="l" />;
} else {
return (
<InvestigateInTimelineButton
asEmptyButton={false}
dataProviders={dataProviders}
timeRange={oldestTimestamp}
keepDataView={true}
data-test-subj="insight-investigate-in-timeline-button"
>
<EuiIcon type="timeline" />
{` ${label} (${totalCount}) - ${description}`}
</InvestigateInTimelineButton>
);
}
};

export { InsightComponent as renderer };

const InsightEditorComponent = ({
node,
Copy link
Contributor

@michaelolo24 michaelolo24 Nov 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These props aren't used

onSave,
onCancel,
}: EuiMarkdownEditorUiPluginEditorProps<InsightComponentProps>) => {
return (
<form>
<input type="text" />
</form>
);
};

export const plugin = {
name: 'insight',
button: {
label: 'Insights',
iconType: 'timeline',
},
helpText: (
<div>
<EuiCodeBlock language="md" fontSize="l" paddingSize="s" isCopyable>
{'!{insight{options}}'}
</EuiCodeBlock>
<EuiSpacer size="s" />
</div>
),
editor: InsightEditorComponent,
};
Loading