Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Transform] Transforms health alerting rule type #112277

Merged
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
b7a5e78
[ML] init public code
darnautov Sep 13, 2021
22764b1
[ML] init transform rule type from the monitoring plugin
darnautov Sep 14, 2021
5528e11
[ML] add UI form
darnautov Sep 15, 2021
4e893cd
[ML] only suggest continuous transforms
darnautov Sep 15, 2021
01d4681
[ML] executor code for transform state check
darnautov Sep 15, 2021
ae1f4b4
[ML] update type
darnautov Sep 15, 2021
90b0057
[ML] add description
darnautov Sep 15, 2021
804e98c
[ML] rename state prop
darnautov Sep 15, 2021
fea3b9f
[ML] rename state prop
darnautov Sep 15, 2021
4d38af1
[ML] update test assertion
darnautov Sep 15, 2021
e99df63
[ML] fix import
darnautov Sep 16, 2021
6c27a24
[ML] fix i18 ids
darnautov Sep 16, 2021
25e205d
[ML] update description
darnautov Sep 16, 2021
9b45e6b
[ML] validation for Included transforms
darnautov Sep 16, 2021
4a66cd5
[ML] fix exclude condtition
darnautov Sep 16, 2021
7143506
[ML] remove transform rule from the monitoring rules
darnautov Sep 16, 2021
b7c5e39
[ML] add line break
darnautov Sep 16, 2021
8777531
Merge remote-tracking branch 'upstream/master' into transform-111945-…
darnautov Sep 16, 2021
bbad483
[ML] disable control
darnautov Sep 16, 2021
2d3db4d
[ML] update context description
darnautov Sep 16, 2021
fe295c2
[ML] update test assertion
darnautov Sep 16, 2021
e8ba26e
Update text
darnautov Sep 22, 2021
2f78ca0
Merge branch 'master' into transform-111945-health-alerting-rule
kibanamachine Sep 22, 2021
5f15ad1
Merge remote-tracking branch 'upstream/master' into transform-111945-…
darnautov Sep 23, 2021
422777a
Merge remote-tracking branch 'upstream/master' into transform-111945-…
darnautov Oct 4, 2021
76ef141
[ML] fix eslint
darnautov Oct 4, 2021
a4f2e28
[ML] fix eslint
darnautov Oct 4, 2021
af04cc9
[ML] api integration tests for transform alerting rule
darnautov Oct 5, 2021
621f45a
Merge branch 'master' into transform-111945-health-alerting-rule
kibanamachine Oct 5, 2021
a6d8fe6
[ML] move transform alert setup, set Stack Rules as a producer
darnautov Oct 6, 2021
c363b2c
[ML] remove stack_alerts project ref
darnautov Oct 6, 2021
de0a7d9
[ML] update mappings
darnautov Oct 6, 2021
0d5253c
[ML] update mappings
darnautov Oct 6, 2021
4cbd7d2
[ML] update unit tests
darnautov Oct 6, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/core/public/doc_links/doc_links_service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,8 @@ export class DocLinksService {
},
transforms: {
guide: `${ELASTICSEARCH_DOCS}transforms.html`,
// TODO add valid docs URL
alertingRules: `${ELASTIC_WEBSITE_URL}guide/en/machine-learning/${DOC_LINK_VERSION}/ml-configuring-alerts.html`,
Comment on lines +286 to +287
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you planning to address this TODO is the current PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no docs page for it yet, so I'm going to add it in the follow-up PR probably as soon as it's ready

Copy link
Contributor

@szabosteve szabosteve Oct 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@darnautov This is the correct link. Docs are available at https://www.elastic.co/guide/en/elasticsearch/reference/master/transform-alerts.html

Suggested change
// TODO add valid docs URL
alertingRules: `${ELASTIC_WEBSITE_URL}guide/en/machine-learning/${DOC_LINK_VERSION}/ml-configuring-alerts.html`,
// TODO add valid docs URL
alertingRules: `${ELASTICSEARCH_DOCS}transform-alerts.html`,

},
visualize: {
guide: `${KIBANA_DOCS}dashboard.html`,
Expand Down
4 changes: 3 additions & 1 deletion x-pack/plugins/monitoring/common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import { i18n } from '@kbn/i18n';
import { CommonAlertParamDetail } from './types/alerts';
import { AlertParamType } from './enums';
import { TRANSFORM_RULE_TYPE } from '../../transform/common';

/**
* Helper string to add as a tag in every logging call
Expand Down Expand Up @@ -538,7 +539,8 @@ export const RULES = [
RULE_THREAD_POOL_WRITE_REJECTIONS,
RULE_CCR_READ_EXCEPTIONS,
RULE_LARGE_SHARD_SIZE,
];
TRANSFORM_RULE_TYPE.TRANSFORM_HEALTH,
] as const;

/**
* A list of all legacy rules, which means they are powered by watcher
Expand Down
2 changes: 1 addition & 1 deletion x-pack/plugins/monitoring/kibana.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,5 @@
],
"server": true,
"ui": true,
"requiredBundles": ["kibanaUtils", "home", "alerting", "kibanaReact", "licenseManagement"]
"requiredBundles": ["kibanaUtils", "home", "alerting", "kibanaReact", "licenseManagement", "transform"]
}
2 changes: 2 additions & 0 deletions x-pack/plugins/monitoring/public/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ import { createMemoryUsageAlertType } from './alerts/memory_usage_alert';
import { createCCRReadExceptionsAlertType } from './alerts/ccr_read_exceptions_alert';
import { createLargeShardSizeAlertType } from './alerts/large_shard_size_alert';
import { setConfig } from './external_config';
import { createTransformHealthRuleType } from '../../transform/public';

interface MonitoringSetupPluginDependencies {
home?: HomePublicPluginSetup;
Expand Down Expand Up @@ -213,6 +214,7 @@ export class MonitoringPlugin
);
ruleTypeRegistry.register(createCCRReadExceptionsAlertType());
ruleTypeRegistry.register(createLargeShardSizeAlertType());
ruleTypeRegistry.register(createTransformHealthRuleType());
const legacyAlertTypes = createLegacyAlertTypes();
for (const legacyAlertType of legacyAlertTypes) {
ruleTypeRegistry.register(legacyAlertType);
Expand Down
4 changes: 1 addition & 3 deletions x-pack/plugins/monitoring/server/plugin.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@

import { coreMock } from 'src/core/server/mocks';
import { MonitoringPlugin } from './plugin';
import { AlertsFactory } from './alerts';

jest.mock('./es_client/instantiate_client', () => ({
instantiateClient: jest.fn().mockImplementation(() => ({
Expand Down Expand Up @@ -72,9 +71,8 @@ describe('Monitoring plugin', () => {
});

it('should register all alerts', async () => {
const alerts = AlertsFactory.getAll();
const plugin = new MonitoringPlugin(initializerContext as any);
await plugin.setup(coreSetup as any, setupPlugins as any);
expect(setupPlugins.alerting.registerType).toHaveBeenCalledTimes(alerts.length);
expect(setupPlugins.alerting.registerType).toHaveBeenCalledTimes(16);
});
});
3 changes: 3 additions & 0 deletions x-pack/plugins/monitoring/server/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ import {
PluginsStart,
RequestHandlerContextMonitoringPlugin,
} from './types';
import { getTransformHealthRuleType } from '../../transform/server';

// This is used to test the version of kibana
const snapshotRegex = /-snapshot/i;
Expand Down Expand Up @@ -127,6 +128,8 @@ export class MonitoringPlugin
for (const alert of alerts) {
plugins.alerting?.registerType(alert.getRuleType());
}
plugins.alerting?.registerType(getTransformHealthRuleType());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any specific reason to do not use a factory?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed it and realized it's not a right fit for the Stack Monitoring rules. We're going to include it in the Stack Rules instead.


const config = createConfig(this.initializerContext.config.get<TypeOf<typeof configSchema>>());

// Register collector objects for stats to show up in the APIs
Expand Down
1 change: 1 addition & 0 deletions x-pack/plugins/monitoring/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,5 +31,6 @@
{ "path": "../observability/tsconfig.json" },
{ "path": "../telemetry_collection_xpack/tsconfig.json" },
{ "path": "../triggers_actions_ui/tsconfig.json" },
{ "path": "../transform/tsconfig.json" },
]
}
24 changes: 24 additions & 0 deletions x-pack/plugins/transform/common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import { i18n } from '@kbn/i18n';

import { LicenseType } from '../../licensing/common/types';
import { TransformHealthTests } from './types/alerting';

export const DEFAULT_REFRESH_INTERVAL_MS = 30000;
export const MINIMUM_REFRESH_INTERVAL_MS = 1000;
Expand Down Expand Up @@ -105,3 +106,26 @@ export const TRANSFORM_FUNCTION = {
} as const;

export type TransformFunction = typeof TRANSFORM_FUNCTION[keyof typeof TRANSFORM_FUNCTION];

export const TRANSFORM_RULE_TYPE = {
TRANSFORM_HEALTH: 'transform_health',
} as const;

export const ALL_TRANSFORMS_SELECTION = '*';

export const TRANSFORM_HEALTH_CHECK_NAMES: Record<
TransformHealthTests,
{ name: string; description: string }
> = {
notStarted: {
name: i18n.translate('xpack.transform.alertTypes.transformHealth.notStartedCheckName', {
defaultMessage: 'Transform is not started',
}),
description: i18n.translate(
'xpack.transform.alertTypes.transformHealth.notStartedCheckDescription',
{
defaultMessage: 'Get alerted if the transform is not started or indexing any data.',
darnautov marked this conversation as resolved.
Show resolved Hide resolved
}
),
},
};
8 changes: 8 additions & 0 deletions x-pack/plugins/transform/common/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

export { TRANSFORM_RULE_TYPE } from './constants';
22 changes: 22 additions & 0 deletions x-pack/plugins/transform/common/types/alerting.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import type { AlertTypeParams } from '../../../alerting/common';

export type TransformHealthRuleParams = {
includeTransforms?: string[];
excludeTransforms?: string[] | null;
testsConfig?: {
notStarted?: {
enabled: boolean;
} | null;
} | null;
} & AlertTypeParams;

export type TransformHealthRuleTestsConfig = TransformHealthRuleParams['testsConfig'];

export type TransformHealthTests = keyof Exclude<TransformHealthRuleTestsConfig, null | undefined>;
4 changes: 4 additions & 0 deletions x-pack/plugins/transform/common/types/common.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,7 @@ export function dictionaryToArray<TValue>(dict: Dictionary<TValue>): TValue[] {
export type DeepPartial<T> = {
[P in keyof T]?: DeepPartial<T[P]>;
};

export function isDefined<T>(argument: T | undefined | null): argument is T {
return argument !== undefined && argument !== null;
}
16 changes: 16 additions & 0 deletions x-pack/plugins/transform/common/utils/alerts.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import type { TransformHealthRuleTestsConfig } from '../types/alerting';

export function getResultTestConfig(config: TransformHealthRuleTestsConfig) {
return {
notStarted: {
enabled: config?.notStarted?.enabled ?? true,
},
};
}
10 changes: 8 additions & 2 deletions x-pack/plugins/transform/kibana.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,14 @@
"management",
"features",
"savedObjects",
"share"
"share",
"triggersActionsUi",
"fieldFormats"
],
"optionalPlugins": [
"security",
"usageCollection"
"usageCollection",
"alerting"
],
"configPath": ["xpack", "transform"],
"requiredBundles": [
Expand All @@ -24,6 +27,9 @@
"kibanaReact",
"ml"
],
"extraPublicDirs": [
"common"
],
"owner": {
"name": "Machine Learning UI",
"githubTeam": "ml-ui"
Expand Down
8 changes: 8 additions & 0 deletions x-pack/plugins/transform/public/alerting/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

export { createTransformHealthRuleType } from './transform_health_rule_type';
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

export { createTransformHealthRuleType } from './register_transform_health_rule';
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { lazy } from 'react';
import { i18n } from '@kbn/i18n';
import { TRANSFORM_RULE_TYPE } from '../../../common';
import type { TransformHealthRuleParams } from '../../../common/types/alerting';
import type { AlertTypeModel } from '../../../../triggers_actions_ui/public';

export function createTransformHealthRuleType(): AlertTypeModel<TransformHealthRuleParams> {
return {
id: TRANSFORM_RULE_TYPE.TRANSFORM_HEALTH,
description: i18n.translate('xpack.transform.alertingRuleTypes.transformHealth.description', {
defaultMessage: 'Alert when transform jobs experience operational issues.',
peteharverson marked this conversation as resolved.
Show resolved Hide resolved
}),
iconClass: 'bell',
documentationUrl(docLinks) {
return docLinks.links.transforms.alertingRules;
},
alertParamsExpression: lazy(() => import('./transform_health_rule_trigger')),
validate: (alertParams: TransformHealthRuleParams) => {
const validationResult = {
errors: {
includeTransforms: new Array<string>(),
} as Record<keyof TransformHealthRuleParams, string[]>,
};

return validationResult;
},
requiresAppContext: false,
defaultActionMessage: i18n.translate(
'xpack.transform.alertTypes.transformHealth.defaultActionMessage',
{
defaultMessage: `[\\{\\{rule.name\\}\\}] Transforms health check result:
peteharverson marked this conversation as resolved.
Show resolved Hide resolved
\\{\\{context.message\\}\\}
\\{\\{#context.results\\}\\}
Transform ID: \\{\\{transform_id\\}\\}
peteharverson marked this conversation as resolved.
Show resolved Hide resolved
\\{\\{#description\\}\\}Transform description: \\{\\{description\\}\\}
\\{\\{/description\\}\\}\\{\\{#transform_state\\}\\}Transform state: \\{\\{transform_state\\}\\}
\\{\\{/transform_state\\}\\}\\{\\{#failure_reason\\}\\}Failure reason: \\{\\{failure_reason\\}\\}
\\{\\{/failure_reason\\}\\}\\{\\{#notification_message\\}\\}Notification message: \\{\\{notification_message\\}\\}
\\{\\{/notification_message\\}\\}\\{\\{#node_name\\}\\}Node name: \\{\\{node_name\\}\\}
\\{\\{/node_name\\}\\}\\{\\{#timestamp\\}\\}Timestamp: \\{\\{timestamp\\}\\}
\\{\\{/timestamp\\}\\}
\\{\\{/context.results\\}\\}
`,
}
),
};
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import React, { FC, useCallback } from 'react';
import { EuiDescribedFormGroup, EuiForm, EuiFormRow, EuiSpacer, EuiSwitch } from '@elastic/eui';
import { FormattedMessage } from '@kbn/i18n/react';

import {
TransformHealthRuleTestsConfig,
TransformHealthTests,
} from '../../../common/types/alerting';
import { getResultTestConfig } from '../../../common/utils/alerts';
import { TRANSFORM_HEALTH_CHECK_NAMES } from '../../../common/constants';

interface TestsSelectionControlProps {
config: TransformHealthRuleTestsConfig;
onChange: (update: TransformHealthRuleTestsConfig) => void;
errors?: string[];
}

export const TestsSelectionControl: FC<TestsSelectionControlProps> = React.memo(
({ config, onChange, errors }) => {
const uiConfig = getResultTestConfig(config);

const updateCallback = useCallback(
(update: Partial<Exclude<TransformHealthRuleTestsConfig, undefined>>) => {
onChange({
...(config ?? {}),
...update,
});
},
[onChange, config]
);

return (
<>
<EuiForm component="div" isInvalid={!!errors?.length} error={errors}>
{(Object.entries(uiConfig) as Array<
[TransformHealthTests, typeof uiConfig[TransformHealthTests]]
>).map(([name, conf], i) => {
return (
<EuiDescribedFormGroup
key={name}
title={<h4>{TRANSFORM_HEALTH_CHECK_NAMES[name]?.name}</h4>}
description={TRANSFORM_HEALTH_CHECK_NAMES[name]?.description}
fullWidth
gutterSize={'s'}
>
<EuiFormRow>
<EuiSwitch
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With only one check currently available, is it worth disabling the switch control until the next check is added?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disabled in darnautov@bbad483

label={
<FormattedMessage
id="xpack.transform.alertTypes.transformHealth.testsSelection.enableTestLabel"
defaultMessage="Enable"
/>
}
onChange={updateCallback.bind(null, {
[name]: {
...uiConfig[name],
enabled: !uiConfig[name].enabled,
},
})}
checked={uiConfig[name].enabled}
/>
</EuiFormRow>
</EuiDescribedFormGroup>
);
})}
</EuiForm>
<EuiSpacer size="l" />
</>
);
}
);
Loading