Introduce basic alerting and actions plugin (elastic#37042)

* Create actions plugin (elastic#35679) * Basic alerting plugin with actions * Remove relative imports * Code cleanup * Split service into 3 parts, change connector structure * Ability to disable plugin, ability to get actions * Add slack connector * Add email connector * Ability to validate params and connector options * Remove connectorOptionsSecrets for now * Fix plugin config validation * Add tests for slack connector * Default connectors register on plugin init, console renamed to log, slack to message_slack * Add remaining API endpoints for action CRUD * Add list connectors API * Change actions CRUD APIs to be closer with saved objects structure * WIP * Fix broken tests * Add encrypted attribute support * Add params and connectorOptions for email * WIP * Remove action's ability to have custom ids * Remove ts-ignore * Fix broken test * Remove default connectors from this branch * Fix API integration tests to use fixture connector * Rename connector terminology to action type * Rename actionTypeOptions to actionTypeConfig * Code cleanup * Fix broken tests * Rename alerting plugin to actions * Some code cleanup and add API unit tests * Change signature of action type service execute function * Add some plugin api integration tests * Fix type check failure * Code cleanup * Create an actions client instead of an action service * Apply Bill's PR feedback * Fix broken test * Find function to have destructured params * Add tests to ensure encrypted attributes are not returned * Fix broken test * Add tests for validation * Ensure actions can be updated without re-passing the config * Remove dead code * Test cleanup * Fix eslint issue * Apply Peter's PR feedback * Code cleanup and fix broken tests * Apply Brandon's PR feedback * Add namespace support * Fix broken test * Pass services to action executors (elastic#37194) * Pass services to action executors * Fix tests * Apply PR feedback * Apply PR feedback pt2 * Cleanup actions plugin (elastic#37250) * Cleanup actions, move code from alerting plugin PR * Rename service terminology to registry * Use static encryption key for encrypted attributes plugin inside of tests * Empty data after create test is done running * Fix type checks * Fix inconsistent naming * add server log action for alerting (elastic#37530) adds the first "builtin" alertType for performing a `server.log()` * Create alerting plugin (elastic#37043) * WIP * Rename fire function and remove @ts-ignore in all places * Change naming in alerting service * Remove alert instance class for now, support interval configuration * Cleanup TS * Split alerting between registry and client * Use saved object alongside task manager instance * Add remaining alerting APIs * Change create structure * Rename some variables, change actionGroups structure * Use handlebars for templating strings at fire time * Fix params given to alert type execute function * Use alert instance class * Alert instances support meta attributes * Move alert instances deserialization * Change interval to be ms * Rename actions es archive * Fix tests to use encrypted esArchive for action record * Add create alert test to demo end to end flow * Fix type check issue * Alerts to use references to action objects * Only update task manager tasks after saved objects are fully updated * Use scope in task manager * Fix type check * Use task manager to execute actions * Convert ids into references and back * Apply PR feedback * Fix broken test * Fix some bugs * Fix test errors * Alert interval to be previous runAt + interval instead of now + interval * Add range support * Remove extra line * Cleanup * Add alert_instance.test.ts * Add alert_type_registry.test.ts * Move tests around * Create generic task manager mock * Add note about saved objects client mock * Create alert_type_registry.mock.ts * Add alerts_client.test.ts * Add create_alert_instance_factory.test.ts * Add create_fire_handler.test.ts * WIP * Fix get_create_task_runner_function.test.ts and make test pass * Make get_create_task_runner_function.test.ts 100% coverage * Add unit tests for routes * Move files around * Created transform_action_params.ts * Add get_next_run_at.ts * Add comment explaining why we copy nextRunAt * Re-use state within alert instance * Finalize code coverage in unit tests * Create base api integration tests * Add a test that ensures end to end functionality of an alert * Fix ui capabilities test * Fix broken plugin api integration test * Fix jest tests with new saved objects client * Fix broken integration tests * Change api integration test fixture to make more sense, add functions for future tests * Move alerts integration testing into own file, prep to add more tests * Add tests to ensure failed task instances get retried * Add get_create_task_runner_function.test.ts for actions, create encrypted saved objects mock * Add action validation tests * Ensure action type validation occurs on update * Test 400 on unregistered alert types * Ensure alertTypeId can't be updated * Add validation test for alert create / update * Fix broken checks / tests * Skip failing test for now * Cleanup jest tests * Ensure action objects can be updated while keeping encrypted attributes readable * Remove partial update sopport, remove ability to change actionTypeId, require config * Ensure actionTypeConfig is validated on create and update * Add alertTypeParams validation support * Fix failing tests * Ensure alert cleanup errors don't replace the original error * Pass callCluster as a service to alerts and actions * Only pass log to alerts client * Pass savedObjectsClient as a service to alerting and actions * Fix failing tests * Remove range support, provide when current and previous task got scheduled * Ensure Joi validation happens before every execute * Remove skipped tests, to be done in future PR * Apply self feedback pt1 * Apply self feedback pt2 * Fix broken tests * Apply PR feedback * PR feedback pt1 * Apply security team PR feedback * PR feedback pt1 * PR feedback pt2 * PR feedback pt3 * Fix broken tests * Fix callCluster to have signature * Revert f11a6ae * PR feedback pt4 * Remove __jest__ folders * PR feedback pt5 * Fix Joi from leaking secrets * Fire instance actions in parallel instead of series * Fix failing jest tests * Accept core api changes * Fix saved objects client mock * PR feedback pt1 * Fix eslint issues * Throw error when alert instance already fired (elastic#39251) * Throw error when alert instance already fired * shouldFire doesn't need its own boolean value * Actions & alerting getting started user guides (elastic#39093) * Initial user guides * Cleanup * Typos, example changes * Switch to tables, use ordered list for usage * Start docs around alert instances and templating * Documentation changes * Some adjustments * Apply PR feedback * Apply suggestions from code review Co-Authored-By: gchaps <[email protected]> * PR feedback pt2 * Provide better examples for alert types * Apply PR feedback * Update README locations
mikecote · Jun 21, 2019 · 4a30b28 · 4a30b28
1 parent 0ea39af
commit 4a30b28
Show file tree

Hide file tree

Showing 123 changed files with 8,239 additions and 7 deletions.
diff --git a/.i18nrc.json b/.i18nrc.json
@@ -23,6 +23,8 @@
     "tagCloud": "src/legacy/core_plugins/tagcloud",
     "tsvb": "src/legacy/core_plugins/metrics",
     "kbnESQuery": "packages/kbn-es-query",
+    "xpack.actions": "x-pack/legacy/plugins/actions",
+    "xpack.alerting": "x-pack/legacy/plugins/alerting",
     "xpack.apm": "x-pack/legacy/plugins/apm",
     "xpack.beatsManagement": "x-pack/legacy/plugins/beats_management",
     "xpack.canvas": "x-pack/legacy/plugins/canvas",

diff --git a/src/core/server/mocks.ts b/src/core/server/mocks.ts
@@ -21,3 +21,4 @@ export { configServiceMock } from './config/config_service.mock';
 export { elasticsearchServiceMock } from './elasticsearch/elasticsearch_service.mock';
 export { httpServiceMock } from './http/http_service.mock';
 export { loggingServiceMock } from './logging/logging_service.mock';
+export { SavedObjectsClientMock } from './saved_objects/service/saved_objects_client.mock';
diff --git a/src/core/server/saved_objects/service/saved_objects_client.ts b/src/core/server/saved_objects/service/saved_objects_client.ts
@@ -142,12 +142,25 @@ export interface SavedObjectsMigrationVersion {
   [pluginName: string]: string;
 }
 
+/**
+ *
+ * @public
+ */
+export type SavedObjectAttribute =
+  | string
+  | number
+  | boolean
+  | null
+  | undefined
+  | SavedObjectAttributes
+  | SavedObjectAttributes[];
+
 /**
  *
  * @public
  */
 export interface SavedObjectAttributes {
-  [key: string]: SavedObjectAttributes | string | number | boolean | null;
+  [key: string]: SavedObjectAttribute | SavedObjectAttribute[];
 }
 
 /**

diff --git a/src/core/server/server.api.md b/src/core/server/server.api.md
@@ -421,8 +421,10 @@ export interface SavedObject<T extends SavedObjectAttributes = any> {
 
 // @public (undocumented)
 export interface SavedObjectAttributes {
+    // Warning: (ae-forgotten-export) The symbol "SavedObjectAttribute" needs to be exported by the entry point index.d.ts
+    // 
     // (undocumented)
-    [key: string]: SavedObjectAttributes | string | number | boolean | null;
+    [key: string]: SavedObjectAttribute | SavedObjectAttribute[];
 }
 
 // @public

diff --git a/x-pack/index.js b/x-pack/index.js
@@ -43,6 +43,8 @@ import { fileUpload } from './legacy/plugins/file_upload';
 import { telemetry } from './legacy/plugins/telemetry';
 import { encryptedSavedObjects } from './legacy/plugins/encrypted_saved_objects';
 import { snapshotRestore } from './legacy/plugins/snapshot_restore';
+import { actions } from './legacy/plugins/actions';
+import { alerting } from './legacy/plugins/alerting';
 
 module.exports = function (kibana) {
   return [
@@ -85,5 +87,7 @@ module.exports = function (kibana) {
     fileUpload(kibana),
     encryptedSavedObjects(kibana),
     snapshotRestore(kibana),
+    actions(kibana),
+    alerting(kibana),
   ];
 };
diff --git a/x-pack/legacy/plugins/actions/README.md b/x-pack/legacy/plugins/actions/README.md
@@ -0,0 +1,177 @@
+# Kibana actions
+
+The Kibana actions plugin provides a common place to execute actions. You can:
+
+- Register an action type
+- View a list of registered types
+- Fire an action either manually or by using an alert
+- Perform CRUD on actions with encrypted configurations
+
+## Terminology
+
+**Action Type**: A programatically defined integration with another service, with an expected set of configuration and parameters.
+
+**Action**: A user-defined configuration that satisfies an action type's expected configuration.
+
+## Usage
+
+1. Develop and register an action type (see action types -> example).
+2. Create an action by using the RESTful API (see actions -> create action).
+3. Use alerts to fire actions or fire manually (see firing actions).
+
+## Action types
+
+### Methods
+
+**server.plugins.actions.registerType(options)**
+
+The following table describes the properties of the `options` object.
+
+|Property|Description|Type|
+|---|---|---|
+|id|Unique identifier for the action type. For convention, ids starting with `.` are reserved for built in action types. We recommend using a convention like `<plugin_id>.mySpecialAction` for your action types.|string|
+|name|A user-friendly name for the action type. These will be displayed in dropdowns when chosing action types.|string|
+|unencryptedAttributes|A list of opt-out attributes that don't need to be encrypted. These attributes won't need to be re-entered on import / export when the feature becomes available. These attributes will also be readable / displayed when it comes to a table / edit screen.|array of strings|
+|validate.params|When developing an action type, it needs to accept parameters to know what to do with the action. (Example to, from, subject, body of an email). Use joi object validation if you would like `params` to be validated before being passed to the executor.|Joi schema|
+|validate.config|Similar to params, a config is required when creating an action (for example host, port, username, and password of an email server). Use the joi object validation if you would like the config to be validated before being passed to the executor.|Joi schema|
+|executor|This is where the code of an action type lives. This is a function gets called for executing an action from either alerting or manually by using the exposed function (see firing actions). For full details, see executor section below.|Function|
+
+### Executor
+
+This is the primary function for an action type. Whenever the action needs to execute, this function will perform the action. It receives a variety of parameters.  The following table describes the properties that the executor receives.
+
+**executor(options)**
+
+|Property|Description|
+|---|---|
+|config|The decrypted configuration given to an action. This comes from the action saved object that is partially or fully encrypted within the data store. If you would like to validate the config before being passed to the executor, define `validate.config` within the action type.|
+|params|Parameters for the execution. These will be given at fire time by either an alert or manually provided when calling the plugin provided fire function.|
+|services.callCluster(path, opts)|Use this to do Elasticsearch queries on the cluster Kibana connects to. This function is the same as any other `callCluster` in Kibana.<br><br>**NOTE**: This currently authenticates as the Kibana internal user, but will change in a future PR.|
+|services.savedObjectsClient|This is an instance of the saved objects client. This provides the ability to do CRUD on any saved objects within the same space the alert lives in.<br><br>**NOTE**: This currently only works when security is disabled. A future PR will add support for enabling security using Elasticsearch API tokens.|
+|services.log(tags, [data], [timestamp])|Use this to create server logs. (This is the same function as server.log)|
+
+### Example
+
+Below is an example email action type. The attributes `host` and `port` are configured to be unencrypted by using the `unencryptedAttributes` attribute.
+
+```
+server.plugins.actions.registerType({
+  id: 'smtp',
+  name: 'Email',
+  unencryptedAttributes: ['host', 'port'],
+  validate: {
+    params: Joi.object()
+      .keys({
+        to: Joi.array().items(Joi.string()).required(),
+        from: Joi.string().required(),
+        subject: Joi.string().required(),
+        body: Joi.string().required(),
+      })
+      .required(),
+    config: Joi.object()
+      .keys({
+        host: Joi.string().required(),
+        port: Joi.number().default(465),
+        username: Joi.string().required(),
+        password: Joi.string().required(),
+      })
+      .required(),
+  },
+  async executor({ config, params, services }) {
+    const transporter = nodemailer. createTransport(config);
+    await transporter.sendMail(params);
+  },
+});
+```
+
+## RESTful API
+
+Using an action type requires an action to be created that will contain and encrypt configuration for a given action type. See below for CRUD operations using the API.
+
+#### `POST /api/action`: Create action
+
+Payload:
+
+|Property|Description|Type|
+|---|---|---|
+|attributes.description|A description to reference and search in the future. This value will be used to populate dropdowns.|string|
+|attributes.actionTypeId|The id value of the action type you want to call when the action executes.|string|
+|attributes.actionTypeConfig|The configuration the action type expects. See related action type to see what attributes is expected. This will also validate against the action type if config validation is defined.|object|
+|references|An array of `name`, `type` and `id`. This is the same as `references` in the saved objects API. See the saved objects API documentation.<br><br>In most cases, you can leave this empty.|Array|
+|migrationVersion|The version of the most recent migrations. This is the same as `migrationVersion` in the saved objects API. See the saved objects API documentation.<br><br>In most cases, you can leave this empty.|object|
+
+#### `DELETE /api/action/{id}`: Delete action
+
+Params:
+
+|Property|Description|Type|
+|---|---|---|
+|id|The id of the action you're trying to delete.|string|
+
+#### `GET /api/action/_find`: Find actions
+
+Params:
+
+See the saved objects API documentation for find. All the properties are the same except that you cannot pass in `type`.
+
+#### `GET /api/action/{id}`: Get action
+
+Params:
+
+|Property|Description|Type|
+|---|---|---|
+|id|The id of the action you're trying to get.|string|
+
+#### `GET /api/action/types` List action types
+
+No parameters.
+
+#### `PUT /api/action/{id}`: Update action
+
+Params:
+
+|Property|Description|Type|
+|---|---|---|
+|id|The id of the action you're trying to update.|string|
+
+Payload:
+
+|Property|Description|Type|
+|---|---|---|
+|attributes.description|A description to reference and search in the future. This value will be used to populate dropdowns.|string|
+|attributes.actionTypeConfig|The configuration the action type expects. See related action type to see what attributes is expected. This will also validate against the action type if config validation is defined.|object|
+|references|An array of `name`, `type` and `id`. This is the same as `references` in the saved objects API. See the saved objects API documentation.<br><br>In most cases, you can leave this empty.|Array|
+|version|The document version when read|string|
+
+## Firing actions
+
+The plugin exposes a fire function that you can use to fire actions.
+
+**server.plugins.actions.fire(options)**
+
+The following table describes the properties of the `options` object.
+
+|Property|Description|Type|
+|---|---|---|
+|id|The id of the action you want to fire.|string|
+|params|The `params` value to give the action type executor.|object|
+|namespace|The saved object namespace the action exists within.|string|
+|basePath|This is a temporary parameter, but we need to capture and track the value of `request.getBasePath()` until future changes are made.<br><br>In most cases this can be `undefined` unless you need cross spaces support.|string|
+
+### Example
+
+This example makes action `3c5b2bd4-5424-4e4b-8cf5-c0a58c762cc5` fire an email. The action plugin will load the saved object and find what action type to call with `params`.
+
+```
+server.plugins.actions.fire({
+  id: '3c5b2bd4-5424-4e4b-8cf5-c0a58c762cc5',
+  params: {
+    from: '[email protected]',
+    to: ['[email protected]'],
+    subject: 'My email subject',
+    body: 'My email body',
+  },
+  namespace: undefined, // The namespace the action exists within
+  basePath: undefined, // Usually `request.getBasePath();` or `undefined`
+});
+```
diff --git a/x-pack/legacy/plugins/actions/index.ts b/x-pack/legacy/plugins/actions/index.ts
@@ -0,0 +1,38 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License;
+ * you may not use this file except in compliance with the Elastic License.
+ */
+
+import { Legacy } from 'kibana';
+import { Root } from 'joi';
+import mappings from './mappings.json';
+import { init } from './server';
+
+export { ActionsPlugin, ActionsClient, ActionType, ActionTypeExecutorOptions } from './server';
+
+export function actions(kibana: any) {
+  return new kibana.Plugin({
+    id: 'actions',
+    configPrefix: 'xpack.actions',
+    require: ['kibana', 'elasticsearch', 'task_manager', 'encrypted_saved_objects'],
+    isEnabled(config: Legacy.KibanaConfig) {
+      return (
+        config.get('xpack.encrypted_saved_objects.enabled') === true &&
+        config.get('xpack.actions.enabled') === true &&
+        config.get('xpack.task_manager.enabled') === true
+      );
+    },
+    config(Joi: Root) {
+      return Joi.object()
+        .keys({
+          enabled: Joi.boolean().default(true),
+        })
+        .default();
+    },
+    init,
+    uiExports: {
+      mappings,
+    },
+  });
+}
diff --git a/x-pack/legacy/plugins/actions/mappings.json b/x-pack/legacy/plugins/actions/mappings.json
@@ -0,0 +1,19 @@
+{
+  "action": {
+    "properties": {
+      "description": {
+        "type": "text"
+      },
+      "actionTypeId": {
+        "type": "keyword"
+      },
+      "actionTypeConfig": {
+        "enabled": false,
+        "type": "object"
+      },
+      "actionTypeConfigSecrets": {
+        "type": "binary"
+      }
+    }
+  }
+}
diff --git a/x-pack/legacy/plugins/actions/server/action_type_registry.mock.ts b/x-pack/legacy/plugins/actions/server/action_type_registry.mock.ts
@@ -0,0 +1,23 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License;
+ * you may not use this file except in compliance with the Elastic License.
+ */
+
+import { ActionTypeRegistry } from './action_type_registry';
+
+type ActionTypeRegistryContract = PublicMethodsOf<ActionTypeRegistry>;
+
+const createActionTypeRegistryMock = () => {
+  const mocked: jest.Mocked<ActionTypeRegistryContract> = {
+    has: jest.fn(),
+    register: jest.fn(),
+    get: jest.fn(),
+    list: jest.fn(),
+  };
+  return mocked;
+};
+
+export const actionTypeRegistryMock = {
+  create: createActionTypeRegistryMock,
+};