Update readme

x-pack/plugins/rule_registry/README.md

@@ -2,62 +2,129 @@
 
 The rule registry plugin aims to make it easy for rule type producers to have their rules produce the data that they need to build rich experiences on top of a unified experience, without the risk of mapping conflicts.
 
-A rule registry creates a template, an ILM policy, and an alias. The template mappings can be configured. It also injects a client scoped to these indices.
+The plugin installs default component templates and a default lifecycle policy that rule type producers can use to create index templates.
 
-It also supports inheritance, which means that producers can create a registry specific to their solution or rule type, and specify additional mappings to be used.
+It also exposes a rule data client that will create or update the index stream that rules will write data to. It will not do so on plugin setup or start, but only when data is written.
 
-The rule registry plugin creates a root rule registry, with the mappings defined needed to create a unified experience. Rule type producers can use the plugin to access the root rule registry, and create their own registry that branches off of the root rule registry. The rule registry client sees data from its own registry, and all registries that branches off of it. It does not see data from its parents.
+## Configuration
 
-## Enabling writing
-
-Set
+By default, these indices will be prefixed with `.alerts`. To change this, for instance to support legacy multitenancy, set the following configuration option:
 
 ```yaml
-xpack.ruleRegistry.unsafe.write.enabled: true
+xpack.ruleRegistry.index: '.kibana-alerts'
 ```
 
-in your Kibana configuration to allow the Rule Registry to write events to the alert indices.
+To disable writing entirely:
+
+```yaml
+xpack.ruleRegistry.write.enabled: false
+```
 
-## Creating a rule registry
+## Setting up the index template
 
-To create a rule registry, producers should add the `ruleRegistry` plugin to their dependencies. They can then use the `ruleRegistry.create` method to create a child registry, with the additional mappings that should be used by specifying `fieldMap`:
+On plugin setup, rule type producers can create the index template as follows:
 
 ```ts
-const observabilityRegistry = plugins.ruleRegistry.create({
-  name: 'observability',
-  fieldMap: {
-    ...pickWithPatterns(ecsFieldMap, 'host.name', 'service.name'),
-  },
-});
-```
+// get the FQN of the component template. All assets are prefixed with the configured `index` value, which is `.alerts` by default.
 
-`fieldMap` is a key-value map of field names and mapping options:
+const componentTemplateName = plugins.ruleRegistry.getFullAssetName(
+  'apm-mappings'
+);
 
-```ts
-{
-  '@timestamp': {
-    type: 'date',
-    array: false,
-    required: true,
-  }
+// if write is disabled, don't install these templates
+if (!plugins.ruleRegistry.isWriteEnabled()) {
+  return;
 }
-```
 
-ECS mappings are generated via a script in the rule registry plugin directory. These mappings are available in x-pack/plugins/rule_registry/server/generated/ecs_field_map.ts.
-
-To pick many fields, you can use `pickWithPatterns`, which supports wildcards with full type support.
+// create or update the component template that should be used
+await plugins.ruleRegistry.createOrUpdateComponentTemplate({
+  name: componentTemplateName,
+  body: {
+    template: {
+      settings: {
+        number_of_shards: 1,
+      },
+      // mappingFromFieldMap is a utility function that will generate an
+      // ES mapping from a field map object. You can also define a literal
+      // mapping.
+      mappings: mappingFromFieldMap({
+        [SERVICE_NAME]: {
+          type: 'keyword',
+        },
+        [SERVICE_ENVIRONMENT]: {
+          type: 'keyword',
+        },
+        [TRANSACTION_TYPE]: {
+          type: 'keyword',
+        },
+        [PROCESSOR_EVENT]: {
+          type: 'keyword',
+        },
+      }),
+    },
+  },
+});
 
-If a registry is created, it will initialise as soon as the core services needed become available. It will create a (versioned) template, alias, and ILM policy, but only if these do not exist yet.
+// Install the index template, that is composed of the component template
+// defined above, and others. It is important that the technical component
+// template is included. This will ensure functional compatibility across
+// rule types, for a future scenario where a user will want to "point" the
+// data from a rule to a different index.
+await plugins.ruleRegistry.createOrUpdateIndexTemplate({
+  name: plugins.ruleRegistry.getFullAssetName('apm-index-template'),
+  body: {
+    index_patterns: [
+      plugins.ruleRegistry.getFullAssetName('observability-apm*'),
+    ],
+    composed_of: [
+      // Technical component template, required
+      plugins.ruleRegistry.getFullAssetName(
+        TECHNICAL_COMPONENT_TEMPLATE_NAME
+      ),
+      componentTemplateName,
+    ],
+  },
+});
 
-## Rule registry client
+// Finally, create the rule data client that can be injected into rule type
+// executors and API endpoints
+const ruleDataClient = new RuleDataClient({
+  alias: plugins.ruleRegistry.getFullAssetName('observability-apm'),
+  getClusterClient: async () => {
+    const coreStart = await getCoreStart();
+    return coreStart.elasticsearch.client.asInternalUser;
+  },
+  ready,
+});
 
-The rule registry client can either be injected in the executor, or created in the scope of a request. It exposes a `search` method and a `bulkIndex` method. When `search` is called, it first gets all the rules the current user has access to, and adds these ids to the search request that it executes. This means that the user can only see data from rules they have access to.
+// to start writing data, call `getWriter().bulk()`. It supports a `namespace`
+// property as well, that for instance can be used to write data to a space-specific
+// index.
+await ruleDataClient.getWriter().bulk({
+  body: eventsToIndex.flatMap((event) => [{ index: {} }, event]),
+});
 
-Both `search` and `bulkIndex` are fully typed, in the sense that they reflect the mappings defined for the registry.
+// to read data, simply call ruleDataClient.getReader().search:
+const response = await ruleDataClient.getReader().search({
+  body: {
+    query: {
+    },
+    size: 100,
+    fields: ['*'],
+    collapse: {
+      field: ALERT_UUID,
+    },
+    sort: {
+      '@timestamp': 'desc',
+    },
+  },
+  allow_no_indices: true,
+});
+```
 
 ## Schema
 
-The following fields are available in the root rule registry:
+The following fields are defined in the technical field component template and should always be used:
 
 - `@timestamp`: the ISO timestamp of the alert event. For the lifecycle rule type helper, it is always the value of `startedAt` that is injected by the Kibana alerting framework.
 - `event.kind`: signal (for the changeable alert document), state (for the state changes of the alert, e.g. when it opens, recovers, or changes in severity), or metric (individual evaluations that might be related to an alert).
@@ -67,7 +134,7 @@ The following fields are available in the root rule registry:
 - `rule.uuid`: the saved objects id of the rule.
 - `rule.name`: the name of the rule (as specified by the user).
 - `rule.category`: the name of the rule type (as defined by the rule type producer)
-- `kibana.rac.producer`: the producer of the rule type. Usually a Kibana plugin. e.g., `APM`.
+- `kibana.rac.alert.producer`: the producer of the rule type. Usually a Kibana plugin. e.g., `APM`.
 - `kibana.rac.alert.id`: the id of the alert, that is unique within the context of the rule execution it was created in. E.g., for a rule that monitors latency for all services in all environments, this might be `opbeans-java:production`.
 - `kibana.rac.alert.uuid`: the unique identifier for the alert during its lifespan. If an alert recovers (or closes), this identifier is re-generated when it is opened again.
 - `kibana.rac.alert.status`: the status of the alert. Can be `open` or `closed`.
@@ -76,5 +143,5 @@ The following fields are available in the root rule registry:
 - `kibana.rac.alert.duration.us`: the duration of the alert, in microseconds. This is always the difference between either the current time, or the time when the alert recovered.
 - `kibana.rac.alert.severity.level`: the severity of the alert, as a keyword (e.g. critical).
 - `kibana.rac.alert.severity.value`: the severity of the alert, as a numerical value, which allows sorting.
-
-This list is not final - just a start. Field names might change or moved to a scoped registry. If we implement log and sequence based rule types the list of fields will grow. If a rule type needs additional fields, the recommendation would be to have the field in its own registry first (or in its producer’s registry), and if usage is more broadly adopted, it can be moved to the root registry.
+- `kibana.rac.alert.evaluation.value`: The measured (numerical value).
+- `kibana.rac.alert.threshold.value`: The threshold that was defined (or, in case of multiple thresholds, the one that was exceeded).

x-pack/plugins/rule_registry/common/assets.ts

@@ -5,8 +5,6 @@
  * 2.0.
  */
 
-export const DEFAULT_ASSET_NAMESPACE = 'alerts';
-
 export const TECHNICAL_COMPONENT_TEMPLATE_NAME = `technical-mappings`;
 export const ECS_COMPONENT_TEMPLATE_NAME = `ecs-mappings`;
 export const DEFAULT_ILM_POLICY_ID = 'ilm-policy';

x-pack/plugins/rule_registry/server/index.ts

@@ -18,9 +18,10 @@ export { createLifecycleRuleTypeFactory } from './utils/create_lifecycle_rule_ty
 export const config = {
   schema: schema.object({
     enabled: schema.boolean({ defaultValue: true }),
-    unsafe: schema.object({
-      write: schema.object({ enabled: schema.boolean({ defaultValue: false }) }),
+    write: schema.object({
+      enabled: schema.boolean({ defaultValue: true }),
     }),
+    index: schema.string({ defaultValue: '.alerts' }),
   }),
 };
 

x-pack/plugins/rule_registry/server/plugin.ts

@@ -18,15 +18,14 @@ export class RuleRegistryPlugin implements Plugin<RuleRegistryPluginSetupContrac
   }
 
   public setup(core: CoreSetup): RuleRegistryPluginSetupContract {
-    const globalConfig = this.initContext.config.legacy.get();
     const config = this.initContext.config.get<RuleRegistryPluginConfig>();
 
     const logger = this.initContext.logger.get();
 
     const service = new RuleDataPluginService({
       logger,
-      isWriteEnabled: config.unsafe.write.enabled,
-      kibanaIndex: globalConfig.kibana.index,
+      isWriteEnabled: config.write.enabled,
+      index: config.index,
       getClusterClient: async () => {
         const [coreStart] = await core.getStartServices();
 

x-pack/plugins/rule_registry/server/rule_data_plugin_service/index.ts

@@ -7,7 +7,6 @@
 import { ClusterPutComponentTemplate } from '@elastic/elasticsearch/api/requestParams';
 import { estypes } from '@elastic/elasticsearch';
 import { ElasticsearchClient, Logger } from 'kibana/server';
-import { DEFAULT_ASSET_NAMESPACE } from '../../common/assets';
 import { technicalComponentTemplate } from '../../common/assets/component_templates/technical_component_template';
 import {
   DEFAULT_ILM_POLICY_ID,
@@ -24,7 +23,7 @@ interface RuleDataPluginServiceConstructorOptions {
   getClusterClient: () => Promise<ElasticsearchClient>;
   logger: Logger;
   isWriteEnabled: boolean;
-  kibanaIndex: string;
+  index: string;
 }
 
 function createSignal() {
@@ -154,6 +153,6 @@ export class RuleDataPluginService {
   }
 
   getFullAssetName(assetName?: string) {
-    return [this.options.kibanaIndex, DEFAULT_ASSET_NAMESPACE, assetName].filter(Boolean).join('-');
+    return [this.options.index, assetName].filter(Boolean).join('-');
   }
 }

x-pack/test/apm_api_integration/configs/index.ts

@@ -17,9 +17,6 @@ const apmFtrConfigs = {
   },
   rules: {
     license: 'trial' as const,
-    kibanaConfig: {
-      'xpack.ruleRegistry.unsafe.write.enabled': 'true',
-    },
   },
 };