Merge branch 'main' of github.com:elastic/kibana into add-feature-o11…

…y-server-less

docs/user/alerting/alerting-setup.asciidoc

@@ -58,26 +58,14 @@ feature. To change rule settings, you must have `all` privileges for the
 such as flapping detection. For more information on configuring roles that
 provide access to features, go to <<kibana-feature-privileges>>.
 
-For details about the prerequisites for each API, refer to <<alerting-apis>>.
+Each rule also has a rule visibility value (or `consumer` in the APIs), which affects the {kib} feature privileges that are required to access it.
+To view or edit a rule that has a `Stack Rules` rule visibility, for example, you must have the appropriate *Management > {stack-rules-feature}* feature privileges.
 
-[float]
-[[alerting-restricting-actions]]
-==== Restrict actions
-
-For security reasons you may wish to limit the extent to which {kib} can connect 
-to external services. <<action-settings>> allows you to disable certain 
-<<action-types>> and allowlist the hostnames that {kib} can connect with.
-
-[float]
-[[alerting-spaces]]
-=== Space isolation
-
-Rules and connectors are isolated to the {kib} space in which they were created. 
-A rule or connector created in one space will not be visible in another. 
+For details about the prerequisites required to run each API, refer to <<alerting-apis>>.
 
 [float]
 [[alerting-authorization]]
-=== Authorization
+==== API keys
 
 Rules are authorized using an API key.
 Its credentials are used to run all background tasks associated with the rule, including condition checks like {es} queries and triggered actions.
@@ -100,11 +88,25 @@ In both cases, the API key is subsequently associated with the rule and used whe
 
 [IMPORTANT]
 ==============================================
-If a rule requires certain privileges, such as index privileges, to run and a user without those privileges updates the rule, the rule will no longer  function.
+If a rule requires certain privileges, such as index privileges, to run and a user without those privileges updates the rule, the rule will no longer function.
 Conversely, if a user with greater or administrator privileges modifies the rule, it will begin running with increased privileges.
 The same behavior occurs when you change the API key in the header of your API calls.
 ==============================================
 
+[float]
+[[alerting-restricting-actions]]
+==== Restrict actions
+
+For security reasons you may wish to limit the extent to which {kib} can connect to external services.
+You can use <<action-settings>> to disable certain <<action-types>> and allowlist the hostnames that {kib} can connect with.
+
+[float]
+[[alerting-spaces]]
+=== Space isolation
+
+Rules and connectors are isolated to the {kib} space in which they were created. 
+A rule or connector created in one space will not be visible in another. 
+
 [float]
 [[alerting-ccs-setup]]
 === {ccs-cap}

docs/user/alerting/rule-types/es-query.asciidoc

@@ -42,6 +42,26 @@ Exclude matches from previous run:: Turn on to avoid alert duplication by
 excluding documents that have already been detected by the previous rule run. This
 option is not available when a grouping field is specified.
 
+You can optionally change the check interval, which defines how often to evaluate the rule conditions.
+
+You must select a scope value, which affects the <<kibana-feature-privileges,{kib} feature privileges>> that are required to access the rule.
+For example when it's set to `Stack Rules`, you must have the appropriate *Management > {stack-rules-feature}* feature privileges to view or edit the rule.
+
+[float]
+=== Test your query
+
+Use the *Test query* feature to verify that your query is valid.
+
+Valid queries are run against the selected indices using the configured time window.
+The number of documents that match the query is displayed.
+For example:
+
+[role="screenshot"]
+image::user/alerting/images/rule-types-es-query-valid.png[Test {es} query returns number of matches when valid]
+// NOTE: This is an autogenerated screenshot. Do not edit it directly.
+
+An error message is shown if the query is invalid.
+
 [float]
 === Add actions
 
@@ -155,24 +175,6 @@ Labels:
 // NOTCONSOLE
 --
 
-[float]
-=== Test your query
-
-Use the *Test query* feature to verify that your query DSL is valid.
-
-* Valid queries are run against the configured *index* using the configured 
-*time window*. The number of documents that match the query is displayed.
-+
-[role="screenshot"]
-image::user/alerting/images/rule-types-es-query-valid.png[Test {es} query returns number of matches when valid]
-// NOTE: This is an autogenerated screenshot. Do not edit it directly.
-
-* An error message is shown if the query is invalid.
-+
-[role="screenshot"]
-image::user/alerting/images/rule-types-es-query-invalid.png[Test {es} query shows error when invalid]
-// NOTE: This is an autogenerated screenshot. Do not edit it directly.
-
 [float]
 === Handling multiple matches of the same document
 

docs/user/production-considerations/alerting-production-considerations.asciidoc

@@ -72,6 +72,11 @@ There are several scenarios where running alerting rules and actions can start t
 
 Running large numbers of rules at very short intervals can quickly clog up Task Manager throughput, leading to higher schedule drift. Use `xpack.alerting.rules.minimumScheduleInterval.value` to set a minimum schedule interval for rules. The default (and recommended) value for this configuration is `1m`. Use `xpack.alerting.rules.minimumScheduleInterval.enforce` to specify whether to strictly enforce this minimum. While the default value for this setting is `false` to maintain backwards compatibility with existing rules, set this to `true` to prevent new and updated rules from running at an interval below the minimum.
 
+Another related setting is `xpack.alerting.rules.maxScheduledPerMinute`, which limits the number of rules that can run per minute.
+For example if it's set to `400`, you can have 400 rules with one minute check intervals or 2,000 rules with 5 minute check intervals.
+You cannot create or edit a rule if its check interval would cause this setting to be exceeded.
+To stay within this limit, delete or disable some rules or update the check intervals so that your rules run less frequently.
+
 [float]
 ==== Rules that run for a long time
 
@@ -106,4 +111,4 @@ xpack.alerting.rules.run:
     connectorTypeOverrides:
       - id: '.email'
         max: 200
---
+--

packages/kbn-management/cards_navigation/src/cards_navigation.tsx

@@ -184,7 +184,7 @@ export const CardsNavigation = ({
                 <EuiCard
                   data-test-subj={`app-card-${app.id}`}
                   layout="horizontal"
-                  icon={<EuiIcon type={app.icon} size="l" color="darkShade" />}
+                  icon={<EuiIcon type={app.icon} size="l" color="text" />}
                   titleSize="xs"
                   title={app.title}
                   description={app.description}

x-pack/plugins/aiops/common/api/log_categorization/schema.ts

@@ -65,3 +65,6 @@ export const categorizationFieldValidationSchema = schema.object({
   indicesOptions: indicesOptionsSchema,
   includeExamples: schema.boolean(),
 });
+export type CategorizationFieldValidationSchema = TypeOf<
+  typeof categorizationFieldValidationSchema
+>;

x-pack/plugins/aiops/server/plugin.ts

@@ -19,9 +19,8 @@ import {
   AiopsPluginSetupDeps,
   AiopsPluginStartDeps,
 } from './types';
-
-import { defineLogRateAnalysisRoute } from './routes';
-import { defineLogCategorizationRoutes } from './routes/log_categorization';
+import { defineRoute as defineLogRateAnalysisRoute } from './routes/log_rate_analysis/define_route';
+import { defineRoute as defineCategorizationFieldValidationRoute } from './routes/categorization_field_validation/define_route';
 import { registerCasesPersistableState } from './register_cases';
 
 export class AiopsPlugin
@@ -59,7 +58,7 @@ export class AiopsPlugin
     // Register server side APIs
     core.getStartServices().then(([coreStart, depsStart]) => {
       defineLogRateAnalysisRoute(router, aiopsLicense, this.logger, coreStart, this.usageCounter);
-      defineLogCategorizationRoutes(router, aiopsLicense, this.usageCounter);
+      defineCategorizationFieldValidationRoute(router, aiopsLicense, this.usageCounter);
     });
 
     return {};

x-pack/plugins/aiops/server/routes/categorization_field_validation/define_route.ts

@@ -0,0 +1,37 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import type { IRouter } from '@kbn/core/server';
+import type { DataRequestHandlerContext } from '@kbn/data-plugin/server';
+import type { UsageCounter } from '@kbn/usage-collection-plugin/server';
+import { categorizationFieldValidationSchema } from '../../../common/api/log_categorization/schema';
+import { AIOPS_API_ENDPOINT } from '../../../common/api';
+import type { AiopsLicense } from '../../types';
+import { routeHandlerFactory } from './route_handler_factory';
+
+export const defineRoute = (
+  router: IRouter<DataRequestHandlerContext>,
+  license: AiopsLicense,
+  usageCounter?: UsageCounter
+) => {
+  router.versioned
+    .post({
+      path: AIOPS_API_ENDPOINT.CATEGORIZATION_FIELD_VALIDATION,
+      access: 'internal',
+    })
+    .addVersion(
+      {
+        version: '1',
+        validate: {
+          request: {
+            body: categorizationFieldValidationSchema,
+          },
+        },
+      },
+      routeHandlerFactory(license, usageCounter)
+    );
+};

x-pack/plugins/aiops/server/routes/categorization_field_validation/route_handler_factory.ts

@@ -0,0 +1,83 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import type {
+  KibanaRequest,
+  RequestHandlerContext,
+  RequestHandler,
+  KibanaResponseFactory,
+} from '@kbn/core/server';
+import { categorizationExamplesProvider } from '@kbn/ml-category-validator';
+import type { UsageCounter } from '@kbn/usage-collection-plugin/server';
+import { wrapError } from '../error_wrapper';
+import { trackAIOpsRouteUsage } from '../../lib/track_route_usage';
+import { AIOPS_TELEMETRY_ID } from '../../../common/constants';
+import { AIOPS_API_ENDPOINT } from '../../../common/api';
+import type { AiopsLicense } from '../../types';
+import type { CategorizationFieldValidationSchema } from '../../../common/api/log_categorization/schema';
+
+export const routeHandlerFactory: (
+  license: AiopsLicense,
+  usageCounter?: UsageCounter
+) => RequestHandler<unknown, unknown, CategorizationFieldValidationSchema> =
+  (license, usageCounter) =>
+  async (
+    context: RequestHandlerContext,
+    request: KibanaRequest<unknown, unknown, CategorizationFieldValidationSchema>,
+    response: KibanaResponseFactory
+  ) => {
+    const { headers } = request;
+    trackAIOpsRouteUsage(
+      `POST ${AIOPS_API_ENDPOINT.CATEGORIZATION_FIELD_VALIDATION}`,
+      headers[AIOPS_TELEMETRY_ID.AIOPS_ANALYSIS_RUN_ORIGIN],
+      usageCounter
+    );
+
+    if (!license.isActivePlatinumLicense) {
+      return response.forbidden();
+    }
+    try {
+      const {
+        elasticsearch: { client },
+      } = await context.core;
+
+      const {
+        indexPatternTitle,
+        timeField,
+        query,
+        size,
+        field,
+        start,
+        end,
+        analyzer,
+        runtimeMappings,
+        indicesOptions,
+        includeExamples,
+      } = request.body;
+
+      const { validateCategoryExamples } = categorizationExamplesProvider(client);
+      const resp = await validateCategoryExamples(
+        indexPatternTitle,
+        query,
+        size,
+        field,
+        timeField,
+        start,
+        end,
+        analyzer ?? {},
+        runtimeMappings,
+        indicesOptions,
+        includeExamples
+      );
+
+      return response.ok({
+        body: resp,
+      });
+    } catch (e) {
+      return response.customError(wrapError(e));
+    }
+  };