Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RAM] add observability feature for server less #168636

Merged
merged 42 commits into from
Oct 31, 2023
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
933a9ce
adding observability feature for serverless
XavierM Oct 11, 2023
b8b0ba6
fix check types
XavierM Oct 11, 2023
0d823c6
fix jest test
XavierM Oct 11, 2023
0fe3c91
Merge branch 'main' of github.com:elastic/kibana into add-feature-o11…
XavierM Oct 11, 2023
005c3b0
Merge branch 'main' of github.com:elastic/kibana into add-feature-o11…
XavierM Oct 16, 2023
f23cd54
cleanup
XavierM Oct 16, 2023
018eae0
update custom threshold serverless integration tests to have observab…
mgiota Oct 17, 2023
adef1e5
fix tests
XavierM Oct 17, 2023
1b98e94
Merge branch 'main' of github.com:elastic/kibana into add-feature-o11…
XavierM Oct 17, 2023
0bd30ef
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Oct 17, 2023
2a0a00d
add serverless integration tests for burn rate rule
mgiota Oct 19, 2023
72b126c
add integration tests for es query rule
mgiota Oct 23, 2023
57c97de
Merge remote-tracking branch 'upstream/main' into add-feature-o11y-se…
mgiota Oct 23, 2023
106d99c
functional tests for es query rule
mgiota Oct 23, 2023
bd6bcd2
fix alert table for SLO and APM
XavierM Oct 23, 2023
0557b44
Merge branch 'main' of github.com:elastic/kibana into add-feature-o11…
XavierM Oct 23, 2023
9204742
Merge branch 'add-feature-o11y-server-less' of github.com:XavierM/kib…
XavierM Oct 23, 2023
8b05b56
use find api for rules
mgiota Oct 23, 2023
d3e3602
more functional tests for apm latency
mgiota Oct 23, 2023
70c1ae0
Merge branch 'main' of github.com:elastic/kibana into add-feature-o11…
XavierM Oct 25, 2023
74a04ba
Merge branch 'add-feature-o11y-server-less' of github.com:XavierM/kib…
XavierM Oct 25, 2023
dec1a3b
fix filter + esQuery
XavierM Oct 27, 2023
2610e3f
Merge branch 'main' of github.com:elastic/kibana into add-feature-o11…
XavierM Oct 27, 2023
632638c
add o11y
XavierM Oct 27, 2023
14a2fc0
add logic to rule registration
XavierM Oct 27, 2023
b30fabe
miss config initialization
XavierM Oct 27, 2023
78fd248
add tests
XavierM Oct 27, 2023
5e056eb
fix type
XavierM Oct 27, 2023
16d5011
fix types
XavierM Oct 28, 2023
053d25a
clean up
XavierM Oct 30, 2023
f224da5
Merge branch 'main' into add-feature-o11y-server-less
XavierM Oct 30, 2023
db77377
fix oblt serverless config file
XavierM Oct 30, 2023
c72ec2b
Merge branch 'add-feature-o11y-server-less' of github.com:XavierM/kib…
XavierM Oct 30, 2023
88798dd
Merge branch 'main' of github.com:elastic/kibana into add-feature-o11…
XavierM Oct 30, 2023
424dd97
fix scope for serverless
XavierM Oct 30, 2023
cbc6f2f
fix api integration test
XavierM Oct 30, 2023
cfb4077
Merge branch 'main' of github.com:elastic/kibana into add-feature-o11…
XavierM Oct 30, 2023
71aa906
fix api integration serverless
XavierM Oct 31, 2023
0671fe0
Merge branch 'main' of github.com:elastic/kibana into add-feature-o11…
XavierM Oct 31, 2023
1b259ee
fix logic for serverless
XavierM Oct 31, 2023
d62c214
fix serverless test
XavierM Oct 31, 2023
fdce9cb
Merge branch 'main' of github.com:elastic/kibana into add-feature-o11…
XavierM Oct 31, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions config/serverless.oblt.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ xpack.serverless.plugin.developer.projectSwitcher.currentType: 'observability'
## Disable adding the component template `.fleet_agent_id_verification-1` to every index template for each datastream for each integration
xpack.fleet.agentIdVerificationEnabled: false

## Enable the capability for the observability feature ID in the serverless environment to take ownership of the rules.
xpack.observability.createO11yGenericFeatureId: true
xpack.apm.rules.useO11yFeatureIdAsOwner: true
xpack.observability.slo.rules.useO11yFeatureIdAsOwner: true

## APM Serverless Onboarding flow
xpack.apm.serverlessOnboarding: true

Expand Down
7 changes: 7 additions & 0 deletions packages/kbn-rule-data-utils/src/rule_types/o11y_rules.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,10 @@
*/

export const OBSERVABILITY_THRESHOLD_RULE_TYPE_ID = 'observability.rules.custom_threshold';

export enum ApmRuleType {
ErrorCount = 'apm.error_rate', // ErrorRate was renamed to ErrorCount but the key is kept as `error_rate` for backwards-compat.
TransactionErrorRate = 'apm.transaction_error_rate',
TransactionDuration = 'apm.transaction_duration',
Anomaly = 'apm.anomaly',
}
8 changes: 1 addition & 7 deletions x-pack/plugins/apm/common/rules/apm_rule_types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import type { ActionGroup } from '@kbn/alerting-plugin/common';
import { formatDurationFromTimeUnitChar } from '@kbn/observability-plugin/common';
import { ML_ANOMALY_SEVERITY } from '@kbn/ml-anomaly-utils/anomaly_severity';
import { ML_ANOMALY_THRESHOLD } from '@kbn/ml-anomaly-utils/anomaly_threshold';
import { ApmRuleType } from '@kbn/rule-data-utils';
import {
ERROR_GROUP_ID,
ERROR_GROUP_NAME,
Expand All @@ -28,13 +29,6 @@ import { getEnvironmentLabel } from '../environment_filter_values';

export const APM_SERVER_FEATURE_ID = 'apm';

export enum ApmRuleType {
ErrorCount = 'apm.error_rate', // ErrorRate was renamed to ErrorCount but the key is kept as `error_rate` for backwards-compat.
TransactionErrorRate = 'apm.transaction_error_rate',
TransactionDuration = 'apm.transaction_duration',
Anomaly = 'apm.anomaly',
}

export enum AggregationType {
Avg = 'avg',
P95 = '95th',
Expand Down
2 changes: 1 addition & 1 deletion x-pack/plugins/apm/common/rules/get_all_groupby_fields.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
*/

import { union } from 'lodash';
import { ApmRuleType } from './apm_rule_types';
import { ApmRuleType } from '@kbn/rule-data-utils';
import {
SERVICE_ENVIRONMENT,
SERVICE_NAME,
Expand Down
3 changes: 2 additions & 1 deletion x-pack/plugins/apm/common/rules/schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@

import { schema, TypeOf } from '@kbn/config-schema';
import { ML_ANOMALY_SEVERITY } from '@kbn/ml-anomaly-utils/anomaly_severity';
import { AggregationType, ApmRuleType } from './apm_rule_types';
import { ApmRuleType } from '@kbn/rule-data-utils';
import { AggregationType } from './apm_rule_types';

export const searchConfigurationSchema = schema.object({
query: schema.object({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,12 @@

import { i18n } from '@kbn/i18n';
import { lazy } from 'react';
import { ALERT_REASON } from '@kbn/rule-data-utils';
import { ALERT_REASON, ApmRuleType } from '@kbn/rule-data-utils';
import type { ObservabilityRuleTypeRegistry } from '@kbn/observability-plugin/public';
import {
getAlertUrlErrorCount,
getAlertUrlTransaction,
} from '../../../../common/utils/formatters';
import { ApmRuleType } from '../../../../common/rules/apm_rule_types';
import {
anomalyMessage,
errorCountMessage,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,8 @@

import React, { useCallback, useMemo } from 'react';
import { useKibana } from '@kbn/kibana-react-plugin/public';
import {
ApmRuleType,
APM_SERVER_FEATURE_ID,
} from '../../../../../common/rules/apm_rule_types';
import { ApmRuleType } from '@kbn/rule-data-utils';
import { APM_SERVER_FEATURE_ID } from '../../../../../common/rules/apm_rule_types';
import { getInitialAlertValues } from '../../utils/get_initial_alert_values';
import { ApmPluginStartDeps } from '../../../../plugin';
import { useServiceName } from '../../../../hooks/use_service_name';
Expand Down Expand Up @@ -70,6 +68,7 @@ export function AlertingFlyout(props: Props) {
start,
end,
} as AlertMetadata,
useRuleProducer: true,
}),
/* eslint-disable-next-line react-hooks/exhaustive-deps */
[
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,8 @@
*/

import { getInitialAlertValues } from './get_initial_alert_values';
import {
ApmRuleType,
RULE_TYPES_CONFIG,
} from '../../../../common/rules/apm_rule_types';
import { RULE_TYPES_CONFIG } from '../../../../common/rules/apm_rule_types';
import { ApmRuleType } from '@kbn/rule-data-utils';

test('handles null rule type and undefined service name', () => {
expect(getInitialAlertValues(null, undefined)).toEqual({ tags: ['apm'] });
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,8 @@
* 2.0.
*/

import {
ApmRuleType,
RULE_TYPES_CONFIG,
} from '../../../../common/rules/apm_rule_types';
import { ApmRuleType } from '@kbn/rule-data-utils';
import { RULE_TYPES_CONFIG } from '../../../../common/rules/apm_rule_types';

export function getInitialAlertValues(
ruleType: ApmRuleType | null,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ export function AlertsOverview() {
}
id={'service-overview-alerts'}
configurationId={AlertConsumers.OBSERVABILITY}
featureIds={[AlertConsumers.APM]}
featureIds={[AlertConsumers.APM, AlertConsumers.OBSERVABILITY]}
query={esQuery}
showAlertStatusWithFlapping
/>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import {
} from '@elastic/eui';
import { i18n } from '@kbn/i18n';
import React, { useState } from 'react';
import { ApmRuleType } from '../../../../../common/rules/apm_rule_types';
import { ApmRuleType } from '@kbn/rule-data-utils';
import { AlertingFlyout } from '../../../alerting/ui_components/alerting_flyout';
import { useApmPluginContext } from '../../../../context/apm_plugin/use_apm_plugin_context';

Expand Down
6 changes: 2 additions & 4 deletions x-pack/plugins/apm/server/feature.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,8 @@ import {
} from '@kbn/licensing-plugin/server';

import { APM_INDEX_SETTINGS_SAVED_OBJECT_TYPE } from '@kbn/apm-data-access-plugin/server/saved_objects/apm_indices';
import {
ApmRuleType,
APM_SERVER_FEATURE_ID,
} from '../common/rules/apm_rule_types';
import { ApmRuleType } from '@kbn/rule-data-utils';
import { APM_SERVER_FEATURE_ID } from '../common/rules/apm_rule_types';

const ruleTypes = Object.values(ApmRuleType);

Expand Down
3 changes: 3 additions & 0 deletions x-pack/plugins/apm/server/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,9 @@ const configSchema = schema.object({
options: { defaultValue: schema.contextRef('serverless') },
}),
}),
rules: schema.object({
useO11yFeatureIdAsOwner: schema.boolean({ defaultValue: false }),
}),
});

// plugin config
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import datemath from '@kbn/datemath';
import type { ESSearchResponse } from '@kbn/es-types';
import {
getAlertUrl,
observabilityFeatureId,
observabilityPaths,
ProcessorEvent,
} from '@kbn/observability-plugin/common';
Expand All @@ -21,6 +22,7 @@ import {
ALERT_EVALUATION_VALUE,
ALERT_REASON,
ALERT_SEVERITY,
ApmRuleType,
} from '@kbn/rule-data-utils';
import { createLifecycleRuleTypeFactory } from '@kbn/rule-registry-plugin/server';
import { addSpaceIdToPath } from '@kbn/spaces-plugin/common';
Expand All @@ -43,7 +45,7 @@ import {
} from '../../../../../common/environment_filter_values';
import {
ANOMALY_ALERT_SEVERITY_TYPES,
ApmRuleType,
APM_SERVER_FEATURE_ID,
formatAnomalyReason,
RULE_TYPES_CONFIG,
} from '../../../../../common/rules/apm_rule_types';
Expand All @@ -63,6 +65,7 @@ const ruleTypeConfig = RULE_TYPES_CONFIG[ApmRuleType.Anomaly];
export function registerAnomalyRuleType({
alerting,
alertsLocator,
apmConfig,
getApmIndices,
basePath,
logger,
Expand Down Expand Up @@ -94,7 +97,9 @@ export function registerAnomalyRuleType({
],
},
category: DEFAULT_APP_CATEGORIES.observability.id,
producer: 'apm',
producer: apmConfig.rules.useO11yFeatureIdAsOwner
? observabilityFeatureId
: APM_SERVER_FEATURE_ID,
minimumLicenseRequired: 'basic',
isExportable: true,
executor: async ({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import {
termQuery,
} from '@kbn/observability-plugin/server';
import { ProcessorEvent } from '@kbn/observability-plugin/common';
import { ApmRuleType } from '@kbn/rule-data-utils';
import {
ERROR_GROUP_ID,
PROCESSOR_EVENT,
Expand All @@ -21,7 +22,6 @@ import { environmentQuery } from '../../../../../common/utils/environment_query'
import { APMEventClient } from '../../../../lib/helpers/create_es_client/create_apm_event_client';
import { getGroupByTerms } from '../utils/get_groupby_terms';
import { getAllGroupByFields } from '../../../../../common/rules/get_all_groupby_fields';
import { ApmRuleType } from '../../../../../common/rules/apm_rule_types';
import {
BarSeriesDataMap,
getFilteredBarSeries,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { GetViewInAppRelativeUrlFnOpts } from '@kbn/alerting-plugin/server';
import {
formatDurationFromTimeUnitChar,
getAlertUrl,
observabilityFeatureId,
observabilityPaths,
ProcessorEvent,
TimeUnitChar,
Expand All @@ -18,6 +19,7 @@ import {
ALERT_EVALUATION_THRESHOLD,
ALERT_EVALUATION_VALUE,
ALERT_REASON,
ApmRuleType,
} from '@kbn/rule-data-utils';
import { createLifecycleRuleTypeFactory } from '@kbn/rule-registry-plugin/server';
import {
Expand All @@ -34,7 +36,6 @@ import {
SERVICE_NAME,
} from '../../../../../common/es_fields/apm';
import {
ApmRuleType,
APM_SERVER_FEATURE_ID,
formatErrorCountReason,
RULE_TYPES_CONFIG,
Expand Down Expand Up @@ -75,6 +76,7 @@ export const errorCountActionVariables = [
export function registerErrorCountRuleType({
alerting,
alertsLocator,
apmConfig,
basePath,
getApmIndices,
logger,
Expand All @@ -96,7 +98,9 @@ export function registerErrorCountRuleType({
context: errorCountActionVariables,
},
category: DEFAULT_APP_CATEGORIES.observability.id,
producer: APM_SERVER_FEATURE_ID,
producer: apmConfig.rules.useO11yFeatureIdAsOwner
? observabilityFeatureId
: APM_SERVER_FEATURE_ID,
minimumLicenseRequired: 'basic',
isExportable: true,
executor: async ({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,8 @@ import {
rangeQuery,
termQuery,
} from '@kbn/observability-plugin/server';
import {
AggregationType,
ApmRuleType,
} from '../../../../../common/rules/apm_rule_types';
import { ApmRuleType } from '@kbn/rule-data-utils';
import { AggregationType } from '../../../../../common/rules/apm_rule_types';
import {
SERVICE_NAME,
TRANSACTION_TYPE,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import {
asDuration,
formatDurationFromTimeUnitChar,
getAlertDetailsUrl,
observabilityFeatureId,
observabilityPaths,
ProcessorEvent,
TimeUnitChar,
Expand All @@ -24,6 +25,7 @@ import {
ALERT_EVALUATION_THRESHOLD,
ALERT_EVALUATION_VALUE,
ALERT_REASON,
ApmRuleType,
} from '@kbn/rule-data-utils';
import { createLifecycleRuleTypeFactory } from '@kbn/rule-registry-plugin/server';
import { addSpaceIdToPath } from '@kbn/spaces-plugin/common';
Expand All @@ -38,7 +40,6 @@ import {
TRANSACTION_TYPE,
} from '../../../../../common/es_fields/apm';
import {
ApmRuleType,
APM_SERVER_FEATURE_ID,
formatTransactionDurationReason,
RULE_TYPES_CONFIG,
Expand Down Expand Up @@ -108,7 +109,9 @@ export function registerTransactionDurationRuleType({
context: transactionDurationActionVariables,
},
category: DEFAULT_APP_CATEGORIES.observability.id,
producer: APM_SERVER_FEATURE_ID,
producer: apmConfig.rules.useO11yFeatureIdAsOwner
? observabilityFeatureId
: APM_SERVER_FEATURE_ID,
minimumLicenseRequired: 'basic',
isExportable: true,
executor: async ({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import {
rangeQuery,
termQuery,
} from '@kbn/observability-plugin/server';
import { ApmRuleType } from '../../../../../common/rules/apm_rule_types';
import { ApmRuleType } from '@kbn/rule-data-utils';
import {
SERVICE_NAME,
TRANSACTION_TYPE,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { GetViewInAppRelativeUrlFnOpts } from '@kbn/alerting-plugin/server';
import {
formatDurationFromTimeUnitChar,
getAlertUrl,
observabilityFeatureId,
observabilityPaths,
ProcessorEvent,
TimeUnitChar,
Expand All @@ -23,6 +24,7 @@ import {
ALERT_EVALUATION_THRESHOLD,
ALERT_EVALUATION_VALUE,
ALERT_REASON,
ApmRuleType,
} from '@kbn/rule-data-utils';
import { createLifecycleRuleTypeFactory } from '@kbn/rule-registry-plugin/server';
import { addSpaceIdToPath } from '@kbn/spaces-plugin/common';
Expand All @@ -39,7 +41,6 @@ import {
} from '../../../../../common/es_fields/apm';
import { EventOutcome } from '../../../../../common/event_outcome';
import {
ApmRuleType,
APM_SERVER_FEATURE_ID,
formatTransactionErrorRateReason,
RULE_TYPES_CONFIG,
Expand Down Expand Up @@ -105,7 +106,9 @@ export function registerTransactionErrorRateRuleType({
context: transactionErrorRateActionVariables,
},
category: DEFAULT_APP_CATEGORIES.observability.id,
producer: APM_SERVER_FEATURE_ID,
producer: apmConfig.rules.useO11yFeatureIdAsOwner
? observabilityFeatureId
: APM_SERVER_FEATURE_ID,
minimumLicenseRequired: 'basic',
isExportable: true,
executor: async ({
Expand Down
5 changes: 4 additions & 1 deletion x-pack/plugins/apm/server/routes/alerts/test_utils/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,10 @@ export const createRuleTypeMocks = () => {
publicBaseUrl: 'http://localhost:5601/eyr',
serverBasePath: '/eyr',
} as IBasePath,
apmConfig: { searchAggregatedTransactions: true } as any as APMConfig,
apmConfig: {
searchAggregatedTransactions: true,
rules: { useO11yFeatureIdAsOwner: false },
} as any as APMConfig,
getApmIndices: async () => ({
error: 'apm-*',
transaction: 'apm-*',
Expand Down
1 change: 1 addition & 0 deletions x-pack/plugins/observability/common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,4 +60,5 @@ export const observabilityAlertFeatureIds: ValidFeatureId[] = [
export const observabilityRuleCreationValidConsumers: RuleCreationValidConsumer[] = [
AlertConsumers.INFRASTRUCTURE,
AlertConsumers.LOGS,
AlertConsumers.OBSERVABILITY,
];
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ export function AlertFlyout(props: Props) {
series: props.series,
},
validConsumers: observabilityRuleCreationValidConsumers,
useRuleProducer: true,
}),
// eslint-disable-next-line react-hooks/exhaustive-deps
[triggersActionsUI, onCloseFlyout]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,7 @@ export function HeaderControl({ isLoading, slo }: Props) {
canChangeTrigger={false}
onClose={onCloseRuleFlyout}
initialValues={{ name: `${slo.name} burn rate`, params: { sloId: slo.id } }}
useRuleProducer
/>
) : null}

Expand Down
Loading