[Response Ops][RAM] Add RBAC Support for Multi-Consumer Rule Creation in O11Y and Stack Management #162605

JiaweiWu · 2023-07-27T00:24:23Z

Summary

Resolves:

Change alerting RBAC logic to only consider consumer #162484
[Response Ops][Actionable Observability] Creating a rule via Observability > Alerts > Manage Rules sets consumer to alerts #160677

This PR adds the ability for logs and/or infrastructure only users to create and modify ES Query and new Generic Threshold rules. The ensureAuthorized function is modified and simplified to support this use case, by skipping producer authorization and only authorizing for consumers. When the consumer is alerts, we will consider this legacy and replace it with the rule’s producer (consumer = ruleType.producer)

There is now a dropdown in the rule form to prompt the user when creating ES Query/Generic threshold rules to select the consumer based on their authorized consumers (we can no longer use alerts for these). If there is only 1 option, then the dropdown will not be shown and the option will be chosen automatically.

Generic threshold rules will have the following possible consumers:

slo
infrastructure
logs
apm
uptime

ES query rules will have the following possible consumers:

slo
infrastructure
logs
apm
uptime
stackAlerts

To Test:

Single Consumer:

Create a user with only logs feature enabled (ensuring stackAlerts is not enabled).
Navigate to the O11Y rule management page
Click the create rule button
Assert that both ES query and generic threshold rules are available
Click ES query and fill out the relevant information and create the rule
Assert that the rule created has logs set in the consumer field
Repeat 5-6 for the generic threshold rule
Repeat 2-7 but on the Stack Management rules page
Repeat 1-8 for the infrastructure feature.

Multiple Consumers:

Create a user with logs, infrastructure and apm features enabled (ensuring stackAlerts is not enabled).
Navigate to the O11Y rule management page
Click the create rule button
Assert that both ES query and generic threshold rules are available
Click ES query and fill out the relevant information and create the rule
A dropdown should prompt the user to select between 1 of the 3 consumers, select 1
Assert that the rule was created with the selected consumer
Repeat 5-7 for the generic threshold rule
Repeat 2-8 but on the Stack Management rules page

Checklist

Unit or functional tests were updated or added to match the most common scenarios

mdefazio · 2023-07-27T11:31:01Z

Nice one! Some thoughts on the modal copy:

Select rule association
This rule needs to be associated with a particular application for proper role access visibility.
[ Save ]

mikecote · 2023-07-27T14:46:05Z

x-pack/plugins/alerting/server/authorization/alerting_authorization.ts

-        throw Boom.forbidden(
-          getUnauthorizedMessage(ruleTypeId, ScopeType.Consumer, consumer, operation, entity)
-        );
+        throw Boom.forbidden(getUnauthorizedMessage(ruleTypeId, consumer, operation, entity));


By looking at the tests failures, if you feel the error messages are now misleading, you could change all the getUnauthorizedMessage calls to something like this (use legacyConsumer):

Suggested change

throw Boom.forbidden(getUnauthorizedMessage(ruleTypeId, consumer, operation, entity));

throw Boom.forbidden(getUnauthorizedMessage(ruleTypeId, legacyConsumer, operation, entity));

…-fix'

elasticmachine · 2023-08-03T20:03:04Z

Pinging @elastic/response-ops (Team:ResponseOps)

JiaweiWu · 2023-08-09T00:08:45Z

x-pack/test/api_integration/apis/maps/maps_telemetry.ts

@@ -30,8 +30,8 @@ export default function ({ getService }: FtrProviderContext) {
          return fieldStat.name === 'geo_point';
        }
      );
-      expect(geoPointFieldStats.count).to.be(31);
-      expect(geoPointFieldStats.index_count).to.be(9);
+      expect(geoPointFieldStats.count).to.be(39);


This value had to be incremented when we enabled the new generic threshold rules: https://github.com/elastic/kibana/pull/162605/files#diff-53c98e8d9e2d61d5c9f37b003fc6de5c3f84570f7886cf4bdd89f596b32a15c9R49

nreese

kibana-gis changes LGTM
code review only

elasticmachine · 2023-08-09T21:02:37Z

Pinging @elastic/apm-ui (Team:APM)

elasticmachine · 2023-08-09T21:02:37Z

Pinging @elastic/uptime (Team:uptime)

…fix'

mikecote

Changes LGTM! I left a few nitpicks but overall the changes are looking great!

x-pack/test/alerting_api_integration/security_and_spaces/group3/tests/alerting/bulk_enable.ts

x-pack/test/alerting_api_integration/security_and_spaces/group3/tests/alerting/bulk_disable.ts

x-pack/plugins/observability/common/constants.ts

mikecote · 2023-08-09T15:45:02Z

x-pack/plugins/observability/server/index.ts

-      enabled: schema.boolean({ defaultValue: false }),
+      enabled: schema.boolean({ defaultValue: true }),


Unless O11y is ready, we may want to reverse this change in the PR and let O11y change the default when they are ready.(unless we can add it to a todo list on the feature branch PR).

If we revert this, the following changes will have to be undone as well:

x-pack/test/plugin_api_integration/test_suites/task_manager/check_registered_task_types.ts

x-pack/test/api_integration/apis/maps/maps_telemetry.ts

I had this code in my POC to bypass having to set the kibana.yml settings for testing.

Let's add it to a to-do list, since we have tests other than maps_telemetry that depend on this change. Also, this is merging to a feature branch so won't merge to main presumably until ES types are enabled in o11y (correct me if im wrong)

@maryam-saeidi or @fkanout are best positioned to say whether this can be set to true already or not.

We decided to not enable it for v8.10 and aim to either enable this in v8.11 or remove it and release the feature in beta.

mikecote · 2023-08-15T12:43:51Z

x-pack/plugins/triggers_actions_ui/public/types.ts

+  | typeof AlertConsumers.UPTIME
+  | typeof AlertConsumers.SLO
+  | 'stackAlerts'
+  | 'discover';


From an offline discussion last week, we'll be moving away from discover as a consumer, in favour of stackAlerts.

We'll need to do one of two things:

Remove this usage if no underlying issues exist

Add it to a list of tasks to complete before the feature branch is merged (if we have a draft PR on the feature branch, we can create a checklist list in the description)

mikecote · 2023-08-15T12:44:38Z

x-pack/plugins/triggers_actions_ui/public/application/constants/index.ts

@@ -15,6 +15,9 @@ export { BASE_ACTION_API_PATH, INTERNAL_BASE_ACTION_API_PATH } from '@kbn/action

 export type Section = 'connectors' | 'rules' | 'alerts' | 'logs';

+export const OBSERVABILITY_THRESHOLD_RULE_TYPE_ID = 'observability.rules.threshold';
+export const ES_QUERY_RULE_TYPE_ID = '.es-query';


nit: can we re-use ES_QUERY_ID from x-pack/plugins/stack_alerts/server/rule_types/es_query/constants.ts, we'll have to move it into a common folder.

Just tried to do this, unfortunately, stack alerts has triggers actions UI as a dependency. If I import from stack alerts in triggers actions UI we get a circular dependency.

If we want to share this constantly then we probably move it to a package.

CoenWarmer · 2023-08-16T21:14:49Z

x-pack/plugins/observability/public/pages/alerts/alerts.tsx

@@ -123,7 +126,7 @@ function InternalAlertsPage() {
    try {
      const response = await loadRuleAggregations({
        http,
-        typesFilter: observabilityRuleTypeRegistry.list(),
+        typesFilter: filteredRuleTypes,


Thank you, way nicer.

x-pack/plugins/observability/server/plugin.ts

…fix'

kibana-ci · 2023-08-16T23:57:21Z

💔 Build Failed

Failed CI Steps

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`triggersActionsUi`	576	577	+1

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`stackAlerts`	14	15	+1
`triggersActionsUi`	545	546	+1
total			+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`observability`	1023.8KB	1023.8KB	+16.0B
`stackAlerts`	77.2KB	76.0KB	-1.1KB
`triggersActionsUi`	1.4MB	1.5MB	+2.8KB
total			+1.7KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`observability`	99.3KB	99.6KB	+273.0B
`stackAlerts`	19.4KB	21.0KB	+1.6KB
`triggersActionsUi`	89.0KB	89.1KB	+82.0B
total			+2.0KB

Unknown metric groups

API count

id	before	after	diff
`stackAlerts`	14	15	+1
`triggersActionsUi`	571	572	+1
total			+2

ESLint disabled line counts

id	before	after	diff
`triggersActionsUi`	120	122	+2

Total ESLint disabled count

id	before	after	diff
`triggersActionsUi`	126	128	+2

History

💔 Build #151240 failed 3bbeb44
💔 Build #151218 failed 0dc5599
💚 Build #149040 succeeded b3b7093
💔 Build #149039 failed 28ab550

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

mdefazio · 2023-08-28T14:14:22Z

A small request on this (I know this is merged and we are close to FF—sorry).

Can we set the defaults of the dropdown to the following?:
If from Stack rule management --> dropdown defaults to Stack alerts
If from O11y rule management --> dropdown defaults to Logs

Rationale:
There are scenarios where customers are creating rules in Stack management vs Observability for purposes of locking down the rule from those without access to Stack Management / only access to O11y. Our concern is if this defaults to an Observability scope, then there's a chance the user may not notice that the current behavior has changed. (Creating a rule in stack and having it scoped to stack).

And, our expectation is that Logs will be the most common use case for this, so it makes sense to default to that.

@katrin-freihofner Can you confirm this request please?

Curious if there are any differing arguments to this as well.

cc/ @JiaweiWu @XavierM @katrin-freihofner

JiaweiWu and others added 2 commits July 26, 2023 17:23

Initial commit

44bb149

Merge branch 'o11y-rbac-rule-feature-branch' into rbac-fixes

6823a67

mikecote reviewed Jul 27, 2023

View reviewed changes

JiaweiWu and others added 14 commits July 27, 2023 16:31

Consolidate alert authorization error message and fix some e2e tests

7450abc

Fix tests

ab9afa6

Fix tests

aadad7c

Fix unit tests

9e1887a

Fix serverless tests

679f1b5

More test fixes

40d28fe

Write unit tests and fix serverless tests

09ae685

fix rule flyout logic

a763eac

fix types

71f6160

Fix some bugs

935456e

O11Y E2E tests

4ef3118

Fix merge conflict

4fc4040

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

527f980

…-fix'

Fix tests and fix bug involving discover

7a5883e

JiaweiWu changed the title ~~[RAM] O11y RBAC consumer selection modal and auth changes~~ [RAM] Add RBAC Support for Multi-Consumer Rules in O11Y and Stack Management Aug 3, 2023

Use better constants

67bb1a9

JiaweiWu changed the title ~~[RAM] Add RBAC Support for Multi-Consumer Rules in O11Y and Stack Management~~ [RAM] Add RBAC Support for Multi-Consumer Rule Creation in O11Y and Stack Management Aug 3, 2023

JiaweiWu marked this pull request as ready for review August 3, 2023 20:02

JiaweiWu requested review from a team as code owners August 3, 2023 20:02

JiaweiWu added release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v8.10.0 labels Aug 3, 2023

JiaweiWu requested a review from a team as a code owner August 9, 2023 00:06

JiaweiWu commented Aug 9, 2023

View reviewed changes

Merge branch 'main' into rbac-fixes

d78d108

nreese approved these changes Aug 9, 2023

View reviewed changes

JiaweiWu added 3 commits August 9, 2023 12:09

Merge branch 'main' into rbac-fixes

01ecdc8

Fix e2e tests

5cadacd

Address comments

ed1deb0

JiaweiWu requested review from a team as code owners August 9, 2023 21:02

botelastic bot added Team:APM All issues that need APM UI Team support Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability labels Aug 9, 2023

kibanamachine and others added 3 commits August 9, 2023 21:09

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

a2f07fa

…fix'

Fix types

28ab550

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

b3b7093

…fix'

mikecote approved these changes Aug 15, 2023

View reviewed changes

CoenWarmer reviewed Aug 16, 2023

View reviewed changes

x-pack/plugins/observability/server/plugin.ts Outdated Show resolved Hide resolved

JiaweiWu and others added 6 commits August 16, 2023 14:29

Address comments

e350f14

Rename rule types variable

0a5db9f

Fix merge conflict

0dc9c2a

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

0dc5599

…fix'

Fix types

3bbeb44

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

7be9ddc

…fix'

JiaweiWu merged commit 47ea286 into elastic:o11y-rbac-rule-feature-branch Aug 16, 2023

mdefazio mentioned this pull request Aug 25, 2023

[Response Ops][Actionable Observability] Creating a rule via Observability > Alerts > Manage Rules sets consumer to alerts #160677

Closed

lcawl mentioned this pull request Oct 30, 2023

Add custom threshold rule elastic/observability-docs#3314

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Response Ops][RAM] Add RBAC Support for Multi-Consumer Rule Creation in O11Y and Stack Management #162605

[Response Ops][RAM] Add RBAC Support for Multi-Consumer Rule Creation in O11Y and Stack Management #162605

JiaweiWu commented Jul 27, 2023 •

edited

Loading

mdefazio commented Jul 27, 2023

mikecote Jul 27, 2023 •

edited

Loading

elasticmachine commented Aug 3, 2023

JiaweiWu Aug 9, 2023

nreese left a comment

elasticmachine commented Aug 9, 2023

elasticmachine commented Aug 9, 2023

mikecote left a comment

mikecote Aug 9, 2023

JiaweiWu Aug 16, 2023 •

edited

Loading

CoenWarmer Aug 16, 2023

maryam-saeidi Aug 17, 2023

mikecote Aug 15, 2023

mikecote Aug 15, 2023

JiaweiWu Aug 16, 2023 •

edited

Loading

CoenWarmer Aug 16, 2023

kibana-ci commented Aug 16, 2023 •

edited

Loading

API count

ESLint disabled line counts

Total ESLint disabled count

mdefazio commented Aug 28, 2023

	throw Boom.forbidden(getUnauthorizedMessage(ruleTypeId, consumer, operation, entity));
	throw Boom.forbidden(getUnauthorizedMessage(ruleTypeId, legacyConsumer, operation, entity));

		enabled: schema.boolean({ defaultValue: false }),
		enabled: schema.boolean({ defaultValue: true }),

[Response Ops][RAM] Add RBAC Support for Multi-Consumer Rule Creation in O11Y and Stack Management #162605

[Response Ops][RAM] Add RBAC Support for Multi-Consumer Rule Creation in O11Y and Stack Management #162605

Conversation

JiaweiWu commented Jul 27, 2023 • edited Loading

Summary

To Test:

Single Consumer:

Multiple Consumers:

Checklist

mdefazio commented Jul 27, 2023

mikecote Jul 27, 2023 • edited Loading

Choose a reason for hiding this comment

elasticmachine commented Aug 3, 2023

JiaweiWu Aug 9, 2023

Choose a reason for hiding this comment

nreese left a comment

Choose a reason for hiding this comment

elasticmachine commented Aug 9, 2023

elasticmachine commented Aug 9, 2023

mikecote left a comment

Choose a reason for hiding this comment

mikecote Aug 9, 2023

Choose a reason for hiding this comment

JiaweiWu Aug 16, 2023 • edited Loading

Choose a reason for hiding this comment

CoenWarmer Aug 16, 2023

Choose a reason for hiding this comment

maryam-saeidi Aug 17, 2023

Choose a reason for hiding this comment

mikecote Aug 15, 2023

Choose a reason for hiding this comment

mikecote Aug 15, 2023

Choose a reason for hiding this comment

JiaweiWu Aug 16, 2023 • edited Loading

Choose a reason for hiding this comment

CoenWarmer Aug 16, 2023

Choose a reason for hiding this comment

kibana-ci commented Aug 16, 2023 • edited Loading

💔 Build Failed

Failed CI Steps

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Page load bundle

API count

ESLint disabled line counts

Total ESLint disabled count

History

mdefazio commented Aug 28, 2023

JiaweiWu commented Jul 27, 2023 •

edited

Loading

mikecote Jul 27, 2023 •

edited

Loading

JiaweiWu Aug 16, 2023 •

edited

Loading

JiaweiWu Aug 16, 2023 •

edited

Loading

kibana-ci commented Aug 16, 2023 •

edited

Loading