Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Security Solution][Detections] SIEM job fetch failure error toast shown for users without ML permissions #73567

Closed
spong opened this issue Jul 28, 2020 · 4 comments
Closed

[Security Solution][Detections] SIEM job fetch failure error toast shown for users without ML permissions #73567

spong opened this issue Jul 28, 2020 · 4 comments
Assignees
@rylnd
Labels
bug Fixes for quality problems that affect the customer experience Feature:Detection Rules Security Solution rules and Detection Engine Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.9.1 v7.10.0

Comments

@spong
Copy link
Member

spong commented Jul 28, 2020

This was uncovered in updating demo.elastic.co from 7.7 -> 7.8, and was introduced in 7.8.

When navigating to any Detections page that queries for SIEM Jobs, the below SIEM job fetch failure error toast will be displayed if the user does not have Full ML License + access:ml:canCreateJob permissions.

This will happen for any use of the useSiemJobs hook, as the ml/modules/recognize API had its permissions tightened as part of #64662 for 7.8. Since the MlPopover component (which uses the useSiemJobs hook) is not wrapped in a ml permissions check, but rather its content is, any usage of this component will result in the below error for users without permissions.

There is no workaround for this issue, and to fix we will have to wrap the useSiemJobs hook in the same permissions check we use for the other ML API's.

Additionally, this use-case should be back-filled with tests to ensure future changes to API permissions do not result in similar issues.
@spong spong added bug Fixes for quality problems that affect the customer experience Team:SIEM v7.10.0 Feature:Detection Rules Security Solution rules and Detection Engine v7.9.1 labels Jul 28, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)

@rylnd rylnd self-assigned this Aug 4, 2020
@rylnd rylnd mentioned this issue Aug 6, 2020
Merged
2 tasks
spong pushed a commit that referenced this issue Aug 13, 2020
@rylnd
[Security Solution][Detections] Refactor ML calls for newest ML permi…
4ca52e6 
…ssions (#74582)

## Summary

Addresses #73567.

ML Users (role: `machine_learning_user`) were previously able to invoke the ML Recognizer API, which we use to get not-yet-installed ML Jobs relevant to our index patterns. As of #64662 this is not true, and so we receive errors from components using the underlying hook, `useSiemJobs`.

To solve this I've created two separate hooks to replace `useSiemJobs`:

* `useSecurityJobs`
  * used on ML Popover
  * includes uninstalled ML Jobs
  * checks (and returns) `isMlAdmin` before fetching data
* `useInstalledSecurityJobs`
  * used on ML Jobs Dropdown and Anomalies Table
  * includes only installed ML Jobs
  * checks (and returns) `isMlUser` before fetching data

Note that we while we now receive the knowledge to do so, we do not always inform the user in the case of invalid permissions, and instead have the following behaviors:

#### User has insufficient license
* ML Popover:  shows an upgrade CTA
* Anomalies Tables: show no data
* Rule Creation: ML Rule option is disabled, shows upgrade CTA
* Rule Details: ML Job Id is displayed as text
#### User is ML User
* ML Popover:  not shown
* Anomalies Tables: show no data
* Rule Creation: ML Rule option is disabled
* Rule Details: ML Job Id is displayed as text
#### User is ML Admin
* ML Popover:  shown
* Anomalies Tables: show data __for installed ML Jobs__
  * This is the same as previous logic, but worth calling out that you can't view historical anomalies
* Rule Creation: ML Rule option is enabled, all ML Jobs available
* Rule Details: ML Job Id is displayed as hyperlink, job status badge shown

### Checklist

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
spong pushed a commit to spong/kibana that referenced this issue Aug 13, 2020
@rylnd @spong
[Security Solution][Detections] Refactor ML calls for newest ML permi…
255b61d 
…ssions (elastic#74582)

## Summary

Addresses elastic#73567.

ML Users (role: `machine_learning_user`) were previously able to invoke the ML Recognizer API, which we use to get not-yet-installed ML Jobs relevant to our index patterns. As of elastic#64662 this is not true, and so we receive errors from components using the underlying hook, `useSiemJobs`.

To solve this I've created two separate hooks to replace `useSiemJobs`:

* `useSecurityJobs`
  * used on ML Popover
  * includes uninstalled ML Jobs
  * checks (and returns) `isMlAdmin` before fetching data
* `useInstalledSecurityJobs`
  * used on ML Jobs Dropdown and Anomalies Table
  * includes only installed ML Jobs
  * checks (and returns) `isMlUser` before fetching data

Note that we while we now receive the knowledge to do so, we do not always inform the user in the case of invalid permissions, and instead have the following behaviors:

#### User has insufficient license
* ML Popover:  shows an upgrade CTA
* Anomalies Tables: show no data
* Rule Creation: ML Rule option is disabled, shows upgrade CTA
* Rule Details: ML Job Id is displayed as text
#### User is ML User
* ML Popover:  not shown
* Anomalies Tables: show no data
* Rule Creation: ML Rule option is disabled
* Rule Details: ML Job Id is displayed as text
#### User is ML Admin
* ML Popover:  shown
* Anomalies Tables: show data __for installed ML Jobs__
  * This is the same as previous logic, but worth calling out that you can't view historical anomalies
* Rule Creation: ML Rule option is enabled, all ML Jobs available
* Rule Details: ML Job Id is displayed as hyperlink, job status badge shown

### Checklist

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
spong added a commit that referenced this issue Aug 13, 2020
@spong @rylnd
[Security Solution][Detections] Refactor ML calls for newest ML permi…
8991c4f 
…ssions (#74582) (#74919)

## Summary

Addresses #73567.

ML Users (role: `machine_learning_user`) were previously able to invoke the ML Recognizer API, which we use to get not-yet-installed ML Jobs relevant to our index patterns. As of #64662 this is not true, and so we receive errors from components using the underlying hook, `useSiemJobs`.

To solve this I've created two separate hooks to replace `useSiemJobs`:

* `useSecurityJobs`
  * used on ML Popover
  * includes uninstalled ML Jobs
  * checks (and returns) `isMlAdmin` before fetching data
* `useInstalledSecurityJobs`
  * used on ML Jobs Dropdown and Anomalies Table
  * includes only installed ML Jobs
  * checks (and returns) `isMlUser` before fetching data

Note that we while we now receive the knowledge to do so, we do not always inform the user in the case of invalid permissions, and instead have the following behaviors:

#### User has insufficient license
* ML Popover:  shows an upgrade CTA
* Anomalies Tables: show no data
* Rule Creation: ML Rule option is disabled, shows upgrade CTA
* Rule Details: ML Job Id is displayed as text
#### User is ML User
* ML Popover:  not shown
* Anomalies Tables: show no data
* Rule Creation: ML Rule option is disabled
* Rule Details: ML Job Id is displayed as text
#### User is ML Admin
* ML Popover:  shown
* Anomalies Tables: show data __for installed ML Jobs__
  * This is the same as previous logic, but worth calling out that you can't view historical anomalies
* Rule Creation: ML Rule option is enabled, all ML Jobs available
* Rule Details: ML Job Id is displayed as hyperlink, job status badge shown

### Checklist

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)

Co-authored-by: Ryland Herrick <[email protected]>
@spong
Copy link
Member Author

spong commented Aug 13, 2020

Resolved by #74582 -- TBD on backporting to 7.9.1

@spong spong closed this as completed Aug 13, 2020
spong pushed a commit to spong/kibana that referenced this issue Aug 18, 2020
@rylnd @spong
[Security Solution][Detections] Refactor ML calls for newest ML permi…
dc5e9ea 
…ssions (elastic#74582)

## Summary

Addresses elastic#73567.

ML Users (role: `machine_learning_user`) were previously able to invoke the ML Recognizer API, which we use to get not-yet-installed ML Jobs relevant to our index patterns. As of elastic#64662 this is not true, and so we receive errors from components using the underlying hook, `useSiemJobs`.

To solve this I've created two separate hooks to replace `useSiemJobs`:

* `useSecurityJobs`
  * used on ML Popover
  * includes uninstalled ML Jobs
  * checks (and returns) `isMlAdmin` before fetching data
* `useInstalledSecurityJobs`
  * used on ML Jobs Dropdown and Anomalies Table
  * includes only installed ML Jobs
  * checks (and returns) `isMlUser` before fetching data

Note that we while we now receive the knowledge to do so, we do not always inform the user in the case of invalid permissions, and instead have the following behaviors:

#### User has insufficient license
* ML Popover:  shows an upgrade CTA
* Anomalies Tables: show no data
* Rule Creation: ML Rule option is disabled, shows upgrade CTA
* Rule Details: ML Job Id is displayed as text
#### User is ML User
* ML Popover:  not shown
* Anomalies Tables: show no data
* Rule Creation: ML Rule option is disabled
* Rule Details: ML Job Id is displayed as text
#### User is ML Admin
* ML Popover:  shown
* Anomalies Tables: show data __for installed ML Jobs__
  * This is the same as previous logic, but worth calling out that you can't view historical anomalies
* Rule Creation: ML Rule option is enabled, all ML Jobs available
* Rule Details: ML Job Id is displayed as hyperlink, job status badge shown

### Checklist

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
spong added a commit that referenced this issue Aug 18, 2020
@spong @rylnd
[Security Solution][Detections] Refactor ML calls for newest ML permi…
57b304a 
…ssions (#74582) (#75287)

## Summary

Addresses #73567.

ML Users (role: `machine_learning_user`) were previously able to invoke the ML Recognizer API, which we use to get not-yet-installed ML Jobs relevant to our index patterns. As of #64662 this is not true, and so we receive errors from components using the underlying hook, `useSiemJobs`.

To solve this I've created two separate hooks to replace `useSiemJobs`:

* `useSecurityJobs`
  * used on ML Popover
  * includes uninstalled ML Jobs
  * checks (and returns) `isMlAdmin` before fetching data
* `useInstalledSecurityJobs`
  * used on ML Jobs Dropdown and Anomalies Table
  * includes only installed ML Jobs
  * checks (and returns) `isMlUser` before fetching data

Note that we while we now receive the knowledge to do so, we do not always inform the user in the case of invalid permissions, and instead have the following behaviors:

#### User has insufficient license
* ML Popover:  shows an upgrade CTA
* Anomalies Tables: show no data
* Rule Creation: ML Rule option is disabled, shows upgrade CTA
* Rule Details: ML Job Id is displayed as text
#### User is ML User
* ML Popover:  not shown
* Anomalies Tables: show no data
* Rule Creation: ML Rule option is disabled
* Rule Details: ML Job Id is displayed as text
#### User is ML Admin
* ML Popover:  shown
* Anomalies Tables: show data __for installed ML Jobs__
  * This is the same as previous logic, but worth calling out that you can't view historical anomalies
* Rule Creation: ML Rule option is enabled, all ML Jobs available
* Rule Details: ML Job Id is displayed as hyperlink, job status badge shown

### Checklist

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)

Co-authored-by: Ryland Herrick <[email protected]>
@spong
Copy link
Member Author

spong commented Aug 18, 2020

This fix will be available in 7.9.1 -- #75287

@MindyRS MindyRS added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Oct 27, 2020
@ghost
Copy link

ghost commented Nov 25, 2020

Bug Conversion:

This ticket requires Dev validation

Thanks!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rylnd rylnd

Labels
bug Fixes for quality problems that affect the customer experience Feature:Detection Rules Security Solution rules and Detection Engine Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.9.1 v7.10.0
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@rylnd @MindyRS @spong @elasticmachine