[Rules migration][UI] Basic rule migrations UI (#10820) #200978

e40pud · 2024-11-20T16:08:54Z

Summary

Internal link to the feature details

This is a very first version of the SIEM rules migrations UI functionality. The main goal is to setup and agree on a folder structure where the feature gonna live. Tests covering feature will follow in a separate PR (see internal link for more details).

The code follows the structure of prebuilt rules feature https://github.com/elastic/kibana/tree/main/x-pack/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/add_prebuilt_rules_table and hidden behind siemMigrationsEnabled feature flag.

Key UI changes

New "SIEM Rules Migrations." rules management sub-page
Navigation between different "finished" migrations
InMemory table with all the translations within the selected migration
Translation details preview flyout with Translation and Overview tabs
User cannot modify translations via UI

Testing locally

Enable the flag

xpack.securitySolution.enableExperimental: ['siemMigrationsEnabled']

Screenshot

Screen.Recording.2024-11-20.at.17.06.50.mov

elasticmachine · 2024-11-20T16:08:58Z

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

elasticmachine · 2024-11-20T16:08:58Z

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

elasticmachine · 2024-11-20T16:08:58Z

Pinging @elastic/security-solution (Team: SecuritySolution)

packages/deeplinks/security/deep_links.ts

x-pack/plugins/security_solution/common/constants.ts

x-pack/plugins/security_solution/public/detection_engine/rule_management/api/api.ts

semd · 2024-11-21T10:11:29Z

@e40pud The SIEM rule migrations process generates detection rules as a result, but it is not part of any security detection workflow, it would be better if we decouple the code from Detections completely.
Can we create a security_solution/public/siem_migrations directory as a sub-plugin, and a /rules directory inside for all the rule migrations specific UI code? We'll have migrations for other entities (dashboards, saved objects) in the future.

This way we have explicit ownership, RBAC, licensing... The code organization is independent of where we locate the link in the nav tree, we can still place the link inside the Rules category/landing.

x-pack/plugins/security_solution/public/siem_migrations/routes.tsx

.../security_solution/public/siem_migrations/rules/components/siem_migrations_table_context.tsx

...curity_solution/public/siem_migrations/rules/components/siem_migrations_no_items_message.tsx

.../plugins/security_solution/public/siem_migrations/rules/api/hooks/use_get_rule_migrations.ts

Additionally: * updated folder structure * simplified namings

semd · 2024-11-22T13:42:31Z

.../plugins/security_solution/public/siem_migrations/rules/hooks/use_filter_rules_to_install.ts

+
+export type TableFilterOptions = Pick<FilterOptions, 'filter'>;
+
+export const useFilterRulesToInstall = ({


We'll need to do this on the ES side once we add pagination, but it's ok for the initial PR

semd

LGTM!

elasticmachine · 2024-11-22T14:22:34Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: c68b368

Failed CI Steps

FTR Configs #82

Test Failures

[job] [logs] FTR Configs #82 / discover/context_awareness extension getAdditionalCellActions data view mode should not render incompatible cell action for message column

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`securitySolution`	6227	6264	+37

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	13.4MB	13.4MB	+18.9KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`securitySolution`	86.2KB	86.4KB	+233.0B
`securitySolutionEss`	11.8KB	11.8KB	+46.0B
`securitySolutionServerless`	25.6KB	25.7KB	+46.0B
total			+325.0B

History

💛 Build #253435 was flaky ab207b7
💔 Build #253390 failed 84581ef
💛 Build #253295 was flaky 8e0d110
💛 Build #252815 was flaky 438de58

cc @e40pud

banderror

Not sure which file I had to review as a code owner and what was the initial code structure in this PR, but @semd's suggestion makes sense and was an important one to address.

The final code structure LGTM 👍 I left a couple of minor comments. I'd recommend to clean up the code in a follow-up PR to make sure you use descriptive names specific to the siem_migrations subdomain. Names like RulesPage, RulesTable, useRulesColumns etc are way too generic and conflict with the rule management terminology, which would make code search more difficult.

Other than that, I'm really glad we're creating a separate page and organizing the code properly 👍 Should allow us to move forward way easier.

Small reminder: when you get to researching and planning how you're going to deal with prebuilt rules, please don't forget to share your plan with me and @xcrzx before starting to work on the implementation 🙏 This would likely save all of us some time.

🚀

banderror · 2024-11-22T14:24:05Z

x-pack/plugins/security_solution/public/siem_migrations/rules/components/rules_table/index.tsx

+export const RulesTable = React.memo(RulesTableComponent);
+RulesTable.displayName = 'RulesTable';


Can we avoid ambiguity with the Rules table in the Rule management subdomain? Could be something like that:

Suggested change

export const RulesTable = React.memo(RulesTableComponent);

RulesTable.displayName = 'RulesTable';

export const RuleMigrationResultsTable = React.memo(RuleMigrationResultsTableComponent);

RuleMigrationResultsTable.displayName = 'RuleMigrationResultsTable';

banderror · 2024-11-22T14:27:29Z

x-pack/plugins/security_solution/public/siem_migrations/rules/pages/index.tsx

+import { useRulePreviewFlyout } from '../hooks/use_rule_preview_flyout';
+import { NoMigrations } from '../components/no_migrations';
+
+const RulesPageComponent: React.FC = () => {


Please make sure we don't leave any copy-pasted names from the rule management / prebuilt rules areas...

kibanamachine · 2024-11-22T14:48:32Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11974804649

kibanamachine · 2024-11-22T14:53:22Z

💔 All backports failed

Status	Branch	Result
❌	8.x	Backport failed because of merge conflicts You might need to backport the following PRs to 8.x: - [Search][a11y] Fixing wrong navigation sequence after Reset or Save Default Settings (#201163) - [Search] Introduce search navigation plugin (#200314) - [Obs AI Assistant] Query remote indices by default (#193462) - [A11y] Fix space button clearing the index name input (#201349) - chore(slo): refactor transform generators and managers dependency injection (#201031) - fix(slo): remove KibanaThemeProvider' (#201257) - [APM] Migrate traces tests to deployment agnostic (#200561) - [React18] Migrate test suites to account for testing library upgrades ml-ui (#201161) - [ML] Trained Models: fix `NaN` in a progress bar during the download task initialization (#201221) - Sustainable Kibana Architecture: Categorise `straightforward` packages (#199630)

Manual backport

To create the backport manually run:

node scripts/backport --pr 200978

Questions ?

Please refer to the Backport tool documentation

e40pud · 2024-11-25T10:53:10Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…ic#200978) ## Summary [Internal link](elastic/security-team#10820) to the feature details This is a very first version of the SIEM rules migrations UI functionality. The main goal is to setup and agree on a folder structure where the feature gonna live. Tests covering feature will follow in a separate PR (see [internal link](elastic/security-team#11232) for more details). The code follows the structure of prebuilt rules feature https://github.com/elastic/kibana/tree/main/x-pack/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/add_prebuilt_rules_table and hidden behind `siemMigrationsEnabled` feature flag. ### Key UI changes * New "SIEM Rules Migrations." rules management sub-page * Navigation between different "finished" migrations * InMemory table with all the translations within the selected migration * Translation details preview flyout with `Translation` and `Overview` tabs * User cannot modify translations via UI ### Testing locally Enable the flag ``` xpack.securitySolution.enableExperimental: ['siemMigrationsEnabled'] ``` ### Screenshot https://github.com/user-attachments/assets/a5a7e777-c5f8-40b4-be1d-1bd07a2729ac (cherry picked from commit a627e01) # Conflicts: # .github/CODEOWNERS

… (#201545) # Backport This will backport the following commits from `main` to `8.x`: - [[Rules migration][UI] Basic rule migrations UI (#10820) (#200978)](#200978)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)

…ic#200978) ## Summary [Internal link](elastic/security-team#10820) to the feature details This is a very first version of the SIEM rules migrations UI functionality. The main goal is to setup and agree on a folder structure where the feature gonna live. Tests covering feature will follow in a separate PR (see [internal link](elastic/security-team#11232) for more details). The code follows the structure of prebuilt rules feature https://github.com/elastic/kibana/tree/main/x-pack/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/add_prebuilt_rules_table and hidden behind `siemMigrationsEnabled` feature flag. ### Key UI changes * New "SIEM Rules Migrations." rules management sub-page * Navigation between different "finished" migrations * InMemory table with all the translations within the selected migration * Translation details preview flyout with `Translation` and `Overview` tabs * User cannot modify translations via UI ### Testing locally Enable the flag ``` xpack.securitySolution.enableExperimental: ['siemMigrationsEnabled'] ``` ### Screenshot https://github.com/user-attachments/assets/a5a7e777-c5f8-40b4-be1d-1bd07a2729ac

## Summary These are the followup updated to address feedback from my previous PRs: * Make sure to use descriptive names specific to the `siem_migrations` subdomain ([comment](#200978 (review))): > Make sure you use descriptive names specific to the siem_migrations subdomain. Names like RulesPage, RulesTable, useRulesColumns etc are way too generic and conflict with the rule management terminology, which would make code search more difficult. * Export the memo component directly everywhere ([comment](#201597 (comment))): > Could we export the memo component directly everywhere? It's shorter and it makes it easier to find the references in the IDE. * Use one hook to access APIs instead of two ([comment](#202494 (comment))): > I see that for every API request we have to implement 2 separate hooks. Why don't we add error handling to the same hook that does the useQuery? so we have everything in one hook. Or is there a reason to have them separate?

## Summary These are the followup updated to address feedback from my previous PRs: * Make sure to use descriptive names specific to the `siem_migrations` subdomain ([comment](elastic#200978 (review))): > Make sure you use descriptive names specific to the siem_migrations subdomain. Names like RulesPage, RulesTable, useRulesColumns etc are way too generic and conflict with the rule management terminology, which would make code search more difficult. * Export the memo component directly everywhere ([comment](elastic#201597 (comment))): > Could we export the memo component directly everywhere? It's shorter and it makes it easier to find the references in the IDE. * Use one hook to access APIs instead of two ([comment](elastic#202494 (comment))): > I see that for every API request we have to implement 2 separate hooks. Why don't we add error handling to the same hook that does the useQuery? so we have everything in one hook. Or is there a reason to have them separate? (cherry picked from commit 4d8f711)

## Summary These are the followup updated to address feedback from my previous PRs: * Make sure to use descriptive names specific to the `siem_migrations` subdomain ([comment](elastic#200978 (review))): > Make sure you use descriptive names specific to the siem_migrations subdomain. Names like RulesPage, RulesTable, useRulesColumns etc are way too generic and conflict with the rule management terminology, which would make code search more difficult. * Export the memo component directly everywhere ([comment](elastic#201597 (comment))): > Could we export the memo component directly everywhere? It's shorter and it makes it easier to find the references in the IDE. * Use one hook to access APIs instead of two ([comment](elastic#202494 (comment))): > I see that for every API request we have to implement 2 separate hooks. Why don't we add error handling to the same hook that does the useQuery? so we have everything in one hook. Or is there a reason to have them separate?

…ic#200978) ## Summary [Internal link](elastic/security-team#10820) to the feature details This is a very first version of the SIEM rules migrations UI functionality. The main goal is to setup and agree on a folder structure where the feature gonna live. Tests covering feature will follow in a separate PR (see [internal link](elastic/security-team#11232) for more details). The code follows the structure of prebuilt rules feature https://github.com/elastic/kibana/tree/main/x-pack/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/add_prebuilt_rules_table and hidden behind `siemMigrationsEnabled` feature flag. ### Key UI changes * New "SIEM Rules Migrations." rules management sub-page * Navigation between different "finished" migrations * InMemory table with all the translations within the selected migration * Translation details preview flyout with `Translation` and `Overview` tabs * User cannot modify translations via UI ### Testing locally Enable the flag ``` xpack.securitySolution.enableExperimental: ['siemMigrationsEnabled'] ``` ### Screenshot https://github.com/user-attachments/assets/a5a7e777-c5f8-40b4-be1d-1bd07a2729ac

## Summary These are the followup updated to address feedback from my previous PRs: * Make sure to use descriptive names specific to the `siem_migrations` subdomain ([comment](elastic#200978 (review))): > Make sure you use descriptive names specific to the siem_migrations subdomain. Names like RulesPage, RulesTable, useRulesColumns etc are way too generic and conflict with the rule management terminology, which would make code search more difficult. * Export the memo component directly everywhere ([comment](elastic#201597 (comment))): > Could we export the memo component directly everywhere? It's shorter and it makes it easier to find the references in the IDE. * Use one hook to access APIs instead of two ([comment](elastic#202494 (comment))): > I see that for every API request we have to implement 2 separate hooks. Why don't we add error handling to the same hook that does the useQuery? so we have everything in one hook. Or is there a reason to have them separate?

[Rules migration][UI] Basic rule migrations UI (elastic#10820)

438de58

e40pud requested a review from semd November 20, 2024 16:08

e40pud self-assigned this Nov 20, 2024

e40pud requested review from a team as code owners November 20, 2024 16:08

e40pud requested a review from nikitaindik November 20, 2024 16:08

e40pud requested a review from P1llus November 20, 2024 16:13

banderror requested review from banderror and removed request for nikitaindik November 20, 2024 17:11

semd reviewed Nov 21, 2024

View reviewed changes

packages/deeplinks/security/deep_links.ts Outdated Show resolved Hide resolved

semd reviewed Nov 21, 2024

View reviewed changes

x-pack/plugins/security_solution/common/constants.ts Outdated Show resolved Hide resolved

semd reviewed Nov 21, 2024

View reviewed changes

x-pack/plugins/security_solution/public/detection_engine/rule_management/api/api.ts Outdated Show resolved Hide resolved

e40pud added 2 commits November 21, 2024 18:00

Review feedback: re-structuring

0d2f181

Remove obsolete code

8e0d110

e40pud removed the Team:Detection Rule Management Security Detection Rule Management Team label Nov 21, 2024

e40pud added 3 commits November 21, 2024 23:18

Set codeowners for the public siem_migrations folder

27296a0

Merge branch 'main' into security/siem-migrations/10820-ui-step-1

84581ef

Merge branch 'main' into security/siem-migrations/10820-ui-step-1

ab207b7

semd reviewed Nov 22, 2024

View reviewed changes

x-pack/plugins/security_solution/public/siem_migrations/routes.tsx Outdated Show resolved Hide resolved

semd reviewed Nov 22, 2024

View reviewed changes

.../security_solution/public/siem_migrations/rules/components/siem_migrations_table_context.tsx Outdated Show resolved Hide resolved

semd reviewed Nov 22, 2024

View reviewed changes

...curity_solution/public/siem_migrations/rules/components/siem_migrations_no_items_message.tsx Outdated Show resolved Hide resolved

semd reviewed Nov 22, 2024

View reviewed changes

.../plugins/security_solution/public/siem_migrations/rules/api/hooks/use_get_rule_migrations.ts Outdated Show resolved Hide resolved

e40pud added 4 commits November 22, 2024 13:04

Review feedback: get rid of the context

7457134

Additionally: * updated folder structure * simplified namings

Review feedback: handle no migraions state properly

3059a9c

Merge branch 'main' into security/siem-migrations/10820-ui-step-1

04ea3d0

Remove tags filter

c68b368

semd reviewed Nov 22, 2024

View reviewed changes

semd approved these changes Nov 22, 2024

View reviewed changes

banderror approved these changes Nov 22, 2024

View reviewed changes

e40pud merged commit a627e01 into elastic:main Nov 22, 2024
46 checks passed

kibanamachine added the v9.0.0 label Nov 22, 2024

e40pud mentioned this pull request Nov 25, 2024

[8.x] [Rules migration][UI] Basic rule migrations UI (#10820) (#200978) #201545

Merged

kibanamachine added the v8.18.0 label Nov 25, 2024

e40pud mentioned this pull request Dec 3, 2024

[Rules migration] Post merge feedback followups #202815

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Rules migration][UI] Basic rule migrations UI (#10820) #200978

[Rules migration][UI] Basic rule migrations UI (#10820) #200978

e40pud commented Nov 20, 2024 •

edited by kibanamachine

Loading

elasticmachine commented Nov 20, 2024

elasticmachine commented Nov 20, 2024

elasticmachine commented Nov 20, 2024

semd commented Nov 21, 2024 •

edited

Loading

semd Nov 22, 2024

semd left a comment

elasticmachine commented Nov 22, 2024

banderror left a comment

banderror Nov 22, 2024

banderror Nov 22, 2024

kibanamachine commented Nov 22, 2024

kibanamachine commented Nov 22, 2024

e40pud commented Nov 25, 2024


		export type TableFilterOptions = Pick<FilterOptions, 'filter'>;

		export const useFilterRulesToInstall = ({

		export const RulesTable = React.memo(RulesTableComponent);
		RulesTable.displayName = 'RulesTable';

[Rules migration][UI] Basic rule migrations UI (#10820) #200978

[Rules migration][UI] Basic rule migrations UI (#10820) #200978

Conversation

e40pud commented Nov 20, 2024 • edited by kibanamachine Loading

Summary

Key UI changes

Testing locally

Screenshot

elasticmachine commented Nov 20, 2024

elasticmachine commented Nov 20, 2024

elasticmachine commented Nov 20, 2024

semd commented Nov 21, 2024 • edited Loading

semd Nov 22, 2024

Choose a reason for hiding this comment

semd left a comment

Choose a reason for hiding this comment

elasticmachine commented Nov 22, 2024

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Async chunks

Page load bundle

History

banderror left a comment

Choose a reason for hiding this comment

banderror Nov 22, 2024

Choose a reason for hiding this comment

banderror Nov 22, 2024

Choose a reason for hiding this comment

kibanamachine commented Nov 22, 2024

kibanamachine commented Nov 22, 2024

💔 All backports failed

Manual backport

Questions ?

e40pud commented Nov 25, 2024

💚 All backports created successfully

Questions ?

e40pud commented Nov 20, 2024 •

edited by kibanamachine

Loading

semd commented Nov 21, 2024 •

edited

Loading