[Security Solution] Add a rule management filters internal endpoint

[maximpn]

Addresses: #137428

Summary

Adds a new internal lightweight endpoint to fetch rules related information like the number of installed prebuilt rules, the number of custom rules and etc.

Details

This PR adds a quite simple and lightweight endpoint for fetching rules related information which is

• the number of installed prebuilt rules
• the number of custom rules
• tags

UI has been updated accordingly. The result of the endpoint are mostly used in the rules table filter but not limited to.

The added endpoint doesn't implement full aggregation for fetching rule numbers so it's planned to be done in the following PR.

Comparison

The following screenshots from the browser's network tab demonstrate that the new endpoint is faster which is good since it's intended to be updated executed relatively often whenever the rules are updated.

Prebuilt rules endpoint which was used for fetching rules related information

The new endpoint

Checklist

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Unit or functional tests were updated or added to match the most common scenarios

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[xcrzx]

Awesome work, thank you, @maximpn 🙌 We're finally moving away from aggregating all rule tags in memory 🎉

I looked through the code and left some initial comments. And I'm going to check out the PR and test it locally later this week.

[xcrzx]

I checked out and tested this PR locally. The new endpoint seems to be working as expected; the rules table filters reflect changes in rules properly. The new endpoint works perceivably faster.

I left some minor suggestions, but overall, the implementation looks good to me 👍

[maximpn]

@elasticmachine merge upstream

[maximpn]

@elasticmachine merge upstream

[maximpn]

@pmuellr I've added unit and functional tests to cover the tags limit. To be honest I think 50 tags is quite low limit since the number of tags can easily be higher. To address this and the other issues there is a task I'm working on in parallel which will allow to optimize the aggregate method to use with arbitrary aggregations.

[pmuellr]

LGTM, thx for the changes!

[maximpn]

@elasticmachine merge upstream

[kibana-ci]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: f6af276

Failed CI Steps

• Security Solution Tests #2

Test Failures

• [job] [logs] Security Solution Tests #2 / Import rules Imports a custom rule with exceptions

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
securitySolution	3391	3392	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	12.6MB	12.6MB	+783.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
securitySolution	51.0KB	50.9KB	-68.0B

Unknown metric groups

ESLint disabled in files

id	before	after	diff
osquery	1	2	+1

ESLint disabled line counts

id	before	after	diff
enterpriseSearch	19	21	+2
fleet	61	67	+6
osquery	109	115	+6
securitySolution	439	445	+6
total			+20

Total ESLint disabled count

id	before	after	diff
enterpriseSearch	20	22	+2
fleet	70	76	+6
osquery	110	117	+7
securitySolution	516	522	+6
total			+21

History

• 💚 Build #95525 succeeded 4317558
• 💔 Build #95523 failed f153369
• 💔 Build #95522 failed 9755803
• 💚 Build #93955 succeeded 8d4c69322bad37c64d446e0a6aacee1684757313
• 💔 Build #93546 failed 4dcc04735b2e01c76dec8cdc3c672b423e443015
• 💛 Build #93137 was flaky 4093f5c0be31f3b7d5bbaa70def09d5f643f1f36

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @maximpn