[Security Solution] Add support for Fleet packages with historical rule versions #148643
Conversation
xcrzx
added
release_note:skip
xcrzx
changed the title
xcrzx
force-pushed
the
historical-rules-package-support
branch
2 times, most recently
from
eff1969 to
3a42c61
Compare
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
xcrzx
force-pushed
the
historical-rules-package-support
branch
from
3a42c61 to
56be187
Compare
xcrzx
force-pushed
the
historical-rules-package-support
branch
2 times, most recently
from
460ed60 to
65ca509
Compare
There was a problem hiding this comment.
Awesome! Tested locally and reviewed the code.
While testing using the provided instructions, I was able to
- Build a package with a few historical versions of a selected rule. Run Kibana looking at a local package registry containing this package.
- Install prebuilt rules from scratch and verify that the latest version out of 3 historical ones gets installed.
- Install prebuilt rules from a package without historical versions; then upgrade the package; then upgrade the rules and verify that the latest version out of 3 historical ones gets installed.
So both the installation and upgrade workflows work fine.
NOTE: While zooming with @xcrzx, we found a bug in Fleet where, if there's a release package in the package registry + a prerelease package that has a higher version, our calls we do from the UI to install or upgrade the package, will fail. This seems to be a bug we will have in prod in 8.6 once the TRADE team releases a new prerelease package compatible with 8.6. @xcrzx will open a bug for that.
Other than that, thank you for this PR @xcrzx! It's great to see that it was so easy to add this support, flat data model FTW 🙂 I left just a few nits.
| options?: Omit<SavedObjectsFindOptions, 'type'> | ||
| ) => Promise<SavedObjectsFindResponse<IRuleAssetSavedObject>>; | ||
| all: () => Promise<IRuleAssetSavedObject[]>; | ||
| export interface RuleAssetsClient { |
There was a problem hiding this comment.
Nit: Let's rename it to IRuleAssetsClient. The idea is to start an interface with I for objects with behavior, and have interfaces without I for data. This allows using classes that implement interfaces if needed (yes I remember you don't like classes!).
export class RuleAssetsClient implements IRuleAssetsClient, IAnotherInterface { ... }Otherwise, it would be Java 😂
export class RuleAssetsClientImpl implements RuleAssetsClient, AnotherInterface { ... }| const finder = savedObjectsClient.createPointInTimeFinder({ | ||
| perPage: DEFAULT_PAGE_SIZE, | ||
| type: ruleAssetSavedObjectType, | ||
| fetchLatestVersions: () => { |
There was a problem hiding this comment.
Finally, a real abstraction! 👍
| { | ||
| rules: AggregationsMultiBucketAggregateBase<{ | ||
| latest_version: AggregationsTopHitsAggregate; | ||
| }>; | ||
| } |
There was a problem hiding this comment.
I didn't know that it's possible to type aggregations for the find method using some built-in types. Very useful!
Here's the issue |
xcrzx
force-pushed
the
historical-rules-package-support
branch
from
65ca509 to
85a05b4
Compare
💚 Build Succeeded
Metrics [docs]Unknown metric groupsESLint disabled line counts
Total ESLint disabled count
History
To update your PR or re-run it, just comment with: cc @xcrzx |
Resolves: #148179
Summary
As a result of this PoC flat package structure was chosen for storing historical detection rules, and this PR adds support for Fleet packages containing multiple historical versions per rule.
Updated API endpoints:
PUT /api/detection_engine/rules/prepackaged)GET /api/detection_engine/rules/prepackaged/_status)The API endpoints can work interchangeably with the current historical rules package structure and the new "flat" structure. Therefore, the API interface has not been changed, and the existing
security-rulesaved objects are still used.How to test this PR
To test this PR, you should create a new
security_detection_enginepackage version. See the documentation on how to spin up a local package registry with custom packages.The package should contain rule saved objects with rule id and version in the name (
security_rule/[ruleId]:[ruleVersion].json) with the following content:{ "id": "[ruleId]:[ruleVersion]", "type": "security-rule", "attributes": { // Rule attributes } }You should be able to upgrade the current rules package to the new "flat" and vice-versa.