Add all historical versions for v8.17.0 and above packages

[shashank-elastic]

Pull Request

Issue link(s): #4150 Smart Limits is being available in 8.17.0 and above kibana version and we can now include all historical versions for our rule packages

Summary - What I changed

• Smart Limits is being available in 8.17.0 and above kibana version and we can now include all historical versions for our rule packages
• We will continue to limit the historical rule versions to latest and latest-1 for packages 8.16.0 and below
• We had considered the idea of introducing a flag for the build function which would require us to change all our workflows of build for supported versions.
• This will introduce a constant version for comparison but have left comments for usage

Please note this is a required PR for the next week release

How To Test

• Building 8.17.0 Package

❯ python -m detection_rules dev build-release 
Loaded config file: /Users/shashankks/elastic_workspace/detection-rules/.detection-rules-cfg.json

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█

[+] Building package 8.17
 - 4 rules excluded from package
Rule changes detected!
 - 1 changed rules
 - 0 new rules
 - 0 newly deprecated rules
run `build-release --update-version-lock` to update version.lock.json and deprecated_rules.json
Package saved to: /Users/shashankks/elastic_workspace/detection-rules/releases/8.17
loaded security_detection_engine manifests from the following package versions: ['8.16.2', '8.16.1', '8.15.11', '8.15.10', '8.15.9', '8.15.8', '8.15.7', '8.15.6', '8.15.5', '8.15.4', '8.15.3', '8.15.2', '8.15.1', '8.14.17', '8.14.16', '8.14.15', '8.14.14', '8.14.13', '8.14.12', '8.14.11', '8.14.10', '8.14.9', '8.14.8', '8.14.7', '8.14.6', '8.14.5', '8.14.4', '8.14.3', '8.14.2', '8.14.1', '8.13.23', '8.13.22', '8.13.21', '8.13.20', '8.13.19', '8.13.18', '8.13.17', '8.13.16', '8.13.15', '8.13.14', '8.13.13', '8.13.12', '8.13.11', '8.13.10', '8.13.9', '8.13.8', '8.13.7', '8.13.6', '8.13.5', '8.13.4', '8.13.3', '8.13.2', '8.13.1', '8.12.26', '8.12.25', '8.12.24', '8.12.23', '8.12.22', '8.12.21', '8.12.20', '8.12.19', '8.12.18', '8.12.17', '8.12.16', '8.12.15', '8.12.14', '8.12.13', '8.12.12', '8.12.11', '8.12.10', '8.12.9', '8.12.8', '8.12.7', '8.12.6', '8.12.5', '8.12.4', '8.12.3', '8.12.2', '8.12.1', '8.11.21', '8.11.20', '8.11.19', '8.11.18', '8.11.17', '8.11.16', '8.11.15', '8.11.14', '8.11.13', '8.11.12', '8.11.11', '8.11.10', '8.11.9', '8.11.8', '8.11.7', '8.11.6', '8.11.5', '8.11.4', '8.11.3', '8.11.2', '8.11.1', '8.10.18', '8.10.17', '8.10.16', '8.10.15', '8.10.14', '8.10.13', '8.10.12', '8.10.11', '8.10.10', '8.10.9', '8.10.8', '8.10.7', '8.10.6', '8.10.5', '8.10.4', '8.10.3', '8.10.2', '8.10.1', '8.9.15', '8.9.14', '8.9.13', '8.9.12', '8.9.11', '8.9.10', '8.9.9', '8.9.8', '8.9.7', '8.9.6', '8.9.5', '8.9.4', '8.9.3', '8.9.2', '8.9.1', '8.8.15', '8.8.14', '8.8.13', '8.8.12', '8.8.11', '8.8.10', '8.8.9', '8.8.8', '8.8.7', '8.8.6', '8.8.5', '8.8.4', '8.8.3', '8.8.2', '8.8.1', '8.7.13', '8.7.12', '8.7.11', '8.7.10', '8.7.9', '8.7.8', '8.7.7', '8.7.6', '8.7.5', '8.7.4', '8.7.3', '8.7.2', '8.7.1', '8.6.10', '8.6.9', '8.6.8', '8.6.7', '8.6.6', '8.6.5', '8.6.4', '8.6.3', '8.6.2', '8.6.1', '8.5.8', '8.5.7', '8.5.6', '8.5.5', '8.5.4', '8.5.3', '8.5.2', '8.5.1', '8.4.5', '8.4.4', '8.4.3', '8.4.2', '8.4.1', '8.3.4', '8.3.3', '8.3.2', '8.3.1', '8.2.1', '8.1.1', '1.0.2', '1.0.1']
[+] Adding all historical rule versions in our release package for version 8.17.0
[+] Adding historical rules from 8.16.2 package
- sha256: 8bb4a464224ae076b577efd1bb9c5f0c0edc25beacc6f53c7df192d9ef6cc908
- 1256 rules included

Building 8.16.0 Package

❯ python -m detection_rules dev build-release 
Loaded config file: /Users/shashankks/elastic_workspace/detection-rules/.detection-rules-cfg.json

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█

[+] Building package 8.16
 - 4 rules excluded from package
Rule changes detected!
 - 0 changed rules
 - 1 new rules
 - 0 newly deprecated rules
run `build-release --update-version-lock` to update version.lock.json and deprecated_rules.json
Package saved to: /Users/shashankks/elastic_workspace/detection-rules/releases/8.16
loaded security_detection_engine manifests from the following package versions: ['8.16.2', '8.16.1', '8.15.11', '8.15.10', '8.15.9', '8.15.8', '8.15.7', '8.15.6', '8.15.5', '8.15.4', '8.15.3', '8.15.2', '8.15.1', '8.14.17', '8.14.16', '8.14.15', '8.14.14', '8.14.13', '8.14.12', '8.14.11', '8.14.10', '8.14.9', '8.14.8', '8.14.7', '8.14.6', '8.14.5', '8.14.4', '8.14.3', '8.14.2', '8.14.1', '8.13.23', '8.13.22', '8.13.21', '8.13.20', '8.13.19', '8.13.18', '8.13.17', '8.13.16', '8.13.15', '8.13.14', '8.13.13', '8.13.12', '8.13.11', '8.13.10', '8.13.9', '8.13.8', '8.13.7', '8.13.6', '8.13.5', '8.13.4', '8.13.3', '8.13.2', '8.13.1', '8.12.26', '8.12.25', '8.12.24', '8.12.23', '8.12.22', '8.12.21', '8.12.20', '8.12.19', '8.12.18', '8.12.17', '8.12.16', '8.12.15', '8.12.14', '8.12.13', '8.12.12', '8.12.11', '8.12.10', '8.12.9', '8.12.8', '8.12.7', '8.12.6', '8.12.5', '8.12.4', '8.12.3', '8.12.2', '8.12.1', '8.11.21', '8.11.20', '8.11.19', '8.11.18', '8.11.17', '8.11.16', '8.11.15', '8.11.14', '8.11.13', '8.11.12', '8.11.11', '8.11.10', '8.11.9', '8.11.8', '8.11.7', '8.11.6', '8.11.5', '8.11.4', '8.11.3', '8.11.2', '8.11.1', '8.10.18', '8.10.17', '8.10.16', '8.10.15', '8.10.14', '8.10.13', '8.10.12', '8.10.11', '8.10.10', '8.10.9', '8.10.8', '8.10.7', '8.10.6', '8.10.5', '8.10.4', '8.10.3', '8.10.2', '8.10.1', '8.9.15', '8.9.14', '8.9.13', '8.9.12', '8.9.11', '8.9.10', '8.9.9', '8.9.8', '8.9.7', '8.9.6', '8.9.5', '8.9.4', '8.9.3', '8.9.2', '8.9.1', '8.8.15', '8.8.14', '8.8.13', '8.8.12', '8.8.11', '8.8.10', '8.8.9', '8.8.8', '8.8.7', '8.8.6', '8.8.5', '8.8.4', '8.8.3', '8.8.2', '8.8.1', '8.7.13', '8.7.12', '8.7.11', '8.7.10', '8.7.9', '8.7.8', '8.7.7', '8.7.6', '8.7.5', '8.7.4', '8.7.3', '8.7.2', '8.7.1', '8.6.10', '8.6.9', '8.6.8', '8.6.7', '8.6.6', '8.6.5', '8.6.4', '8.6.3', '8.6.2', '8.6.1', '8.5.8', '8.5.7', '8.5.6', '8.5.5', '8.5.4', '8.5.3', '8.5.2', '8.5.1', '8.4.5', '8.4.4', '8.4.3', '8.4.2', '8.4.1', '8.3.4', '8.3.3', '8.3.2', '8.3.1', '8.2.1', '8.1.1', '1.0.2', '1.0.1']
[+] Limit historical rule versions in our release package for version 8.16.0
[+] Adding historical rules from 8.16.2 package
- sha256: bc1bf8d7e607c8fe4361a60055f9e7667f228d885c371c78f4de6c87817ff892
- 1256 rules included

Checklist

• Added a label for the type of pr: bug, enhancement, schema, maintenance, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
• Added the meta:rapid-merge label if planning to merge within 24 hours
• Secret and sensitive material has been managed correctly
• Automated testing was updated or added to match the most common scenarios
• Documentation and comments were added for features that require explanation

Contributor checklist

• Have you signed the contributor license agreement?
• Have you followed the contributor guidelines?

[protectionsmachine]

Enhancement - Guidelines

These guidelines serve as a reminder set of considerations when addressing adding a feature to the code.

Documentation and Context

• Describe the feature enhancement in detail (alternative solutions, description of the solution, etc.) if not already documented in an issue.
• Include additional context or screenshots.
• Ensure the enhancement includes necessary updates to the documentation and versioning.

Code Standards and Practices

• Code follows established design patterns within the repo and avoids duplication.
• Code changes do not introduce new warnings or errors.
• Variables and functions are well-named and descriptive.
• Any unnecessary / commented-out code is removed.
• Ensure that the code is modular and reusable where applicable.
• Check for proper exception handling and messaging.

Testing

• New unit tests have been added to cover the enhancement.
• Existing unit tests have been updated to reflect the changes.
• Provide evidence of testing and validating the enhancement (e.g., test logs, screenshots).
• Validate that any rules affected by the enhancement are correctly updated.
• Ensure that performance is not negatively impacted by the changes.
• Verify that any release artifacts are properly generated and tested.

Additional Checks

• Ensure that the enhancement does not break existing functionality.
• Review the enhancement with a peer or team member for additional insights.
• Verify that the enhancement works across all relevant environments (e.g., different OS versions).
• Confirm that all dependencies are up-to-date and compatible with the changes.
• Confirm that the proper version label is applied to the PR patch, minor, major.

[eric-forte-elastic]

Peer reviewed, LGTM 👍