Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Cloud Security] Increase retention period on queries related to 3rd party data loading #195636

Merged
merged 8 commits into from
Oct 11, 2024

Conversation

maxcold
Copy link
Contributor

@maxcold maxcold commented Oct 9, 2024

Summary

Increase retention on Cloud Security queries to accommodate a longer retention period on third-party CDR integrations, such as Wiz and AWS SecurityHub. This introduces regression for #142198
This is meant is a temporary workaround until we find a robust way to get full posture for third-party CDR integrations
This change goes together with increasing retention period on Wiz:

fixes:

How to test

The CI deployed a serverless project where I installed Wiz and CSP integrations and ingested some data.

@maxcold maxcold added release_note:skip Skip the PR/issue when compiling release notes v9.0.0 Team:Cloud Security Cloud Security team related ci:cloud-deploy Create or update a Cloud deployment ci:project-deploy-security Create a Security Serverless Project v8.16.0 labels Oct 9, 2024
@maxcold maxcold marked this pull request as ready for review October 10, 2024 13:55
@maxcold maxcold requested a review from a team as a code owner October 10, 2024 13:55
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

Copy link
Contributor

@CohenIdo CohenIdo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@maxcold maxcold added the backport:prev-major Backport to (8.x, 8.17, 8.16, 8.15) the previous major branch and other branches in development label Oct 11, 2024
@maxcold maxcold enabled auto-merge (squash) October 11, 2024 07:48
@elasticmachine
Copy link
Contributor

elasticmachine commented Oct 11, 2024

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/cloud-security-posture-common 106 107 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 20.7MB 20.7MB -205.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
cloudSecurityPosture 18.6KB 18.6KB +42.0B
Unknown metric groups

API count

id before after diff
@kbn/cloud-security-posture-common 108 109 +1

History

@maxcold maxcold merged commit e18c52e into main Oct 11, 2024
32 checks passed
@maxcold maxcold deleted the csp-increase-retention-period-for-3rd-party branch October 11, 2024 09:34
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.15, 8.x

https://github.com/elastic/kibana/actions/runs/11289985502

@kibanamachine
Copy link
Contributor

💔 Some backports could not be created

Status Branch Result
8.15 Backport failed because of merge conflicts
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 195636

Questions ?

Please refer to the Backport tool documentation

@maxcold maxcold removed the backport:prev-major Backport to (8.x, 8.17, 8.16, 8.15) the previous major branch and other branches in development label Oct 11, 2024
@maxcold maxcold added the backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) label Oct 11, 2024
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11290291224

@maxcold
Copy link
Contributor Author

maxcold commented Oct 11, 2024

used incorrect label, no need to backport to 8.15

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 11, 2024
…party data loading (elastic#195636)

## Summary

Increase retention on Cloud Security queries to accommodate a longer
retention period on third-party CDR integrations, such as Wiz and AWS
SecurityHub. This introduces regression for
elastic#142198
This is meant is a temporary workaround until we find a robust way to
get full posture for third-party CDR integrations
This change goes together with increasing retention period on Wiz:
- elastic/integrations#11393

fixes:
- elastic/security-team#10683

## How to test
The CI deployed a serverless project where I installed Wiz and CSP
integrations and ingested some data.

(cherry picked from commit e18c52e)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 11, 2024
…o 3rd party data loading (#195636) (#195871)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Cloud Security] Increase retention period on queries related to 3rd
party data loading
(#195636)](#195636)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Maxim
Kholod","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-10-11T09:34:12Z","message":"[Cloud
Security] Increase retention period on queries related to 3rd party data
loading (#195636)\n\n## Summary\r\n\r\nIncrease retention on Cloud
Security queries to accommodate a longer\r\nretention period on
third-party CDR integrations, such as Wiz and AWS\r\nSecurityHub. This
introduces regression
for\r\nhttps://github.com//issues/142198\r\nThis is meant
is a temporary workaround until we find a robust way to\r\nget full
posture for third-party CDR integrations\r\nThis change goes together
with increasing retention period on Wiz: \r\n-
https://github.com/elastic/integrations/pull/11393\r\n\r\nfixes:\r\n-
https://github.com/elastic/security-team/issues/10683\r\n\r\n## How to
test\r\nThe CI deployed a serverless project where I installed Wiz and
CSP\r\nintegrations and ingested some
data.","sha":"e18c52eec2cb18dc2590b61d7649de4507f060a7","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Cloud
Security","backport:prev-major","ci:cloud-deploy","ci:project-deploy-security","v8.16.0"],"title":"[Cloud
Security] Increase retention period on queries related to 3rd party data
loading","number":195636,"url":"https://github.com/elastic/kibana/pull/195636","mergeCommit":{"message":"[Cloud
Security] Increase retention period on queries related to 3rd party data
loading (#195636)\n\n## Summary\r\n\r\nIncrease retention on Cloud
Security queries to accommodate a longer\r\nretention period on
third-party CDR integrations, such as Wiz and AWS\r\nSecurityHub. This
introduces regression
for\r\nhttps://github.com//issues/142198\r\nThis is meant
is a temporary workaround until we find a robust way to\r\nget full
posture for third-party CDR integrations\r\nThis change goes together
with increasing retention period on Wiz: \r\n-
https://github.com/elastic/integrations/pull/11393\r\n\r\nfixes:\r\n-
https://github.com/elastic/security-team/issues/10683\r\n\r\n## How to
test\r\nThe CI deployed a serverless project where I installed Wiz and
CSP\r\nintegrations and ingested some
data.","sha":"e18c52eec2cb18dc2590b61d7649de4507f060a7"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/195636","number":195636,"mergeCommit":{"message":"[Cloud
Security] Increase retention period on queries related to 3rd party data
loading (#195636)\n\n## Summary\r\n\r\nIncrease retention on Cloud
Security queries to accommodate a longer\r\nretention period on
third-party CDR integrations, such as Wiz and AWS\r\nSecurityHub. This
introduces regression
for\r\nhttps://github.com//issues/142198\r\nThis is meant
is a temporary workaround until we find a robust way to\r\nget full
posture for third-party CDR integrations\r\nThis change goes together
with increasing retention period on Wiz: \r\n-
https://github.com/elastic/integrations/pull/11393\r\n\r\nfixes:\r\n-
https://github.com/elastic/security-team/issues/10683\r\n\r\n## How to
test\r\nThe CI deployed a serverless project where I installed Wiz and
CSP\r\nintegrations and ingested some
data.","sha":"e18c52eec2cb18dc2590b61d7649de4507f060a7"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Maxim Kholod <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) ci:cloud-deploy Create or update a Cloud deployment ci:project-deploy-security Create a Security Serverless Project release_note:skip Skip the PR/issue when compiling release notes Team:Cloud Security Cloud Security team related v8.16.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants