[AWS] Add dimensions to firewall, transit gateway and vpn data streams #6339
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: constanca-m <[email protected]>
Signed-off-by: constanca-m <[email protected]>
Signed-off-by: constanca-m <[email protected]>
Signed-off-by: constanca-m <[email protected]>
Signed-off-by: constanca-m <[email protected]>
Signed-off-by: constanca-m <[email protected]>
Signed-off-by: constanca-m <[email protected]>
Signed-off-by: constanca-m <[email protected]>
aspacca
approved these changes
Jun 5, 2023
aspacca
reviewed
Jun 5, 2023
packages/aws/changelog.yml
Outdated
| @@ -1,4 +1,9 @@ | |||
| # newer versions go on top | |||
| - version: "1.36.9" | |||
There was a problem hiding this comment.
nit: in #6365 we bumped minor version, here only patch one
why the difference? :)
There was a problem hiding this comment.
1.37 changes were not related to TSDB, that is why I bumped to 1.38 in that PR. Unlike here, since 1.36.8 was a change to add dimensions.
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
🤖 GitHub commentsExpand to view the GitHub comments
To re-run your PR in the CI, just comment with:
|
🌐 Coverage report
|
Signed-off-by: constanca-m <[email protected]>
|
Package aws - 1.38.3 containing this change is available at https://epr.elastic.co/search?package=aws |
sodhikirti07
pushed a commit
that referenced
this pull request
Jun 15, 2023
#6339) * Add dimensions VPN. Signed-off-by: constanca-m <[email protected]> * Add dimensions transit gateway. Signed-off-by: constanca-m <[email protected]> * Add dimensions firewall. Signed-off-by: constanca-m <[email protected]> * Update version. Signed-off-by: constanca-m <[email protected]> * Update changelog. Signed-off-by: constanca-m <[email protected]> * Add 2 ecs dimension fields. Signed-off-by: constanca-m <[email protected]> * Remove cloud.availability_zone as dimension (not present) Signed-off-by: constanca-m <[email protected]> * Update md. Signed-off-by: constanca-m <[email protected]> * Resolve conflict. Signed-off-by: constanca-m <[email protected]> --------- Signed-off-by: constanca-m <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Add dimensions to firewall, transit gateway and vpn data streams.
Transit gateway: A transit gateway can have more than one attachment, so it needs to be set as dimensions as well. A transit gateway is unique per AWS account.
VPN: Each tunnel belongs to a VPN. Combination tunnel id + vpn id is unique within the AWS scope account (account id + region).
Note: for each VPN site to site connection, there are four documents sent to ES. Is this expected?
Firewall: The firewall name is unique per region in an account. A firewall can have more than one engine and one Availability Zone, so both need to be set as dimension to avoid documents being overwritten. Since we can have more than one custom action in the same firewall, we also set that as dimension.
Question:
CustomActionfields holds the name of the action. Since a firewall can have different groups (names are unique, but we don't have access to that field in this data stream), then we can also have different custom actions with the exact same name associated with one firewall. I tried to reproduce this, but even with multiple rules with the same name I kept receiving just one document for each rule. Without considering TSDB, can we receive two documents for the same action at the same time? If so, dimensions are not enough for this data stream.Regarding ECS fields, more info can be found here #5193 (comment).
Checklist
changelog.ymlfile.Author's Checklist
How to test this PR locally
Refer to #6293.
Related issues
Relates to #6293.