Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Change field type in aws cloudtrail fileset to flattened #19020

Closed

Conversation

leehinman
Copy link
Contributor

What does this PR do?

A string representation of the following fields was stored and indexed
as keyword. Changing to a flattened object.

  • request_parameters
  • response_elements
  • additional_eventdata
  • service_event_details

Why is it important?

Changing to flattened object makes searching easier and avoids mapping explosion

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

TESTING_FILEBEAT_MODULES=aws TESTING_FILEBEAT_FILESETS=cloudtrail mage -v pythonIntegTest

Related issues

@leehinman leehinman added enhancement Filebeat Filebeat needs_backport PR is waiting to be backported to other branches. Team:SIEM labels Jun 5, 2020
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 5, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jun 5, 2020
A string representation of the following fields was stored and indexed
as keyword.  Changing to a flattened object which is more easily
searched.

- request_parameters
- response_elements
- additional_eventdata
- service_event_details

Closes elastic#18866
@@ -21,9 +22,6 @@
"fileset.name": "cloudtrail",
"input.type": "log",
"log.offset": 0,
"related.user": [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

related.user is lost after this change, but can't see why it happens. Can it be fixed?

@leehinman leehinman closed this Jun 11, 2020
@leehinman leehinman deleted the 18866_cloudtrail_flattened branch October 5, 2020 19:12
@andrewkroh andrewkroh removed the needs_backport PR is waiting to be backported to other branches. label Dec 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[filebeat][aws][cloudtrail] accessKeyId should be searchable
4 participants