Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] googlecloud audit: use flattened type for request and response fields #20826

Closed
ynirk opened this issue Aug 27, 2020 · 5 comments
Closed

Comments

@ynirk
Copy link

ynirk commented Aug 27, 2020

While ingesting gouglecloud audit logs we noticed mappings errors similar to #18465 where a same field can either be text or object.
Mapping errors mainly come from GKE control plan logs on the following fields:

  • response.status (already handled in 7.9)
  • response.metadata.labels.istio
  • response.metadata.labels.app
  • response.spec.volumes
  • request.target
  • request.instances
  • request.spec.volumes
  • request.metadata.labels.app

Also recently we encountered Limit of mapping depth errors on spec field (ex: response.spec.validation.openAPIV3Schema.properties.spec.properties.nodeSets.items.properties.volumeClaimTemplates.items.properties.status.properties.conditions.items.properties.status)

It would be helpful to do something similar to Cloudtrail #19121 and flatten response & request fields

@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@andrewkroh
Copy link
Member

What's the version of Filebeat you were using when you encountered the Limit of mapping depth error?

@ynirk
Copy link
Author

ynirk commented Aug 27, 2020

I'm using logstash with filters mapping filebeat ingestion pipeline - sorry for the confusion this one might not be relevant to filebeat

@andrewkroh
Copy link
Member

No problem. Just wanted to make sure there wasn't an existing problem in the module. Because right now I expect the module to only create fields that we know are safely handled in the mapping and leave the rest in the raw event.original blob.

@botelastic
Copy link

botelastic bot commented Jul 28, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@botelastic botelastic bot added the Stalled label Jul 28, 2021
@botelastic botelastic bot closed this as completed Aug 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants