Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add checks to "RENAME X to event.original" processors across Filebeat module pipelines #38224

Open
rseldner opened this issue Mar 7, 2024 · 2 comments
Labels
enhancement Filebeat Filebeat Team:Obs-InfraObs Label for the Observability Infrastructure Monitoring team Team:Security-Edge Integrations Security Edge Integrations Team Team:Security-Service Integrations Security Service Integrations Team

Comments

@rseldner
Copy link
Contributor

rseldner commented Mar 7, 2024

Describe the enhancement:
Update filebeat module pipelines to include a conditional check for the existence of event.original

Describe a specific use case for the enhancement or feature:

When renaming a field to event.original, it appears the ingest pipelines for filebeat modules generally do not take into account whether event.original already exists, which can occur when using Logstash. This can result in "field [event.original] already exists" errors.

This has been addressed in elastic agent integrations' pipelines by including a conditional check:
if: ctx.event?.original == null
PR: elastic/integrations#7822

It would be nice to implement the same logic in the filebeat ingest pipelines.

Example:

Filebeat IIS Module IIS Integration
- rename:
field: message
target_field: event.original
- rename:
field: message
target_field: event.original
ignore_missing: true
if: 'ctx.event?.original == null'
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Mar 7, 2024
@rseldner rseldner added the Team:Ingest Management Label for the Ingest Management team label Mar 7, 2024
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Mar 7, 2024
@ishleenk17 ishleenk17 added Team:Obs-InfraObs Label for the Observability Infrastructure Monitoring team and removed Team:Ingest Management Label for the Ingest Management team labels Mar 8, 2024
@andrewkroh andrewkroh added Team:Security-Service Integrations Security Service Integrations Team Team:Security-Edge Integrations Security Edge Integrations Team labels Apr 3, 2024
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-edge-integrations (Team:Security-Edge Integrations)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Filebeat Filebeat Team:Obs-InfraObs Label for the Observability Infrastructure Monitoring team Team:Security-Edge Integrations Security Edge Integrations Team Team:Security-Service Integrations Security Service Integrations Team
Projects
None yet
Development

No branches or pull requests

4 participants