Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Cisco module asa-ftd pipeline pollutes the mapping when it fails #18391

Closed
weltenwort opened this issue May 8, 2020 · 1 comment · Fixed by #18392
Closed

[Filebeat] Cisco module asa-ftd pipeline pollutes the mapping when it fails #18391

weltenwort opened this issue May 8, 2020 · 1 comment · Fixed by #18392
Assignees
Labels

Comments

@weltenwort
Copy link
Member

weltenwort commented May 8, 2020

Filebeat version: 7.4.2

The asa-ftd ingest pipeline of the cisco Filebeat module leaves a lot of _temp_.* fields in the ingested documents if the pipeline fails at the wrong processor. This means that the index mapping size grows dramatically due to the dynamic mapping mechanism and which causes problems when querying the cluster state.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 8, 2020
@adriansr adriansr self-assigned this May 8, 2020
adriansr added a commit to adriansr/beats that referenced this issue May 8, 2020
The shared pipeline for Cisco ASA and FTD creates temporary fields under
the _temp_ object. If a failure happens in the middle of the pipeline,
all those fields would be indexed, causing the index mapping to grow too
big.

Fixes elastic#18391
@adriansr adriansr added Team:SIEM and removed needs_team Indicates that the issue/PR needs a Team:* label labels May 8, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

adriansr added a commit that referenced this issue May 13, 2020
The shared pipeline for Cisco ASA and FTD creates temporary fields under
the _temp_ object. If a failure happens in the middle of the pipeline,
all those fields would be indexed, causing the index mapping to grow too
big.

Fixes #18391
adriansr added a commit to adriansr/beats that referenced this issue May 13, 2020
The shared pipeline for Cisco ASA and FTD creates temporary fields under
the _temp_ object. If a failure happens in the middle of the pipeline,
all those fields would be indexed, causing the index mapping to grow too
big.

Fixes elastic#18391

(cherry picked from commit 3605c47)
adriansr added a commit to adriansr/beats that referenced this issue May 13, 2020
The shared pipeline for Cisco ASA and FTD creates temporary fields under
the _temp_ object. If a failure happens in the middle of the pipeline,
all those fields would be indexed, causing the index mapping to grow too
big.

Fixes elastic#18391

(cherry picked from commit 3605c47)
adriansr added a commit to adriansr/beats that referenced this issue May 13, 2020
The shared pipeline for Cisco ASA and FTD creates temporary fields under
the _temp_ object. If a failure happens in the middle of the pipeline,
all those fields would be indexed, causing the index mapping to grow too
big.

Fixes elastic#18391

(cherry picked from commit 3605c47)
adriansr added a commit that referenced this issue May 13, 2020
The shared pipeline for Cisco ASA and FTD creates temporary fields under
the _temp_ object. If a failure happens in the middle of the pipeline,
all those fields would be indexed, causing the index mapping to grow too
big.

Fixes #18391

(cherry picked from commit 3605c47)
adriansr added a commit that referenced this issue May 13, 2020
The shared pipeline for Cisco ASA and FTD creates temporary fields under
the _temp_ object. If a failure happens in the middle of the pipeline,
all those fields would be indexed, causing the index mapping to grow too
big.

Fixes #18391

(cherry picked from commit 3605c47)
adriansr added a commit that referenced this issue May 14, 2020
The shared pipeline for Cisco ASA and FTD creates temporary fields under
the _temp_ object. If a failure happens in the middle of the pipeline,
all those fields would be indexed, causing the index mapping to grow too
big.

Fixes #18391

(cherry picked from commit 3605c47)
leweafan pushed a commit to leweafan/beats that referenced this issue Apr 28, 2023
…ic#18476)

The shared pipeline for Cisco ASA and FTD creates temporary fields under
the _temp_ object. If a failure happens in the middle of the pipeline,
all those fields would be indexed, causing the index mapping to grow too
big.

Fixes elastic#18391

(cherry picked from commit 9c3faed)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants