Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve logging by log separation for better troubleshooting purposes #22414

Closed
111andre111 opened this issue Nov 4, 2020 · 4 comments
Closed
Labels
enhancement libbeat Stalled Team:Services (Deprecated) Label for the former Integrations-Services team

Comments

@111andre111
Copy link
Contributor

Describe the enhancement:
I would like to have a better segregated logging in filebeat, metricbeat,... per module/input/output

Describe a specific use case for the enhancement or feature:
Sometimes it happens that we have more complex filebeat configuration files with lots of enabled inputs and lots of enabled modules it can sometimes get very tricky even with enabled DEBUG logging to turn down certain issues where they are really located when it comes to logical configuration issues

Looking away to logstash there are really helpful tools for troubleshooting this.

  1. Since if I recall correctly 7.4 we always have a prefix in the logstash logs, that always log as well the pipeline where a certain event was happening.
  2. You additionally have the possibility to segregate the logs per pipeline using setting pipeline.separate_logs
    https://www.elastic.co/guide/en/logstash/current/logstash-settings-file.html

Now having this said, can't we have a similar functionality in the beats logs, that tell when something happened in an input/output/processor/module which one really was it?
Can we separate these log files per input/output/module as well. I think that should help for such issues to take the burden a little bit away from troubleshooting knowing exactly where certain things are happening?

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Nov 4, 2020
@kvch kvch added the Team:Services (Deprecated) Label for the former Integrations-Services team label Nov 4, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Nov 4, 2020
@111andre111 111andre111 changed the title improve logging for better troubleshooting purposes improve logging by log separation for better troubleshooting purposes Nov 4, 2020
@andresrc
Copy link
Contributor

Related: #15699

@botelastic
Copy link

botelastic bot commented Jan 27, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@botelastic botelastic bot added the Stalled label Jan 27, 2022
@jlind23
Copy link
Collaborator

jlind23 commented Mar 31, 2022

Backlog grooming: Closing for now.

@jlind23 jlind23 closed this as completed Mar 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement libbeat Stalled Team:Services (Deprecated) Label for the former Integrations-Services team
Projects
None yet
Development

No branches or pull requests

5 participants