Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow custom tags to be specified at the point of log file ingestion (ie., filebeat) #463

Closed
mmguero opened this issue Apr 23, 2024 · 2 comments
Assignees
Labels
beats Relating to Malcolm's use of Beats enhancement New feature or request
Milestone

Comments

@mmguero
Copy link
Collaborator

mmguero commented Apr 23, 2024

At the point where logs are collected (by filebeat) and forwarded along (to logstash), we're already using adding tags at the source.

We'd like to add a custom setting (environment variable) for adding additional user-defined tags. This would allow for an easy way to do custom user-defined groupings and labels of traffic.

@mmguero mmguero added beats Relating to Malcolm's use of Beats enhancement New feature or request falcon labels Apr 23, 2024
@mmguero mmguero added this to the v24.05.0 milestone Apr 23, 2024
@mmguero mmguero added this to Malcolm Apr 23, 2024
@mmguero mmguero moved this to Todo (develop) in Malcolm Apr 23, 2024
@mmguero mmguero removed the falcon label May 7, 2024
@mmguero mmguero self-assigned this May 15, 2024
@mmguero mmguero moved this from Todo (develop) to In Progress (external) in Malcolm May 15, 2024
@mmguero mmguero moved this from In Progress (external) to In Progress in Malcolm May 15, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue May 16, 2024
@mmguero mmguero moved this from In Progress to Testing in Malcolm May 16, 2024
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue May 16, 2024
@mmguero
Copy link
Collaborator Author

mmguero commented May 16, 2024

On hedgehog for configuration:

Image

Image

And the tags being applied to zeek, suricata, and arkime logs:

Image

I think the only thing I have left to do is some sanitizing of the inputs for the tags (figure out what characters can/can't be allowed) and it's done.

@mmguero mmguero moved this from Testing to In Progress in Malcolm May 20, 2024
@mmguero
Copy link
Collaborator Author

mmguero commented May 20, 2024

Need to handle this at the Malcolm level as well (not just hedgehog)

mmguero added a commit to mmguero-dev/Malcolm that referenced this issue May 20, 2024
@mmguero mmguero moved this from In Progress to Done in Malcolm May 21, 2024
@mmguero mmguero moved this from Done to Testing in Malcolm May 21, 2024
@mmguero mmguero moved this from Testing to Done in Malcolm May 28, 2024
@mmguero mmguero closed this as completed May 28, 2024
This was referenced May 29, 2024
@mmguero mmguero moved this from Done to Released in Malcolm May 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
beats Relating to Malcolm's use of Beats enhancement New feature or request
Projects
Status: Released
Development

No branches or pull requests

1 participant