[Filebeat] Add Zeek Signatures fileset #23772

legoguy1000 · 2021-01-30T22:03:02Z

What does this PR do?

Add the Signature fileset to the Zeek module for Filbeat.

Why is it important?

Its one of the last zeek logs that isn't parsed by Filebeat currently.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

cla-checker-service · 2021-01-30T22:03:05Z

💚 CLA has been signed

elasticmachine · 2021-01-30T22:08:23Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #23772 updated
Start Time: 2021-02-16T21:01:49.762+0000
Duration: 49 min 39 sec
Commit: 1ca05f2

Test stats 🧪

Test	Results
Failed	0
Passed	13046
Skipped	2075
Total	15121

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	13046
Skipped	2075
Total	15121

elasticmachine · 2021-02-06T20:26:28Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

legoguy1000 · 2021-02-06T20:35:56Z

I tried to do the local testing per the documentation but i don't know if I did it correctly and think someone with more beats module dev experience should look at this and make whatever changes are needed.

andrewstucki

Thanks for the contribution!

I left a suggestion about a configuration change you likely were intending.

Additionally you'll also want to revert the Makefile change, change the top-level zeek/_meta/config.yml file that I mention, and run the mage commands for generating the expected document and the generated configuration files.

Let me know if you have any questions or need some help with running the generators.

x-pack/filebeat/Makefile

x-pack/filebeat/module/zeek/signature/config/signature.yml

x-pack/filebeat/module/zeek/signature/test/signature-json.log-expected.json

x-pack/filebeat/modules.d/zeek.yml.disabled

x-pack/filebeat/module/zeek/signature/test/signature-json.log

x-pack/filebeat/module/zeek/signature/config/signature.yml

andrewkroh · 2021-02-10T01:43:00Z

jenkins, run tests

andrewkroh

Please add a changelog entry into the CHANGELOG.next.asciidoc file under the Added/Filebeat section.

legoguy1000 · 2021-02-10T13:56:11Z

Please add a changelog entry into the CHANGELOG.next.asciidoc file under the Added/Filebeat section.

I will try to get to this today

legoguy1000 · 2021-02-10T17:14:18Z

Please add a changelog entry into the CHANGELOG.next.asciidoc file under the Added/Filebeat section.

I will try to get to this today

Done

andrewkroh · 2021-02-12T04:44:10Z

jenkins, run tests

legoguy1000 · 2021-02-16T14:08:21Z

@andrewkroh can u rerun the Jenkins tests?

andrewkroh · 2021-02-16T14:48:55Z

jenkins, run tests

andrewkroh

LGTM assuming CI is green.

Add the Signature fileset to the Zeek module for Filbeat. Co-authored-by: Andrew Kroh <[email protected]> (cherry picked from commit e332d9d)

andrewkroh · 2021-02-17T00:00:10Z

Thanks for your contribution. I opened a PR to move this into the 7.x branch so that it's included in the next minor release.

Add the Signature fileset to the Zeek module for Filbeat. Co-authored-by: Andrew Kroh <[email protected]> (cherry picked from commit e332d9d) Co-authored-by: Alex Resnick <[email protected]>

…-arm * upstream/master: [CI] install docker-compose with retry (elastic#24069) Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052) updating manifest files for filebeat threatintel module (elastic#24074) Add Zeek Signatures (elastic#23772) Update Beats to ECS 1.8.0 (elastic#23465) Support running Docker logging plugin on ARM64 (elastic#24034) Fix ec2 metricset fields.yml and add integration test (elastic#23726) Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060) [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773) [Elastic Agent] Enroll with Fleet Server (elastic#23865) [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944) [Ingest Management] Fix reloading of log level for services (elastic#24055) Add Agent standalone k8s manifest (elastic#23679)

…dows-7 * upstream/master: (332 commits) Use ECS v1.8.0 (elastic#24086) Add support for postgresql csv logs (elastic#23334) [Heartbeat] Refactor config system (elastic#23467) [CI] install docker-compose with retry (elastic#24069) Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052) updating manifest files for filebeat threatintel module (elastic#24074) Add Zeek Signatures (elastic#23772) Update Beats to ECS 1.8.0 (elastic#23465) Support running Docker logging plugin on ARM64 (elastic#24034) Fix ec2 metricset fields.yml and add integration test (elastic#23726) Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060) [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773) [Elastic Agent] Enroll with Fleet Server (elastic#23865) [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944) [Ingest Management] Fix reloading of log level for services (elastic#24055) Add Agent standalone k8s manifest (elastic#23679) [Metricbeat][Kubernetes] Extend state_node with more conditions (elastic#23905) [CI] googleStorageUploadExt step (elastic#24048) Check fields are documented for aws metricsets (elastic#23887) Update go-concert to 0.1.0 (elastic#23770) ...

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 30, 2021

legoguy1000 changed the title ~~init~~ [Filebeat] Add Zeek Signatures fileset Jan 30, 2021

legoguy1000 force-pushed the zeek-signatures branch from e814af5 to 0b632a2 Compare January 30, 2021 22:36

andresrc added the Team:Security-External Integrations label Jan 31, 2021

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 31, 2021

andrewkroh added enhancement Filebeat Filebeat labels Feb 1, 2021

legoguy1000 marked this pull request as ready for review February 6, 2021 20:26

legoguy1000 force-pushed the zeek-signatures branch from 0b632a2 to cd2c30e Compare February 6, 2021 20:34

andrewstucki suggested changes Feb 8, 2021

View reviewed changes

legoguy1000 force-pushed the zeek-signatures branch from cd2c30e to 3e4e1ec Compare February 8, 2021 18:27

andrewkroh reviewed Feb 10, 2021

View reviewed changes

legoguy1000 force-pushed the zeek-signatures branch 2 times, most recently from 89578a9 to 9e80f28 Compare February 10, 2021 16:49

legoguy1000 force-pushed the zeek-signatures branch from 9e80f28 to 8e7a3a7 Compare February 12, 2021 03:46

legoguy1000 force-pushed the zeek-signatures branch 2 times, most recently from cdcc1e5 to db21eda Compare February 16, 2021 14:05

andrewkroh approved these changes Feb 16, 2021

View reviewed changes

Add Zeek Signatures

bf2c64d

legoguy1000 force-pushed the zeek-signatures branch from db21eda to bf2c64d Compare February 16, 2021 16:52

Merge branch 'master' into zeek-signatures

1ca05f2

andrewkroh requested a review from andrewstucki February 16, 2021 23:56

andrewkroh merged commit e332d9d into elastic:master Feb 16, 2021

andrewkroh mentioned this pull request Feb 16, 2021

Cherry-pick #23772 to 7.x: [Filebeat] Add Zeek Signatures fileset #24078

Merged

6 tasks

andrewkroh added the v7.12.0 label Feb 16, 2021

andrewkroh pushed a commit to andrewkroh/beats that referenced this pull request Feb 16, 2021

Add Zeek Signatures (elastic#23772)

68bddb5

Add the Signature fileset to the Zeek module for Filbeat. Co-authored-by: Andrew Kroh <[email protected]> (cherry picked from commit e332d9d)

legoguy1000 deleted the zeek-signatures branch February 24, 2021 23:05

111andre111 mentioned this pull request May 6, 2022

Additional zeek module log files elastic/integrations#3288

Closed

28 tasks

This was referenced Aug 9, 2021

Zeek Integration missing elastic/integrations#1461

Closed

[Zeek] Add NTP and Signature data streams elastic/integrations#1515

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Filebeat] Add Zeek Signatures fileset #23772

[Filebeat] Add Zeek Signatures fileset #23772

legoguy1000 commented Jan 30, 2021 •

edited by andrewkroh

Loading

cla-checker-service bot commented Jan 30, 2021 •

edited

Loading

elasticmachine commented Jan 30, 2021 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

Trends 🧪

Test stats 🧪

elasticmachine commented Feb 6, 2021

legoguy1000 commented Feb 6, 2021

andrewstucki left a comment

andrewkroh commented Feb 10, 2021

andrewkroh left a comment

legoguy1000 commented Feb 10, 2021

legoguy1000 commented Feb 10, 2021

andrewkroh commented Feb 12, 2021

legoguy1000 commented Feb 16, 2021

andrewkroh commented Feb 16, 2021

andrewkroh left a comment

andrewkroh commented Feb 17, 2021

[Filebeat] Add Zeek Signatures fileset #23772

[Filebeat] Add Zeek Signatures fileset #23772

Conversation

legoguy1000 commented Jan 30, 2021 • edited by andrewkroh Loading

What does this PR do?

Why is it important?

Checklist

cla-checker-service bot commented Jan 30, 2021 • edited Loading

elasticmachine commented Jan 30, 2021 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

💚 Flaky test report

Test stats 🧪

elasticmachine commented Feb 6, 2021

legoguy1000 commented Feb 6, 2021

andrewstucki left a comment

Choose a reason for hiding this comment

andrewkroh commented Feb 10, 2021

andrewkroh left a comment

Choose a reason for hiding this comment

legoguy1000 commented Feb 10, 2021

legoguy1000 commented Feb 10, 2021

andrewkroh commented Feb 12, 2021

legoguy1000 commented Feb 16, 2021

andrewkroh commented Feb 16, 2021

andrewkroh left a comment

Choose a reason for hiding this comment

andrewkroh commented Feb 17, 2021

legoguy1000 commented Jan 30, 2021 •

edited by andrewkroh

Loading

cla-checker-service bot commented Jan 30, 2021 •

edited

Loading

elasticmachine commented Jan 30, 2021 •

edited by jenkins-beats-ci bot

Loading