Fortinet Filebeat Module #17890
Fortinet Filebeat Module #17890
Conversation
P1llus
added
enhancement
in progress
|
Pinging @elastic/siem (Team:SIEM) |
P1llus
changed the title
P1llus
added
review
and removed
in progress
…conds to nano, added geo for nat IP's, updated utm pipeline condition to be backwards compatible, updated session_id description. Updated documentation with references
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
|
There was a problem hiding this comment.
Wow, this is great.
a few suggestions on mappings, and overall it would be good to add all the users to "related.user", also I see in the docs some hashes and those should probably be copied to "related.hash" as well.
And one completely optional questions. Would it be easier to do the some of the mappings if we had a large switch statement based on the logid? This would look a lot like the Winlogbeat security fileset.
x-pack/filebeat/module/fortinet/firewall/test/fortinet.log-expected.json
Outdated
Show resolved
Hide resolved
…the starting pipeline, added some new ECS mapping, added requested changes from PR comments
…ing to add a field mapping for file.hash.crc32 upon comment on slack
|
Okay phew, going to try to summarise the latest changes.
Added the documentation changes from @andrewkroh |
|
jenkins, test this please |
CHANGELOG.next.asciidoc
Outdated
| @@ -281,6 +281,8 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d | |||
| - Enhance `elasticsearch/slowlog` fileset to handle ECS-compatible logs emitted by Elasticsearch. {issue}17715[17715] {pull}17729[17729] | |||
| - Improve ECS categorization field mappings in misp module. {issue}16026[16026] {pull}17344[17344] | |||
| - Added Unix stream socket support as an input source and a syslog input source. {pull}17492[17492] | |||
| - Added new Fortigate Syslog filebeat module. {pull}17890[17890] | |||
| - Improve ECS categorization field mappings for nginx module. {issue}16174[16174] {pull}17844[17844] | |||
There was a problem hiding this comment.
Is this non-related entry on purpose?
| - name: tags | ||
| default: [fortinet-firewall] | ||
| - name: syslog_port | ||
| default: 9001 |
There was a problem hiding this comment.
This default already collides with other 4 filesets. Can you change it to 9004 which is unused?
| default: 9001 | ||
| - name: input | ||
| default: syslog | ||
| - name: log_level |
There was a problem hiding this comment.
Seems to me that this variable is not used, just carried over from using Cisco's module as a base.
| # Set the log level from 1 (alerts only) to 7 (include all messages). | ||
| # Messages with a log level higher than the specified will be dropped. | ||
| # See https://www.cisco.com/c/en/us/td/docs/security/asa/syslog/b_syslog/syslogs-sev-level.html | ||
| #var.log_level: 7 |
There was a problem hiding this comment.
Also this is unused and references Cisco ASA docs.
…s and removed unused ones
|
@adriansr I have added your changes, if you could take a look if they match? For some reason nosetests does not work anymore after pulling in the new master, I did test them also without any modifications and they still fail for some reason. Did work on all earlier commits as I always run it before a commit. There is alao the thing withthe changelog, this seems to have been included after a merge, i removed the entry but that might ve the wrong approach here? |
|
The tests are working for me 👍 |
💔 Build Failed
Expand to view the summary
Build stats
Test stats 🧪
Steps errorsExpand to view the steps failures
Log outputExpand to view the last 100 lines of log output
|
Got them to run now, forgot to add MODULE_PATH env. Everything else looks okay @adriansr ? If you look at the changes to CHANGELOG.next.asciidoc, I do believe that it is the wrong approach correct? |
| "fortinet-firewall" | ||
| ], | ||
| "url.domain": "elastic.co", | ||
| "url.path": "https://example.com/" |
There was a problem hiding this comment.
What is currently in url.path I would expect to be in url.full or url.original since it looks like a full url. For url.path I would expect "/" with a url.full of "https://example.com/". Also it still looks like the url.domain and what is the full url don't match. For a url.full of "https://example.com/" I would expect url.domain to be "example.com"
There was a problem hiding this comment.
This was a typo in my dataset and not the mapping, fixing it now
This PR Introduces the Fortinet filebeat module. Focusing currently on Fortigate Firewall, but should include other Fortinet products as separate PR's later on. Many thanks to the continuous support from @enotspe and the project https://github.com/enotspe/fortinet-2-elasticsearch helping out with feedback, comments, documentation and much more! (cherry picked from commit bc39eb8)
This PR Introduces the Fortinet filebeat module. Focusing currently on Fortigate Firewall, but should include other Fortinet products as separate PR's later on. Many thanks to the continuous support from @enotspe and the project https://github.com/enotspe/fortinet-2-elasticsearch helping out with feedback, comments, documentation and much more! (cherry picked from commit bc39eb8)
|
I will piggy back on @enotspe comment above. Please add the logic to get url.full when the log type is utm and sub type is virus, instead of using url.path. Let us know if we need to clarify. |
|
Hello! DNS requests can include multiple IP addresses in the form of
Fabricated log line for testing:
|
|
@P1llus You said other products from fortinet will be available later on. Any idea when they will be available please ? I would really appreciate to get this information, even if it's an approximate date. Thank you in advance for your response Sir. |
|
@sidahmed-malaoui Unfortunately there is no current way to say this. Though I have created issues for some of them, and you are always free to follow those issues to get any information on updates. If there is a specific product missing, you can create your own issue and request it and we can see if there is any possibilities to add it in. As always there is no real promise or timeline on this, it was rather an explanation that if we are to support multiple fortinet products then it would be in a separate fileset. There is also quite some documentation around fortisandbox and fortiweb, but i'm still researching if those will be possible |
There are some logstash pipelines for fortimail and fortiweb if that helps |
What does this PR do?
This PR Introduces the Fortinet filebeat module. Focusing currently on Fortigate Firewall, but should include other Fortinet products as separate PR's later on.
Many thanks to the continuous support from @enotspe and the project https://github.com/enotspe/fortinet-2-elasticsearch helping out with feedback, comments, documentation and much more!
Why is it important?
Adding more supported products to the filebeat portfolio.
Checklist
CHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Nosetest passes with:
INTEGRATION_TESTS=1 BEAT_STRICT_PERMS=false TESTING_FILEBEAT_MODULES=fortinet nosetests -v -s tests/system/test_xpack_modules.pyRelated issues