[Filebeat] Improve ECS field mappings in suricata module #16843
Conversation
leehinman
added
enhancement
Filebeat
|
Pinging @elastic/siem (Team:SIEM) |
- destination.domain - dns.question.top_level_domain - event.category - event.kind - event.outcome - event.type - related.hash - related.ip - rule.category - rule.id - rule.name - tls.client.server_name - tls.resumed - tls.server.certificate - tls.server.certificate_chain - tls.server.hash.sha1 - tls.server.issuer - tls.server.ja3s - tls.server.not_after - tls.server.not_before - tls.server.subject - tls.version - tls.version_protocol Closes elastic#16181
leehinman
force-pushed
the
16181_suricata_ecs_1.4
branch
from
8344241 to
9a86d35
Compare
| ] | ||
| ], | ||
| "tls.client.server_name": "p33-btmmdns.icloud.com.", | ||
| "tls.server.hash.sha1": "6a:ff:ac:a6:5f:8a:05:e7:a9:8c:76:29:b9:08:c7:69:ad:dc:72:47", |
There was a problem hiding this comment.
In ECS we say to use upper-case hex without any colons.
| @@ -186,16 +221,21 @@ | |||
| "dns.id": "12308", | |||
| "dns.response_code": "NOERROR", | |||
| "dns.type": "answer", | |||
| "event.category": "network_traffic", | |||
| "event.category": "network", | |||
There was a problem hiding this comment.
event.category should be an array field.
| "event.dataset": "suricata.eve", | ||
| "event.kind": "event", | ||
| "event.module": "suricata", | ||
| "event.original": "{\"timestamp\":\"2018-07-05T15:51:20.213418-0400\",\"flow_id\":1684780223079543,\"in_iface\":\"en0\",\"event_type\":\"dns\",\"src_ip\":\"192.168.86.1\",\"src_port\":53,\"dest_ip\":\"192.168.86.85\",\"dest_port\":39464,\"proto\":\"UDP\",\"dns\":{\"type\":\"answer\",\"id\":12308,\"rcode\":\"NOERROR\",\"rrname\":\"clients.l.google.com\",\"rrtype\":\"A\",\"ttl\":299,\"rdata\":\"172.217.13.110\"}}", | ||
| "event.type": "protocol", |
There was a problem hiding this comment.
event.type should be an array field
| @@ -350,22 +390,31 @@ | |||
| "destination.address": "17.142.164.13", | |||
| "destination.as.number": 714, | |||
| "destination.as.organization.name": "Apple Inc.", | |||
| "destination.domain": "p33-btmmdns.icloud.com.", | |||
There was a problem hiding this comment.
The trailing period in the DNS name should be removed
| "tls.server.subject": "CN=*.icloud.com/OU=management:idms.group.506364/O=Apple Inc./ST=California/C=US", | ||
| "tls.version": "1.2", | ||
| "tls.version_protocol": "tls", | ||
| "url.domain": "p33-btmmdns.icloud.com." |
There was a problem hiding this comment.
Does this come from the SNI?
I agree that in most cases, TLS is tunneling http. But I don't think Suricata relays what the next/tunneled protocol is, on TLS events. I'd recommend sticking to putting this in destination.domain only, even if most of the time also putting it in url.domain would be accurate.
| evt.AppendTo("event.category", "network"); | ||
| evt.AppendTo("event.category", "intrusion_detection"); | ||
| break; | ||
| case "anomally": |
There was a problem hiding this comment.
| case "anomally": | |
| case "anomaly": |
Typo?
| evt.Put("event.kind", "event"); | ||
| evt.AppendTo("event.category", "network"); | ||
| break; | ||
| case "http": |
There was a problem hiding this comment.
For http, dns, tls, ftp and a whole lot of other IANA protocols reported by Suricata, I'd recommend setting network.protocol explicitly as well. I've noticed the handling of suricata.eve.app_proto below in the pipeline, but that doesn't seem to fill network.protocol reliably. So when suricata.eve.app_proto is empty, I think we should set it explicitly based on the Suricata event type.
The expectation for event.type:protocol is that these events are a protocol-specific breakdown, and the protocol name should be in a standard field as well.
Examples:
{ "event": { "type": ["protocol", ...], ... },
"network": { "transport": "udp", "protocol": "dns" ... },
"dns": { ... }
}{ "event": { "type": ["protocol", ...], ... },
"network": { "transport": "tcp", "protocol": "tls" ... },
"tls": { ... }
}Note: We haven't defined that many protocols per se in ECS yet, though. So outside "http", "dns" and "tls", the protocol breakdown details are fine in custom fields.
- Uppercase and remove : from hash - event.category always an array - event.type always an array - remove trailing period from dns names - don't populate url.domain unless protocol is http - fix "anomally" typo - set network.protocol if suricata has identified the protocol via event_type
leehinman
added
v7.7.0
and removed
needs_backport
* Improve ECS field mappings in suricata module - destination.domain - dns.question.top_level_domain - event.category - event.kind - event.outcome - event.type - related.hash - related.ip - rule.category - rule.id - rule.name - tls.client.server_name - tls.resumed - tls.server.certificate - tls.server.certificate_chain - tls.server.hash.sha1 - tls.server.issuer - tls.server.ja3s - tls.server.not_after - tls.server.not_before - tls.server.subject - tls.version - tls.version_protocol Closes elastic#16181 (cherry picked from commit 7eb2fba)
…6990) * Improve ECS field mappings in suricata module - destination.domain - dns.question.top_level_domain - event.category - event.kind - event.outcome - event.type - related.hash - related.ip - rule.category - rule.id - rule.name - tls.client.server_name - tls.resumed - tls.server.certificate - tls.server.certificate_chain - tls.server.hash.sha1 - tls.server.issuer - tls.server.ja3s - tls.server.not_after - tls.server.not_before - tls.server.subject - tls.version - tls.version_protocol Closes #16181 (cherry picked from commit 7eb2fba)
Closes #16181