Cisco FTD issues parsing Security Event messages

[adriansr]

The shared Cisco ASA/FTD pipeline doesn't parse the following message correctly:

2020-03-01T01:02:36Z CISCO-SENSOR-3D Alerts %NGIPS-0-430003: DeviceUUID: 1c8ff662-08f3-11e4-85c0-bc960372972f, AccessControlRuleAction: Allow, AccessControlRuleReason: IP Monitor, SrcIP: 3.3.3.3, DstIP: 2.2.2.2, SrcPort: 65090, DstPort: 80, Protocol: tcp, IngressInterface: s1p1, EgressInterface: s1p2, IngressZone: Inside-DMZ-Interface-Inline, EgressZone: Inside-DMZ-Interface-Inline, ACPolicy: COOL-POLICY-3D, AccessControlRuleName: Inside DMZ-Rule-Inline, Prefilter Policy: Unknown, User: No Authentication Required, UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36, Client: Chrome, ClientVersion: 80.0.3987.87, ApplicationProtocol: HTTP, ConnectionDuration: 20, InitiatorPackets: 4, ResponderPackets: 4, InitiatorBytes: 729, ResponderBytes: 246, NAPPolicy: State-Backbone, SecIntMatchingIP: Destination, IPReputationSICategory: Malware, HTTPReferer: http://eyedropper-color-pick.info/mk?c=1581483445764, ReferencedHost: eyedropper-color-pick.info, URL: http://bad-malwaresite-grr.info/favicon.ico

The output is:

[
    {
        "@timestamp": "2020-02-29T23:02:00.000-02:00",
        "cisco.ftd.destination_interface": "s1p2",
        "cisco.ftd.message_id": "430002",
        "cisco.ftd.rule_name": [
            "COOL-POLICY-3D",
            "Inside DMZ-Rule-Inline"
        ],
        "cisco.ftd.security.ac_policy": "COOL-POLICY-3D",
        "cisco.ftd.security.access_control_rule_action": "Allow",
        "cisco.ftd.security.access_control_rule_name": "Inside DMZ-Rule-Inline",
        "cisco.ftd.security.access_control_rule_reason": "IP Monitor",
        "cisco.ftd.security.dst_ip": "2.2.2.2",
        "cisco.ftd.security.dst_port": "80",
        "cisco.ftd.security.egress_interface": "s1p2",
        "cisco.ftd.security.egress_zone": "Inside-DMZ-Interface-Inline",
        "cisco.ftd.security.ingress_interface": "s1p1",
        "cisco.ftd.security.ingress_zone": "Inside-DMZ-Interface-Inline",
        "cisco.ftd.security.prefilter_policy": "Unknown",
        "cisco.ftd.security.protocol": "tcp",
        "cisco.ftd.security.src_ip": "3.3.3.3",
        "cisco.ftd.security.src_port": "65090",
        "cisco.ftd.security.user": "No Authentication Required",
        "cisco.ftd.security.user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML",
        "cisco.ftd.source_interface": "s1p1",
        "destination.address": "2.2.2.2",
        "destination.as.number": 3215,
        "destination.as.organization.name": "Orange",
        "destination.geo.continent_name": "Europe",
        "destination.geo.country_iso_code": "FR",
        "destination.geo.location.lat": 48.8582,
        "destination.geo.location.lon": 2.3387,
        "destination.ip": "2.2.2.2",
        "destination.port": 80,
        "event.action": "connection-started",
        "event.code": 430002,
        "event.dataset": "cisco.ftd",
        "event.module": "cisco",
        "event.original": "36Z CISCO-SENSOR-3D Alerts %NGIPS-0-430003: DeviceUUID: 1c8ff662-08f3-11e4-85c0-bc960372972f, AccessControlRuleAction: Allow, AccessControlRuleReason: IP Monitor, SrcIP: 3.3.3.3, DstIP: 2.2.2.2, SrcPort: 65090, DstPort: 80, Protocol: tcp, IngressInterface: s1p1, EgressInterface: s1p2, IngressZone: Inside-DMZ-Interface-Inline, EgressZone: Inside-DMZ-Interface-Inline, ACPolicy: COOL-POLICY-3D, AccessControlRuleName: Inside DMZ-Rule-Inline, Prefilter Policy: Unknown, User: No Authentication Required, UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36, Client: Chrome, ClientVersion: 80.0.3987.87, ApplicationProtocol: HTTP, ConnectionDuration: 20, InitiatorPackets: 4, ResponderPackets: 4, InitiatorBytes: 729, ResponderBytes: 246, NAPPolicy: State-Backbone, SecIntMatchingIP: Destination, IPReputationSICategory: Malware, HTTPReferer: http://eyedropper-color-pick.info/mk?c=1581483445764, ReferencedHost: eyedropper-color-pick.info, URL: http://bad-malwaresite-grr.info/favicon.ico",
        "event.outcome": "allow",
        "event.severity": 7,
        "event.timezone": "-02:00",
        "fileset.name": "ftd",
        "input.type": "log",
        "log.level": "debug",
        "log.offset": 0,
        "network.iana_number": 6,
        "network.transport": "tcp",
        "service.type": "cisco",
        "source.address": "3.3.3.3",
        "source.geo.city_name": "Seattle",
        "source.geo.continent_name": "North America",
        "source.geo.country_iso_code": "US",
        "source.geo.location.lat": 47.6348,
        "source.geo.location.lon": -122.3451,
        "source.geo.region_iso_code": "US-WA",
        "source.geo.region_name": "Washington",
        "source.ip": "3.3.3.3",
        "source.port": 65090,
        "tags": [
            "cisco-ftd"
        ],
        "user.id": "No Authentication Required",
        "user.name": "No Authentication Required",
        "user_agent.original": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML"
    }
]

There's three bugs to fix and one improvement in to do:

1. (bug) Date seconds not correctly parsed. ("event.original": "36Z [...").
2. (improvement) The message label %NGIPS-0-430003 is not understood because it doesn't start with the %ASA nor %FTD prefix. The pipeline needs to be extended to support arbitrary prefixes.
3. (bug) The missing %FTD header triggers event-type-ID detection code for compatibility with FTD versions older than 6.3. This is not working correctly as it detects the message as type 430002 instead of 430003.
4. (bug) IPReputationSICategory and the URL fields are not extracted from the message.

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)