[Filebeat] [SIEM] Fileset for Cisco FTD logs (#13286)

This adds a new `ftd` fileset to the `cisco` module for parsing
Firepower Threat Defense logs.

As the FTD logs are a superset of the Cisco ASA logs, this PR
introduces a shared ingest pipeline that is used both by the new `ftd`
and the existing `asa` filesets.

As a side effect of this, it improves the existing ASA fileset so that
it uses custom syslog message decoding instead of relying on Filebeat's
syslog input, which has caused compatibility problems with some ASA
devices.

Closes #12690

CHANGELOG.next.asciidoc

@@ -28,6 +28,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 - Add read_buffer configuration option. {pull}11739[11739]
 - `convert_timezone` option is removed and locale is always added to the event so timezone is used when parsing the timestamp, this behaviour can be overriden with processors. {pull}12410[12410]
 - Fix a race condition in the TCP input when close the client socket. {pull}13038[13038]
+- cisco/asa fileset: Renamed log.original to event.original and cisco.asa.list_id to cisco.asa.rule_name. {pull}13286[13286]
 
 *Heartbeat*
 
@@ -284,6 +285,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 - Update PAN-OS fileset to use the ECS NAT fields. {issue}13320[13320] {pull}13330[13330]
 - Add fields to the Zeek DNS fileset for ECS DNS. {issue}13320[13320] {pull}13324[13324]
 - Add container image in Kubernetes metadata {pull}13356[13356] {issue}12688[12688]
+- Add module for ingesting Cisco FTD logs over syslog. {pull}13286[13286]
 
 *Heartbeat*
 

filebeat/docs/fields.asciidoc

@@ -1381,10 +1381,10 @@ type: keyword
 
 --
 
-*`cisco.asa.list_id`*::
+*`cisco.asa.rule_name`*::
 +
 --
-Name of the Access Control List that matched this event.
+Name of the Access Control List rule that matched this event.
 
 
 type: keyword
@@ -1501,6 +1501,184 @@ type: short
 
 --
 
+[float]
+=== ftd
+
+Fields for Cisco Firepower Threat Defense Firewall.
+
+
+
+*`cisco.ftd.message_id`*::
++
+--
+The Cisco FTD message identifier.
+
+
+type: keyword
+
+--
+
+*`cisco.ftd.suffix`*::
++
+--
+Optional suffix after %FTD identifier.
+
+
+type: keyword
+
+example: session
+
+--
+
+*`cisco.ftd.source_interface`*::
++
+--
+Source interface for the flow or event.
+
+
+type: keyword
+
+--
+
+*`cisco.ftd.destination_interface`*::
++
+--
+Destination interface for the flow or event.
+
+
+type: keyword
+
+--
+
+*`cisco.ftd.rule_name`*::
++
+--
+Name of the Access Control List rule that matched this event.
+
+
+type: keyword
+
+--
+
+*`cisco.ftd.source_username`*::
++
+--
+Name of the user that is the source for this event.
+
+
+type: keyword
+
+--
+
+*`cisco.ftd.destination_username`*::
++
+--
+Name of the user that is the destination for this event.
+
+
+type: keyword
+
+--
+
+*`cisco.ftd.mapped_source_ip`*::
++
+--
+The translated source IP address. Use ECS source.nat.ip.
+
+
+type: ip
+
+--
+
+*`cisco.ftd.mapped_source_port`*::
++
+--
+The translated source port. Use ECS source.nat.port.
+
+
+type: long
+
+--
+
+*`cisco.ftd.mapped_destination_ip`*::
++
+--
+The translated destination IP address. Use ECS destination.nat.ip.
+
+
+type: ip
+
+--
+
+*`cisco.ftd.mapped_destination_port`*::
++
+--
+The translated destination port. Use ECS destination.nat.port.
+
+
+type: long
+
+--
+
+*`cisco.ftd.threat_level`*::
++
+--
+Threat level for malware / botnet traffic. One of very-low, low, moderate, high or very-high.
+
+
+type: keyword
+
+--
+
+*`cisco.ftd.threat_category`*::
++
+--
+Category for the malware / botnet traffic. For example: virus, botnet, trojan, etc.
+
+
+type: keyword
+
+--
+
+*`cisco.ftd.connection_id`*::
++
+--
+Unique identifier for a flow.
+
+
+type: keyword
+
+--
+
+*`cisco.ftd.icmp_type`*::
++
+--
+ICMP type.
+
+
+type: short
+
+--
+
+*`cisco.ftd.icmp_code`*::
++
+--
+ICMP code.
+
+
+type: short
+
+--
+
+*`cisco.ftd.security`*::
++
+--
+Raw fields for Security Events.
+
+type: object
+
+--
+
 [float]
 === ios
 

filebeat/docs/modules/cisco.asciidoc

@@ -12,10 +12,12 @@ This file is generated! See scripts/docs_collector.py
 
 beta[]
 
-This is a module for Cisco network device's logs. The `asa` fileset supports
-Cisco ASA firewall logs received over syslog or read from a file. And the `ios`
-fileset supports Cisco IOS router and switch logs received over syslog or read
-from a file.
+This is a module for Cisco network device's logs. It includes the following
+filesets for receiving logs over syslog or read from a file:
+
+- `asa` fileset: supports Cisco ASA firewall logs.
+- `ftd` fileset: supports Cisco Firepower Threat Defense logs.
+- `ios` fileset: supports Cisco IOS router and switch logs.
 
 Cisco ASA devices also support exporting flow records using NetFlow, which is
 supported by the {filebeat-ref}/filebeat-module-netflow.html[netflow module] in
@@ -103,6 +105,148 @@ The UDP port to listen for syslog traffic. Defaults to 9001.
 
 :fileset_ex!:
 
+[float]
+==== `ftd` fileset settings
+
+The Cisco FTD fileset primarily supports parsing IPv4 and IPv6 access list log
+messages similar to that of ASA devices as well as Security Event Syslog
+Messages for Intrusion, Connection, File and Malware events.
+
+*ECS Field mapping*
+
+The `ftd` fileset maps Security Event Syslog Messages to the Elastic Common
+Schema (ECS) format. The following table illustrates the mapping from
+Security Event fields to ECS. The `cisco.ftd` prefix is used when there is no
+corresponding ECS field available.
+
+Mappings for Intrusion events fields:
+[options="header"]
+|====================================
+| FTD Field | Mapped fields
+| ApplicationProtocol | network.protocol
+| DstIP | destination.ip
+| DstPort | destination.port
+| EgressInterface | cisco.ftd.destination_interface
+| GID | service.id
+| HTTPResponse | http.response.status_code
+| IngressInterface | cisco.ftd.source_interface
+| InlineResult | event.outcome
+| IntrusionPolicy | cisco.ftd.rule_name
+| Message | message
+| Protocol | network.transport
+| SrcIP | source.ip
+| SrcPort | source.port
+| User | user.id, user.name
+| WebApplication | network.application
+|====================================
+
+Mappings for Connection and Security Intelligence events fields:
+[options="header"]
+|====================================
+| FTD Field | Mapped fields
+| ACPolicy | cisco.ftd.rule_name
+| AccessControlRuleAction | event.outcome
+| AccessControlRuleName | cisco.ftd.rule_name
+| ApplicationProtocol | network.protocol
+| ConnectionDuration | event.duration
+| DNSQuery | dns.question.name
+| DNSRecordType | dns.question.type
+| DNSResponseType | dns.response_code
+| DstIP | destination.ip
+| DstPort | destination.port
+| EgressInterface | cisco.ftd.destination_interface
+| HTTPReferer | http.request.referrer
+| HTTPResponse | http.response.status_code
+| IngressInterface | cisco.ftd.source_interface
+| InitiatorBytes | source.bytes
+| InitiatorPackets | source.packets
+| NetBIOSDomain | host.hostname
+| Protocol | network.transport
+| ReferencedHost | url.domain
+| ResponderBytes | destination.bytes
+| ResponderPackets | destination.packets
+| SSLActualAction | event.outcome
+| SSLServerName | server.domain
+| SrcIP | source.ip
+| SrcPort | source.port
+| URL | url.original
+| User | user.name
+| UserAgent | user_agent.original
+| WebApplication | network.application
+| originalClientSrcIP | client.ip
+|====================================
+
+Mappings for File and Malware events fields:
+[options="header"]
+|====================================
+| FTD Field | Mapped fields
+| ApplicationProtocol | network.protocol
+| ArchiveFileName | file.name
+| ArchiveSHA256 | file.hash.sha256
+| Client | network.application
+| DstIP | destination.ip
+| DstPort | destination.port
+| FileName | file.name
+| FilePolicy | cisco.ftd.rule_name
+| FileSHA256 | file.hash.sha256
+| FileSize | file.size
+| FirstPacketSecond | event.start
+| Protocol | network.transport
+| SrcIP | source.ip
+| SrcPort | source.port
+| URI | url.original
+| User | user.name
+| WebApplication | network.application
+|====================================
+
+*Example configuration:*
+
+[source,yaml]
+----
+- module: cisco
+  ftd:
+    var.syslog_host: 0.0.0.0
+    var.syslog_port: 9003
+    var.log_level: 5
+----
+
+include::../include/var-paths.asciidoc[]
+
+*`var.log_level`*::
+
+An integer between 1 and 7 that allows to filter messages based on the
+severity level. The different severity levels supported by the Cisco ASA are:
+
+[width="30%",cols="^1,2",options="header"]
+|===========================
+| log_level | severity
+|     1     | Alert
+|     2     | Critical
+|     3     | Error
+|     4     | Warning
+|     5     | Notification
+|     6     | Informational
+|     7     | Debugging
+|===========================
+
+A value of 7 (default) will not filter any messages. A lower value will drop
+any messages with a severity level higher than the specified value. For
+example, `var.log_level: 3` will allow messages of level 1 (Alert), 2 (Critical)
+and 3 (Error). All other messages will be dropped.
+
+*`var.syslog_host`*::
+
+The interface to listen to UDP based syslog traffic. Defaults to localhost.
+Set to 0.0.0.0 to bind to all available interfaces.
+
+*`var.syslog_port`*::
+
+The UDP port to listen for syslog traffic. Defaults to 9003.
+
+:has-dashboards!:
+
+:fileset_ex!:
+
 [float]
 ==== `ios` fileset settings