diff --git a/rfcs/text/0008-threat-intel.md b/rfcs/text/0008-threat-intel.md index 1e12232415..90b0584459 100644 --- a/rfcs/text/0008-threat-intel.md +++ b/rfcs/text/0008-threat-intel.md @@ -1,7 +1,7 @@ # 0008: Cyber Threat Intelligence Fields -- Stage: **0 (strawperson)** +- Stage: **1 (proposal)** - Date: **2020-10-15** Elastic Security Solution will be adding the capability to ingest, process and utilize threat intelligence information for increasing detection coverage and helping analysts making quicker investigation decisions. Threat intelligence can be collected from a number of sources with a variety of structured and semi-structured data representations. This makes threat intelligence an ideal candidate for ECS mappings. Threat intelligence data will require ECS mappings to normalize it and make it usable in our security solution. This RFC is focused on identifying new field sets and values that need to be created for threat intelligence data. Existing ECS field reuse will be prioritized where possible. If new fields are required we will utilize [STIX Cyber Observable data model](https://docs.oasis-open.org/cti/stix/v2.1/cs01/stix-v2.1-cs01.html#_mlbmudhl16lr) as guidance. @@ -21,6 +21,46 @@ Stage 0: Provide a high level summary of the premise of these changes. Briefly d Stage 1: Describe at a high level how this change affects fields. Which fieldsets will be impacted? How many fields overall? Are we primarily adding fields, removing fields, or changing existing fields? The goal here is to understand the fundamental technical implications and likely extent of these changes. ~2-5 sentences. --> +### Proposed New Fields for Threat fieldset + + * threat.time_first_seen + _The date and time when intelligence souce first reported sighting this indicator._ + * threat.time_last_seen + _The date and time when intelligence source last reported sighting this indicator._ + * threat.sightings + _Number of times this indicator was observed conducting threat activity._ + * threat.type + _Type of indicator as reprsented by Cyber Observable in STIX 2.0_ + * threat.description + _Describes the type of action conducted by the threat._ + * threat.tlp + _Traffic Light Protocol, which dictates sharing policies_ + * threat.classification + _Describes type of threat delivery (Hacktool etc.) and family name. + * threat.scanner_stats + _Count of Anti virus/EDR that successfully detected malicious file or URL. Sources like VirusTotal, Reversing Labs often provide these statistics._ + +### Proposed New Values + + * event.kind:enrichment _Propose adding this value to capture outcome of this event. It could also appy to other types of contextual data such as directory services, IPAM data, asset lists._ + * event.category:threat _Proposed threat.type field would be a subcategory for this value of event.category_ + * event.type:indicator _Proposed value represents type of threat information. In future this could be extended to other STIX 2.0 Standard Data Objects like Actor, Infrastucture etc._ + +### Existing ECS Fields For Nested Use in Threat.* + + * file.* + * file.hash.* + * url.* + * user.* + * registry.* + * as.* + * host.* + * network.* + * x509.* + * pe.* + * source.* + * destination.* + @@ -35,12 +75,41 @@ Stage 3: Add or update all remaining field definitions. The list should now be e Stage 1: Describe at a high-level how these field changes will be used in practice. Real world examples are encouraged. The goal here is to understand how people would leverage these fields to gain insights or solve problems. ~1-3 paragraphs. --> +The additions to threat.* fields and nested use will be used to represent data collected threat intelligence sources. A new rule type Indicator match will be introduced in 7.10 and ECS threat fields will enable a new category of detection alerts that matches incoming log and event data against threat intelligence sources and generates an alert. Additionally in the future we will be able to enrich events and alerts with context from threat intelligence to assist analysts in their investigative workflows. + ## Source data +There are many sources of threat intelligence including open source, closed source and membership based ISAC's. Depending on the source the level of details can vary from atomic indicators of compromise (IoC) to higher order context around threat tactics, infrastructure and motivations. Generally freely available (open source) intelligence sources will provide details more focused on IoC's and commercial intelligence services will provide higher order details. + +These sources typically provide intelligence that can be downloaded through REST API or in some cases downloadable CSV's or text files. These intelligence sources will update their data repositories at varying intervals. + +Some examples of open source intelligence are: + * [Abuse.ch Feodo Tracker](https://feodotracker.abuse.ch/downloads/ipblocklist.csv) - see below for sample data + * [Phish Tank](https://www.phishtank.com/) + +Some examples of commercial intelligence include: + * [Anomali ThreatStream](https://www.anomali.com/products/threatstream) + * [Virus Total](https://www.virustotal.com/gui/intelligence-overview) + * [Domain Tools](https://www.domaintools.com/products/api-integration/) + +#### Abuse.ch Feodo Tracker + +``` +# Firstseen,DstIP,DstPort,LastOnline,Malware +2020-10-29 19:16:38,181.120.29.49,80,2020-11-02,Heodo +2020-10-29 19:16:35,190.45.24.210,80,2020-11-02,Heodo +2020-10-29 19:16:32,109.242.153.9,80,2020-11-02,Heodo +2020-10-29 19:16:28,169.1.39.242,80,2020-11-02,Heodo +2020-10-29 19:14:24,201.171.244.130,80,2020-11-02,Heodo +2020-10-29 19:14:20,64.207.182.168,8080,2020-11-02,Heodo +2020-10-29 19:14:19,173.173.254.105,80,2020-11-02,Heodo +2020-10-29 19:14:16,153.204.122.254,80,2020-10-30,Heodo +2020-10-29 19:14:13,201.163.74.203,80,2020-11-02,Heodo +``` @@ -64,6 +133,7 @@ The goal here is to research and understand the impact of these changes on users +There is a proposal to nest all IoC fields under `threat.ioc.*` instead of the current `threat.* structure.` This would make it consistent with taxonomy structure for `threat.tactic.*` and `threat.techinique.*` . This needs to be resovled in Stage 2 of the RFC process. +* @shimonmodi | author +* @peasead | subject matter expert +* @MikePaquette | subject matter expert ## References @@ -112,6 +172,7 @@ e.g.: * Stage 0: https://github.com/elastic/ecs/pull/986 +* Stage 1: https://github.com/elastic/ecs/pull/1037