Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Expand OSINT, add threat_intelligence #1255

Closed
wants to merge 15 commits into from
Closed

Expand OSINT, add threat_intelligence #1255

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
53f9722
add new `osint_type_id`
jonrau-at-queryai Nov 19, 2024
b80e201
`osint.type_id` description updates
jonrau-at-queryai Nov 19, 2024
b0c0dc7
alphabetizing the `osint` object
jonrau-at-queryai Nov 19, 2024
63686b7
add `related_analytics`, `name`, `uid`
jonrau-at-queryai Nov 19, 2024
f74fa37
constraints & requirements update
jonrau-at-queryai Nov 19, 2024
9c1dfb1
stage `threat_intelligence`
jonrau-at-queryai Nov 19, 2024
28b2099
adding more attributes to threat intelligence
jonrau-at-queryai Nov 19, 2024
8297023
add `threat_intelligence` to dictionary
jonrau-at-queryai Nov 19, 2024
2ae85ce
add to `osint`, update `threat_intelligence` attrs
jonrau-at-queryai Nov 19, 2024
19fe08e
add first/last seen to osint and threat intel
jonrau-at-queryai Nov 19, 2024
a165a62
finish `threat_intelligence`
jonrau-at-queryai Nov 19, 2024
8c4301f
Add `comment`
jonrau-at-queryai Nov 19, 2024
4487e7c
Update CHANGELOG.md
jonrau-at-queryai Nov 19, 2024
c2a15e3
Update CHANGELOG.md
jonrau-at-queryai Nov 19, 2024
8ac137c
Merge branch 'main' into osint_additions
jonrau-at-queryai Nov 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ Thankyou! -->
1. Added new `11: Basic Authentication` enum value to `auth_protocol_id`. #1239
1. Added `values` as an array of `string_t`. #1251
1. Added `kernel_release` as a `string_t`.
1. Added `objective` as a `string_t` to use in the new `threat_intelligence` object. #1255
* #### Objects
1. Added `environment_variable` object. #1172
1. Added `advisory` object. #1176
Expand All @@ -83,6 +84,7 @@ Thankyou! -->
1. Added `discovery_details`, `encryption_details`, `occurrence_details` objects. #1245
1. Added `scim` object. #1239
1. Added `sso` object. #1239
1. Added `threat_intelligence` object. #1255

### Improved
* #### Event Classes
Expand Down Expand Up @@ -127,6 +129,7 @@ Thankyou! -->
1. Added `hostname`, `ip`, and `name` to `resource_details` for purposes of assigning an Observable number. #1250
1. Added `values` to `key_value_object`. #1251
1. Added `kernel_release` to `os` object.
1. Added `first_seen_at`, `last_seen_at`, `related_analytics`, and `threat_intelligence` to `osint`. Additionally, added new enums (Registry Key, Registry Value, and Command Line) to `osint.type_id`. #1255

### Bugfixes
1. Added sibling definition to `confidence_id` in dictionary, accurately associating `confidence` as its sibling. #1180
Expand Down
10 changes: 10 additions & 0 deletions dictionary.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3232,6 +3232,11 @@
"description": "The number of the entity. See specific usage.",
"type": "integer_t"
},
"objective": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Term very general. Suggest: threat_objective

"caption": "Objective",
"description": "The primary goal, objective, desired outcome, or intended effect of a campaign or threat actor. See specific usage.",
"type": "string_t"
},
"observables": {
"caption": "Observables",
"description": "The observables associated with the event or a finding.",
Expand Down Expand Up @@ -4833,6 +4838,11 @@
"description": "The time when the entity was terminated. See specific usage.",
"type": "timestamp_t"
},
"threat_intelligence": {
"caption": "Threat Intelligence",
"description": "The normalized information pertaining to a behavior, campaign, and/or specific threat actor related to an indicator, Open Source Intelligence (OSINT) analysis, or a generic security finding or report.",
"type": "threat_intelligence"
},
"ticket": {
"caption": "Ticket",
"description": "The linked ticket in the ticketing system.",
Expand Down
258 changes: 152 additions & 106 deletions objects/osint.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,136 @@
"description":"The OSINT (Open Source Intelligence) object contains details related to an indicator such as the indicator itself, related indicators, geolocation, registrar information, subdomains, analyst commentary, and other contextual information. This information can be used to further enrich a detection or finding by providing decisioning support to other analysts and engineers.",
"extends":"_entity",
"attributes":{
"answers":{
"caption":"Related DNS Answers",
"description":"Any pertinent DNS answers information related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"attacks":{
"description":"MITRE ATT&CK Tactics, Techniques, and/or Procedures (TTPs) pertinent to an indicator or OSINT analysis.",
"requirement":"optional"
},
"autonomous_system":{
"description":"Any pertinent autonomous system information related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"comment":{
"caption":"Analyst Comments",
"description":"Analyst commentary or source commentary about an indicator or OSINT analysis.",
"requirement":"optional"
},
"confidence":{
"description":"The confidence of an indicator being malicious and/or pertinent, normalized to the caption of the confidence_id value. In the case of 'Other', it is defined by the event source or analyst.",
"requirement":"optional"
},
"confidence_id":{
"description":"The normalized confidence refers to the accuracy of collected information related to the OSINT or how pertinent an indicator or analysis is to a specific event or finding. A low confidence means that the information collected or analysis conducted lacked detail or is not accurate enough to qualify an indicator as fully malicious.",
"requirement":"recommended"
},
"email":{
"caption":"Related Email",
"description":"Any email information pertinent to an indicator or OSINT analysis.",
"requirement":"optional"
},
"email_auth":{
"caption":"Related Email Authentication",
"description":"Any email authentication information pertinent to an indicator or OSINT analysis.",
"requirement":"optional"
},
"file":{
"caption":"Related File",
"description":"Any pertinent file information related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"first_seen_time": {
"description": "The first time an indicator was seen.",
"requirement": "optional"
},
"kill_chain":{
"description":"Lockheed Martin Kill Chain Phases pertinent to an indicator or OSINT analysis.",
"requirement":"optional"
},
"location":{
"description":"Any pertinent geolocation information related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"last_seen_time": {
"description": "The last time an indicator was seen.",
"requirement": "optional"
},
"name": {
"caption": "OSINT Name",
"description": "The name of an OSINT analysis, a name of an indicator in a system, or other human-readable label or identifier.",
"requirement":"optional"
},
"related_analytics": {
"caption": "Related Analytics",
"description": "A collection of security analytics for a given indicator or OSINT analysis.",
"requirement": "optional"
},
"reputation":{
"description":"Related reputational analysis from third-party engines and analysts for a given indicator or OSINT analysis.",
"requirement":"optional"
},
"script":{
"caption":"Related Script Data",
"description":"Any pertinent script information related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"signatures":{
"caption":"Related Digital Signatures",
"description":"Any digital signatures or hashes related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"src_url":{
"description":"The source URL of an indicator or OSINT analysis, e.g., a URL back to a TIP, report, or otherwise.",
"requirement":"optional"
},
"subnet":{
"caption":"Related Subnet",
"description":"A CIDR or network block related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"subdomains":{
"caption":"Related Subdomains",
"description":"Any pertinent subdomain information - such as those generated by a Domain Generation Algorithm - related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"threat_intelligence": {
"requirement": "recommended"
},
"tlp":{
"caption":"Traffic Light Protocol",
"description":"The <a target='_blank' href='https://www.first.org/tlp/'>Traffic Light Protocol</a> was created to facilitate greater sharing of potentially sensitive information and more effective collaboration. TLP provides a simple and intuitive schema for indicating with whom potentially sensitive information can be shared.",
"enum":{
"RED":{
"caption":"TLP:RED",
"description":"TLP:RED is for the eyes and ears of individual recipients only, no further disclosure. Sources may use TLP:RED when information cannot be effectively acted upon without significant risk for the privacy, reputation, or operations of the organizations involved. Recipients may therefore not share TLP:RED information with anyone else. In the context of a meeting, for example, TLP:RED information is limited to those present at the meeting."
},
"AMBER":{
"caption":"TLP:AMBER",
"description":"TLP:AMBER is for limited disclosure, recipients can only spread this on a need-to-know basis within their organization and its clients. Note that TLP:AMBER+STRICT restricts sharing to the organization only. Sources may use TLP:AMBER when information requires support to be effectively acted upon, yet carries risk to privacy, reputation, or operations if shared outside of the organizations involved. Recipients may share TLP:AMBER information with members of their own organization and its clients, but only on a need-to-know basis to protect their organization and its clients and prevent further harm. Note: if the source wants to restrict sharing to the organization only, they must specify TLP:AMBER+STRICT."
},
"AMBER STRICT":{
"caption":"TLP:AMBER+STRICT",
"description":"TLP:AMBER is for limited disclosure, recipients can only spread this on a need-to-know basis within their organization and its clients. Note that TLP:AMBER+STRICT restricts sharing to the organization only. Sources may use TLP:AMBER when information requires support to be effectively acted upon, yet carries risk to privacy, reputation, or operations if shared outside of the organizations involved. Recipients may share TLP:AMBER information with members of their own organization and its clients, but only on a need-to-know basis to protect their organization and its clients and prevent further harm. Note: if the source wants to restrict sharing to the organization only, they must specify TLP:AMBER+STRICT."
},
"GREEN":{
"caption":"TLP:GREEN",
"description":"TLP:GREEN is for limited disclosure, recipients can spread this within their community. Sources may use TLP:GREEN when information is useful to increase awareness within their wider community. Recipients may share TLP:GREEN information with peers and partner organizations within their community, but not via publicly accessible channels. TLP:GREEN information may not be shared outside of the community. Note: when “community” is not defined, assume the cybersecurity/defense community."
},
"CLEAR":{
"caption":"TLP:CLEAR",
"description":"TLP:CLEAR denotes that recipients can spread this to the world, there is no limit on disclosure. Sources may use TLP:CLEAR when information carries minimal or no foreseeable risk of misuse, in accordance with applicable rules and procedures for public release. Subject to standard copyright rules, TLP:CLEAR information may be shared without restriction."
}
},
"requirement":"recommended",
"type":"string_t"
},
"type_id":{
"caption":"Indicator Type ID",
"description":"The OSINT indicator type ID.",
"requirement":"required",
"requirement":"recommended",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change. A consumer could previously assume this attribute to be present but, if made optional, it can no longer make that assumption.

"enum":{
"0":{
"caption":"Unknown",
Expand Down Expand Up @@ -57,6 +183,18 @@
"caption":"File",
"description":"A file or metadata about a file."
},
"12":{
"caption":"Registry Key",
"description":"An entry within the system registry that may indicate system configuration changes or malicious modifications."
},
"13":{
"caption":"Registry Value",
"description":"Data stored within a registry key, potentially reflecting specific settings or malicious alterations."
},
"14":{
"caption":"Command Line",
"description":"A string or command executed in a system environment, potentially indicating system activity or malicious behavior."
},
"99":{
"caption":"Other",
"description":"The indicator type is not directly listed."
Expand All @@ -67,128 +205,36 @@
"description":"The OSINT indicator type.",
"requirement":"optional"
},
"uid": {
"caption": "OSINT ID",
"description": "The identifier of an OSINT analysis, a UID of an indicator in a system, or other machine-readable label or identifier.",
"requirement":"optional"
},
"value":{
"caption":"Indicator",
"description":"The actual indicator value in scope, e.g., a SHA-256 hash hexdigest or a domain name.",
"requirement":"required"
},
"tlp":{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason why these attrbutes have been moved in the file? It makes it difficult for reviewers to see what has been deleted, which would be a breaking change. Would you be able to re-submit this without the movement of attributes?

"caption":"Traffic Light Protocol",
"description":"The <a target='_blank' href='https://www.first.org/tlp/'>Traffic Light Protocol</a> was created to facilitate greater sharing of potentially sensitive information and more effective collaboration. TLP provides a simple and intuitive schema for indicating with whom potentially sensitive information can be shared.",
"enum":{
"RED":{
"caption":"TLP:RED",
"description":"TLP:RED is for the eyes and ears of individual recipients only, no further disclosure. Sources may use TLP:RED when information cannot be effectively acted upon without significant risk for the privacy, reputation, or operations of the organizations involved. Recipients may therefore not share TLP:RED information with anyone else. In the context of a meeting, for example, TLP:RED information is limited to those present at the meeting."
},
"AMBER":{
"caption":"TLP:AMBER",
"description":"TLP:AMBER is for limited disclosure, recipients can only spread this on a need-to-know basis within their organization and its clients. Note that TLP:AMBER+STRICT restricts sharing to the organization only. Sources may use TLP:AMBER when information requires support to be effectively acted upon, yet carries risk to privacy, reputation, or operations if shared outside of the organizations involved. Recipients may share TLP:AMBER information with members of their own organization and its clients, but only on a need-to-know basis to protect their organization and its clients and prevent further harm. Note: if the source wants to restrict sharing to the organization only, they must specify TLP:AMBER+STRICT."
},
"AMBER STRICT":{
"caption":"TLP:AMBER+STRICT",
"description":"TLP:AMBER is for limited disclosure, recipients can only spread this on a need-to-know basis within their organization and its clients. Note that TLP:AMBER+STRICT restricts sharing to the organization only. Sources may use TLP:AMBER when information requires support to be effectively acted upon, yet carries risk to privacy, reputation, or operations if shared outside of the organizations involved. Recipients may share TLP:AMBER information with members of their own organization and its clients, but only on a need-to-know basis to protect their organization and its clients and prevent further harm. Note: if the source wants to restrict sharing to the organization only, they must specify TLP:AMBER+STRICT."
},
"GREEN":{
"caption":"TLP:GREEN",
"description":"TLP:GREEN is for limited disclosure, recipients can spread this within their community. Sources may use TLP:GREEN when information is useful to increase awareness within their wider community. Recipients may share TLP:GREEN information with peers and partner organizations within their community, but not via publicly accessible channels. TLP:GREEN information may not be shared outside of the community. Note: when “community” is not defined, assume the cybersecurity/defense community."
},
"CLEAR":{
"caption":"TLP:CLEAR",
"description":"TLP:CLEAR denotes that recipients can spread this to the world, there is no limit on disclosure. Sources may use TLP:CLEAR when information carries minimal or no foreseeable risk of misuse, in accordance with applicable rules and procedures for public release. Subject to standard copyright rules, TLP:CLEAR information may be shared without restriction."
}
},
"requirement":"recommended",
"type":"string_t"
},
"confidence_id":{
"description":"The normalized confidence refers to the accuracy of collected information related to the OSINT or how pertinent an indicator or analysis is to a specific event or finding. A low confidence means that the information collected or analysis conducted lacked detail or is not accurate enough to qualify an indicator as fully malicious.",
"requirement":"recommended"
},
"confidence":{
"description":"The confidence of an indicator being malicious and/or pertinent, normalized to the caption of the confidence_id value. In the case of 'Other', it is defined by the event source or analyst.",
"requirement":"optional"
},
"vendor_name":{
"description":"The vendor name of a tool which generates intelligence or provides indicators.",
"requirement":"optional"
},
"src_url":{
"description":"The source URL of an indicator or OSINT analysis, e.g., a URL back to a TIP, report, or otherwise.",
"requirement":"optional"
},
"comment":{
"caption":"Analyst Comments",
"description":"Analyst commentary or source commentary about an indicator or OSINT analysis.",
"requirement":"optional"
},
"email":{
"caption":"Related Email",
"description":"Any email information pertinent to an indicator or OSINT analysis.",
"requirement":"optional"
},
"email_auth":{
"caption":"Related Email Authentication",
"description":"Any email authentication information pertinent to an indicator or OSINT analysis.",
"requirement":"optional"
},
"kill_chain":{
"description":"Lockheed Martin Kill Chain Phases pertinent to an indicator or OSINT analysis.",
"requirement":"optional"
},
"attacks":{
"description":"MITRE ATT&CK Tactics, Techniques, and/or Procedures (TTPs) pertinent to an indicator or OSINT analysis.",
"requirement":"optional"
},
"vulnerabilities":{
"caption":"Related Vulnerabilities",
"description":"Any vulnerabilities related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"signatures":{
"caption":"Related Digital Signatures",
"description":"Any digital signatures or hashes related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"subdomains":{
"caption":"Related Subdomains",
"description":"Any pertinent subdomain information - such as those generated by a Domain Generation Algorithm - related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"answers":{
"caption":"Related DNS Answers",
"description":"Any pertinent DNS answers information related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"whois":{
"description":"Any pertinent WHOIS information related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"autonomous_system":{
"description":"Any pertinent autonomous system information related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"location":{
"description":"Any pertinent geolocation information related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"file":{
"caption":"Related File",
"description":"Any pertinent file information related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"reputation":{
"description":"Related reputational analysis from third-party engines and analysts for a given indicator or OSINT analysis.",
"requirement":"optional"
},
"subnet":{
"caption":"Related Subnet",
"description":"A CIDR or network block related to an indicator or OSINT analysis.",
"requirement":"optional"
},
"script":{
"caption":"Related Script Data",
"description":"Any pertinent script information related to an indicator or OSINT analysis.",
"requirement":"optional"
}
},
"constraints": {
"at_least_one": [
"name",
"type_id",
"uid",
"value"
]
}
}
Loading
Loading