[FIRST EPSS] Create playbook compatible internal enrichment connector

internal-enrichment/first-epss/Dockerfile

@@ -0,0 +1,19 @@
+FROM python:3.11-alpine
+ENV CONNECTOR_TYPE=INTERNAL_ENRICHMENT
+
+# Copy the connector
+COPY src /opt/opencti-connector-first-epss
+
+# Install Python modules
+# hadolint ignore=DL3003
+RUN apk update && apk upgrade && \
+    apk --no-cache add git build-base libmagic libffi-dev libxml2-dev libxslt-dev
+
+RUN cd /opt/opencti-connector-first-epss && \
+    pip3 install --no-cache-dir -r requirements.txt && \
+    apk del git build-base
+
+# Expose and entrypoint
+COPY entrypoint.sh /
+RUN chmod +x /entrypoint.sh
+ENTRYPOINT ["/entrypoint.sh"]

internal-enrichment/first-epss/README.md

@@ -0,0 +1,156 @@
+# OpenCTI Internal Enrichment Connector FIRST EPSS
+
+<!--
+General description of the connector
+* What it does
+* How it works
+* Special requirements
+* Use case description
+* ...
+* Please find an example of expected documentation below
+* REQUIRED CHANGES => Check https://docs.opencti.io/latest/development/connectors/
+-->
+
+Table of Contents
+
+- [OpenCTI Internal Enrichment Connector FIRST EPSS](#opencti-internal-enrichment-connector-first)
+  - [Introduction](#introduction)
+  - [Installation](#installation)
+    - [Requirements](#requirements)
+  - [Configuration variables](#configuration-variables)
+    - [OpenCTI environment variables](#opencti-environment-variables)
+    - [Base connector environment variables](#base-connector-environment-variables)
+    - [Connector extra parameters environment variables](#connector-extra-parameters-environment-variables)
+  - [Deployment](#deployment)
+    - [Docker Deployment](#docker-deployment)
+    - [Manual Deployment](#manual-deployment)
+  - [Usage](#usage)
+  - [Behavior](#behavior)
+  - [Debugging](#debugging)
+  - [Additional information](#additional-information)
+
+## Introduction
+
+## Installation
+
+### Requirements
+
+- OpenCTI Platform >= 
+
+## Configuration variables
+
+There are a number of configuration options, which are set either in `docker-compose.yml` (for Docker) or
+in `config.yml` (for manual deployment).
+
+### OpenCTI environment variables
+
+Below are the parameters you'll need to set for OpenCTI:
+
+| Parameter     | config.yml | Docker environment variable | Mandatory | Description                                          |
+|---------------|------------|-----------------------------|-----------|------------------------------------------------------|
+| OpenCTI URL   | url        | `OPENCTI_URL`               | Yes       | The URL of the OpenCTI platform.                     |
+| OpenCTI Token | token      | `OPENCTI_TOKEN`             | Yes       | The default admin token set in the OpenCTI platform. |
+
+### Base connector environment variables
+
+Below are the parameters you'll need to set for running the connector properly:
+
+| Parameter           | config.yml | Docker environment variable | Default         | Mandatory | Description                                                                              |
+|---------------------|------------|-----------------------------|-----------------|-----------|------------------------------------------------------------------------------------------|
+| Connector ID        | id         | `CONNECTOR_ID`              | /               | Yes       | A unique `UUIDv4` identifier for this connector instance.                                |
+| Connector Type      | type       | `CONNECTOR_TYPE`            | EXTERNAL_IMPORT | Yes       | Should always be set to `INTERNAL_ENRICHMENT` for this connector.                        |
+| Connector Name      | name       | `CONNECTOR_NAME`            | /               | Yes       | Name of the connector.                                                                   |
+| Connector Scope     | scope      | `CONNECTOR_SCOPE`           | `vulnerability` | Yes       | The scope or type of data the connector is importing, either a MIME type or Stix Object. |
+| Connector Log Level | log_level  | `CONNECTOR_LOG_LEVEL`       | info            | Yes       | Determines the verbosity of the logs. Options are `debug`, `info`, `warn`, or `error`.   |
+| Connector Auto      | auto 	     | `CONNECTOR_AUTO`            | False           | Yes       | Must be `true` or `false` to enable or disable auto-enrichment of observables            |
+
+### Connector extra parameters environment variables
+
+Below are the parameters you'll need to set for the connector:
+
+| Parameter    | config.yml   | Docker environment variable | Default                              | Mandatory | Description |
+|--------------|--------------|-----------------------------|--------------------------------------|-----------|-------------|
+| API base URL | api_base_url | FIRST_EPSS_API_BASE_URL     | `https://api.first.org/data/v1/epss` | No        |             |
+| Max TLP      | max_tlp      | FIRST_EPSS_MAX_TLP          | /                                    | No        |             |
+
+## Deployment
+
+### Docker Deployment
+
+Before building the Docker container, you need to set the version of pycti in `requirements.txt` equal to whatever
+version of OpenCTI you're running. Example, `pycti==5.12.20`. If you don't, it will take the latest version, but
+sometimes the OpenCTI SDK fails to initialize.
+
+Build a Docker Image using the provided `Dockerfile`.
+
+Example:
+
+```shell
+# Replace the IMAGE NAME with the appropriate value
+docker build . -t [IMAGE NAME]:latest
+```
+
+Make sure to replace the environment variables in `docker-compose.yml` with the appropriate configurations for your
+environment. Then, start the docker container with the provided docker-compose.yml
+
+```shell
+docker compose up -d
+# -d for detached
+```
+
+### Manual Deployment
+
+Create a file `config.yml` based on the provided `config.yml.sample`.
+
+Replace the configuration variables (especially the "**ChangeMe**" variables) with the appropriate configurations for
+you environment.
+
+Install the required python dependencies (preferably in a virtual environment):
+
+```shell
+pip3 install -r requirements.txt
+```
+
+Then, start the connector from recorded-future/src:
+
+```shell
+python3 main.py
+```
+
+## Usage
+
+After Installation, the connector should require minimal interaction to use, and should update automatically at a regular interval specified in your `docker-compose.yml` or `config.yml` in `duration_period`.
+
+However, if you would like to force an immediate download of a new batch of entities, navigate to:
+
+`Data management` -> `Ingestion` -> `Connectors` in the OpenCTI platform.
+
+Find the connector, and click on the refresh button to reset the connector's state and force a new
+download of data by re-running the connector.
+
+## Behavior
+
+<!--
+Describe how the connector functions:
+* What data is ingested, updated, or modified
+* Important considerations for users when utilizing this connector
+* Additional relevant details
+-->
+
+
+## Debugging
+
+The connector can be debugged by setting the appropiate log level.
+Note that logging messages can be added using `self.helper.connector_logger,{LOG_LEVEL}("Sample message")`, i.
+e., `self.helper.connector_logger.error("An error message")`.
+
+<!-- Any additional information to help future users debug and report detailed issues concerning this connector -->
+
+## Additional information
+
+<!--
+Any additional information about this connector
+* What information is ingested/updated/changed
+* What should the user take into account when using this connector
+* ...
+-->

internal-enrichment/first-epss/docker-compose.yml

@@ -0,0 +1,32 @@
+version: '3'
+services:
+  connector-first-epss:
+    image: opencti/connector-first-epss:6.2.12
+    environment:
+      # Connector's generic execution parameters
+      - OPENCTI_URL=http://localhost
+      - OPENCTI_TOKEN=CHANGEME
+      # Connector's definition parameters REQUIRED
+      - CONNECTOR_ID=CHANGEME
+      - CONNECTOR_NAME='FIRST EPSS'
+      - CONNECTOR_SCOPE=vulnerability
+      - CONNECTOR_LOG_LEVEL=error
+      - CONNECTOR_AUTO=false
+
+      # Connector's custom execution parameters
+      - FIRST_EPSS_API_BASE_URL=https://api.first.org/data/v1/epss
+      - FIRST_EPSS_MAX_TLP=TLP:CLEAR # Available values: TLP:CLEAR, TLP:WHITE, TLP:GREEN, TLP:AMBER, TLP:AMBER+STRICT, TLP:RED
+
+      # Add proxy parameters below if needed
+      # - HTTP_PROXY=CHANGEME
+      # - HTTPS_PROXY=CHANGEME
+      # - NO_PROXY=CHANGEME
+    restart: always
+    # networks:
+    #   - docker_default
+
+# networks:
+#   default:
+#     external: true
+#     name: docker_default
+

internal-enrichment/first-epss/entrypoint.sh

@@ -0,0 +1,7 @@
+#!/bin/sh
+
+# Go to the right directory
+cd /opt/opencti-connector-first-epss
+
+# Launch the worker
+python3 main.py

internal-enrichment/first-epss/src/config.yml.sample

@@ -0,0 +1,15 @@
+opencti:
+  url: 'http://localhost:PORT'
+  token: 'ChangeMe'
+
+connector:
+  id: 'ChangeMe'
+  type: 'INTERNAL_ENRICHMENT'
+  name: 'Internal Enrichment Connector FIRST'
+  scope: 'vulnerability'
+  log_level: 'info'
+  auto: false # Enable/disable auto-enrichment of observables
+
+first_epss:
+  api_base_url: 'https://api.first.org/data/v1/epss'
+  max_tlp: "TLP:AMBER" # Available values: TLP:CLEAR, TLP:WHITE, TLP:GREEN, TLP:AMBER, TLP:AMBER+STRICT, TLP:RED

internal-enrichment/first-epss/src/first_epss_connector_tests/common_fixtures.py

@@ -0,0 +1,14 @@
+from unittest.mock import Mock
+
+import pytest
+
+
+@pytest.fixture(scope="class")
+def setup_config(request):
+    """
+    Setup configuration for class method
+    Create fake pycti OpenCTI helper
+    """
+    request.cls.mock_helper = Mock()
+
+    yield

internal-enrichment/first-epss/src/first_epss_connector_tests/test_first_epss_connector.py

@@ -0,0 +1,17 @@
+# import pytest
+
+# from .common_fixtures import setup_config # noqa: F401
+
+
+# @pytest.mark.usefixtures("setup_config")
+class TestFirstEPSSConnector(object):
+    def test_to_do(self) -> None:
+        """
+        Check if running test works
+        """
+        # Use the helper
+        # self.mock_helper()
+        value_received = "Value is True"
+        expected_result = "Value is True"
+
+        assert value_received == expected_result

internal-enrichment/first-epss/src/internal_enrichment_connector/__init__.py

@@ -0,0 +1,3 @@
+from .connector import FirstEPSSConnector
+
+__all__ = ["FirstEPSSConnector"]

internal-enrichment/first-epss/src/internal_enrichment_connector/client_api.py

@@ -0,0 +1,55 @@
+import requests
+
+
+class ConnectorClient:
+    def __init__(self, helper, config):
+        """
+        Initialize the client with necessary configurations
+        """
+        self.helper = helper
+        self.config = config
+
+        # Define headers in session and update when needed
+        headers = {}
+        self.session = requests.Session()
+        self.session.headers.update(headers)
+
+    def _request_data(self, api_url: str, params=None):
+        """
+        Internal method to handle API requests
+        :return: Response in JSON format
+        """
+        try:
+            response = self.session.get(api_url, params=params)
+
+            self.helper.connector_logger.info(
+                "[API] HTTP Get Request to endpoint", {"url_path": api_url}
+            )
+
+            response.raise_for_status()
+
+            return response
+
+        except requests.RequestException as err:
+            error_msg = "[API] Error while fetching data: "
+            self.helper.connector_logger.error(
+                error_msg, {"url_path": {api_url}, "error": {str(err)}}
+            )
+
+            return None
+
+    def get_entity(self, params=None) -> dict:
+        """
+        If params is None, retrieve all CVEs in National Vulnerability Database
+        :param params: Optional Params to filter what list to return
+        :return: A list of dicts of the complete collection of CVE from NVD
+        """
+
+        try:
+            response = self._request_data(self.config.api_base_url, params=params)
+
+            return response.json()
+
+        except Exception as err:
+            error_msg = "[API] Error while parsing data: "
+            self.helper.connector_logger.error(error_msg, {"error": {str(err)}})

internal-enrichment/first-epss/src/internal_enrichment_connector/config_variables.py

@@ -0,0 +1,59 @@
+import os
+from pathlib import Path
+
+import yaml
+from pycti import get_config_variable
+
+
+class ConfigConnector:
+    def __init__(self):
+        """
+        Initialize the connector with necessary configurations
+        """
+
+        # Load configuration file
+        self.load = self._load_config()
+        self._initialize_configurations()
+
+    @staticmethod
+    def _load_config() -> dict:
+        """
+        Load the configuration from the YAML file
+        :return: Configuration dictionary
+        """
+
+        config_file_path = Path(__file__).parents[1].joinpath("config.yml")
+        config = (
+            yaml.load(open(config_file_path), Loader=yaml.FullLoader)
+            if os.path.isfile(config_file_path)
+            else {}
+        )
+
+        return config
+
+    def _initialize_configurations(self) -> None:
+        """
+        Connector configuration variables
+        :return: None
+        """
+
+        # OpenCTI configurations
+
+        # Connector extra parameters
+        self.api_base_url = get_config_variable(
+            "FIRST_EPSS_API_BASE_URL",
+            ["first_epss", "api_base_url"],
+            self.load,
+            False,
+            "https://api.first.org/data/v1/epss",
+            True,
+        )
+
+        self.max_tlp = get_config_variable(
+            "FIRST_EPSS_MAX_TLP",
+            ["first_epss", "max_tlp"],
+            self.load,
+            False,
+            None,
+            False,
+        )