E2E Research: Detecting malware using Yara integration

[mauromalara]

Description

This issue aims to test manually the Wazuh integration with Yara to define the requirements to develop an automated E2E test.

To achieve this we will follow these guides:

1. https://documentation.wazuh.com/current/proof-of-concept-guide/detect-malware-yara-integration.html
2. https://documentation.wazuh.com/current/user-manual/capabilities/active-response/ar-use-cases/wazuh-with-yara.html

Tasks

• (T1): Configure the Yara integration in both Manager and Agent endpoints
• (T2): Generate and query the alerts

Conclusion 🟢

Requirements
A Manager with CentOS 8
An Agent with Ubuntu 20.04
The AR script and the script to download the malware samples
The query to get the alerts from the manager

[mauromalara]

Task 1: Configure the Yara integration in both Manager and Agent endpoints 🟢

Endpoint type	OS
Manager	CentOS 8
Agent	Ubuntu 20.04

Step by step

1. Install the Wazuh manager (all-in-one)

sudo bash ./wazuh-install.sh -a -i

2. Install the Wazuh agent (pre-release package for deb system used)
3. Add the following rules to /var/ossec/etc/rules/local_rules.xml (MANAGER):

<group name="syscheck,">
    <rule id="100300" level="7">
        <if_sid>550</if_sid>
        <field name="file">/tmp/yara/malware/</field>
        <description>File modified in /tmp/yara/malware/ directory.</description>
    </rule>
    <rule id="100301" level="7">
        <if_sid>554</if_sid>
        <field name="file">/tmp/yara/malware/</field>
        <description>File added to /tmp/yara/malware/ directory.</description>
    </rule>
</group>

<group name="yara,">
    <rule id="108000" level="0">
        <decoded_as>yara_decoder</decoded_as>
        <description>Yara grouping rule</description>
    </rule>
    <rule id="108001" level="12">
        <if_sid>108000</if_sid>
        <match>wazuh-yara: INFO - Scan result: </match>
        <description>File "$(yara_scanned_file)" is a positive match. Yara rule: $(yara_rule)</description>
    </rule>
</group>

4. Add the following decoders to /var/ossec/etc/decoders/local_decoder.xml (MANAGER):

<decoder name="yara_decoder">
    <prematch>wazuh-yara:</prematch>
</decoder>

<decoder name="yara_decoder1">
    <parent>yara_decoder</parent>
    <regex>wazuh-yara: (\S+) - Scan result: (\S+) (\S+)</regex>
    <order>log_type, yara_rule, yara_scanned_file</order>
</decoder>

5. Add the following configuration to /var/ossec/etc/ossec.conf (MANAGER):

  <command>
      <name>yara</name>
      <executable>yara.sh</executable>
      <extra_args>-yara_path /usr/local/bin -yara_rules /tmp/yara/rules/yara_rules.yar</extra_args>
      <timeout_allowed>no</timeout_allowed>
   </command>
  <active-response>
      <command>yara</command>
      <location>local</location>
      <rules_id>100300,100301</rules_id>
  </active-response>

6. Restart the manager systemctl restart wazuh-manager
7. Compile and install Yara at the agent:

apt update
apt install -y make gcc autoconf libtool libssl-dev pkg-config
curl -LO https://github.com/VirusTotal/yara/archive/v4.0.2.tar.gz
tar -xvzf v4.0.2.tar.gz -C /usr/local/bin/ && rm -f v4.0.2.tar.gz
cd /usr/local/bin/yara-4.0.2
./bootstrap.sh && ./configure && make && sudo make install && make check

8. Download the rules of Yara:

mkdir -p /tmp/yara/rules
curl 'https://valhalla.nextron-systems.com/api/v1/get' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
-H 'Accept-Language: en-US,en;q=0.5' \
--compressed \
-H 'Referer: https://valhalla.nextron-systems.com/' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-H 'DNT: 1' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' \
--data 'demo=demo&apikey=1111111111111111111111111111111111111111111111111111111111111111&format=text' \
-o /tmp/yara/rules/yara_rules.yar

9. Download a malware sample and run a Yara scan:

mkdir -p /tmp/yara/malware
curl -L https://wazuh-demo.s3-us-west-1.amazonaws.com/mirai -o /tmp/yara/malware/mirai
/usr/local/bin/yara /tmp/yara/rules/yara_rules.yar /tmp/yara/malware/mirai

Output:

MAL_ELF_LNX_Mirai_Oct10_2_RID2F3A /tmp/yara/malware/mirai
Mirai_Botnet_Malware_RID2EF6 /tmp/yara/malware/mirai

10. Create a script called yara.sh in /var/ossec/active-response/bin/ with the following content: yara_script_content.txt
11. Change the permissions and the owner of the script:

chown root:wazuh /var/ossec/active-response/bin/yara.sh
chmod 750 /var/ossec/active-response/bin/yara.sh

12. Run the following command apt install -y jq
13. In the /var/ossec/etc/ossec.conf file (AGENT) add the following option inside the <syscheck> tag:

<directories whodata="yes">/tmp/yara/malware</directories>

14. Restart the agent: systemctl restart wazuh-agent

[mauromalara]

Task 2: Generate and query the alerts 🟢

1. Create the /tmp/yara/malware/malware_downloader.sh script with the following content: malware_downloader_script_content.txt
2. Run the script bash /tmp/yara/malware/malware_downloader.sh
3. Query the alerts in the manager:

curl --insecure -XGET -u 'USER:PASSWORD' "https://localhost:9200/wazuh-alerts-*/_search?pretty" -H 'Content-Type: application/json' -d '
{
  "query": {
    "term": {
      "rule.groups": "yara"
    }
  }
}'

Result

Query result (alerts from wazuh-alerts-* index)

{
  "took" : 9,
  "timed_out" : false,
  "_shards" : {
    "total" : 3,
    "successful" : 3,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 6,
      "relation" : "eq"
    },
    "max_score" : 4.8075256,
    "hits" : [
      {
        "_index" : "wazuh-alerts-4.x-2022.05.04",
        "_type" : "_doc",
        "_id" : "QDfMkIABBVef_0zH4qix",
        "_score" : 4.8075256,
        "_source" : {
          "agent" : {
            "ip" : "10.0.2.15",
            "name" : "agent-2860",
            "id" : "001"
          },
          "manager" : {
            "name" : "manager-2860"
          },
          "data" : {
            "log_type" : "INFO",
            "yara_rule" : "Mirai_Botnet_Malware_RID2EF6",
            "yara_scanned_file" : "/tmp/yara/malware/mirai"
          },
          "rule" : {
            "firedtimes" : 2,
            "mail" : true,
            "level" : 12,
            "description" : "File \"/tmp/yara/malware/mirai\" is a positive match. Yara rule: Mirai_Botnet_Malware_RID2EF6",
            "groups" : [
              "yara"
            ],
            "id" : "108001"
          },
          "decoder" : {
            "name" : "yara_decoder"
          },
          "full_log" : "wazuh-yara: INFO - Scan result: Mirai_Botnet_Malware_RID2EF6 /tmp/yara/malware/mirai",
          "input" : {
            "type" : "log"
          },
          "@timestamp" : "2022-05-04T20:39:43.849Z",
          "location" : "/var/ossec/logs/active-responses.log",
          "id" : "1651696783.1065484",
          "timestamp" : "2022-05-04T20:39:43.849+0000"
        }
      },
      {
        "_index" : "wazuh-alerts-4.x-2022.05.04",
        "_type" : "_doc",
        "_id" : "SjfMkIABBVef_0zH4qix",
        "_score" : 4.385922,
        "_source" : {
          "agent" : {
            "ip" : "10.0.2.15",
            "name" : "agent-2860",
            "id" : "001"
          },
          "manager" : {
            "name" : "manager-2860"
          },
          "data" : {
            "log_type" : "INFO",
            "yara_rule" : "MAL_ELF_LNX_Mirai_Oct10_2_RID2F3A",
            "yara_scanned_file" : "/tmp/yara/malware/mirai"
          },
          "rule" : {
            "firedtimes" : 5,
            "mail" : true,
            "level" : 12,
            "description" : "File \"/tmp/yara/malware/mirai\" is a positive match. Yara rule: MAL_ELF_LNX_Mirai_Oct10_2_RID2F3A",
            "groups" : [
              "yara"
            ],
            "id" : "108001"
          },
          "decoder" : {
            "name" : "yara_decoder"
          },
          "full_log" : "wazuh-yara: INFO - Scan result: MAL_ELF_LNX_Mirai_Oct10_2_RID2F3A /tmp/yara/malware/mirai",
          "input" : {
            "type" : "log"
          },
          "@timestamp" : "2022-05-04T20:39:45.810Z",
          "location" : "/var/ossec/logs/active-responses.log",
          "id" : "1651696785.1073953",
          "timestamp" : "2022-05-04T20:39:45.810+0000"
        }
      },
      {
        "_index" : "wazuh-alerts-4.x-2022.05.04",
        "_type" : "_doc",
        "_id" : "SzfMkIABBVef_0zH4qix",
        "_score" : 4.385922,
        "_source" : {
          "agent" : {
            "ip" : "10.0.2.15",
            "name" : "agent-2860",
            "id" : "001"
          },
          "manager" : {
            "name" : "manager-2860"
          },
          "data" : {
            "log_type" : "INFO",
            "yara_rule" : "Mirai_Botnet_Malware_RID2EF6",
            "yara_scanned_file" : "/tmp/yara/malware/mirai"
          },
          "rule" : {
            "firedtimes" : 6,
            "mail" : true,
            "level" : 12,
            "description" : "File \"/tmp/yara/malware/mirai\" is a positive match. Yara rule: Mirai_Botnet_Malware_RID2EF6",
            "groups" : [
              "yara"
            ],
            "id" : "108001"
          },
          "decoder" : {
            "name" : "yara_decoder"
          },
          "full_log" : "wazuh-yara: INFO - Scan result: Mirai_Botnet_Malware_RID2EF6 /tmp/yara/malware/mirai",
          "input" : {
            "type" : "log"
          },
          "@timestamp" : "2022-05-04T20:39:45.852Z",
          "location" : "/var/ossec/logs/active-responses.log",
          "id" : "1651696785.1074392",
          "timestamp" : "2022-05-04T20:39:45.852+0000"
        }
      },
      {
        "_index" : "wazuh-alerts-4.x-2022.05.04",
        "_type" : "_doc",
        "_id" : "PzfMkIABBVef_0zH4qix",
        "_score" : 4.1083508,
        "_source" : {
          "agent" : {
            "ip" : "10.0.2.15",
            "name" : "agent-2860",
            "id" : "001"
          },
          "manager" : {
            "name" : "manager-2860"
          },
          "data" : {
            "log_type" : "INFO",
            "yara_rule" : "MAL_ELF_LNX_Mirai_Oct10_2_RID2F3A",
            "yara_scanned_file" : "/tmp/yara/malware/mirai"
          },
          "rule" : {
            "firedtimes" : 1,
            "mail" : true,
            "level" : 12,
            "description" : "File \"/tmp/yara/malware/mirai\" is a positive match. Yara rule: MAL_ELF_LNX_Mirai_Oct10_2_RID2F3A",
            "groups" : [
              "yara"
            ],
            "id" : "108001"
          },
          "decoder" : {
            "name" : "yara_decoder"
          },
          "full_log" : "wazuh-yara: INFO - Scan result: MAL_ELF_LNX_Mirai_Oct10_2_RID2F3A /tmp/yara/malware/mirai",
          "input" : {
            "type" : "log"
          },
          "@timestamp" : "2022-05-04T20:39:43.807Z",
          "location" : "/var/ossec/logs/active-responses.log",
          "id" : "1651696783.1065045",
          "timestamp" : "2022-05-04T20:39:43.807+0000"
        }
      },
      {
        "_index" : "wazuh-alerts-4.x-2022.05.04",
        "_type" : "_doc",
        "_id" : "QTfMkIABBVef_0zH4qix",
        "_score" : 4.1083508,
        "_source" : {
          "agent" : {
            "ip" : "10.0.2.15",
            "name" : "agent-2860",
            "id" : "001"
          },
          "manager" : {
            "name" : "manager-2860"
          },
          "data" : {
            "log_type" : "INFO",
            "yara_rule" : "MAL_ELF_LNX_Mirai_Oct10_2_RID2F3A",
            "yara_scanned_file" : "/tmp/yara/malware/mirai"
          },
          "rule" : {
            "firedtimes" : 3,
            "mail" : true,
            "level" : 12,
            "description" : "File \"/tmp/yara/malware/mirai\" is a positive match. Yara rule: MAL_ELF_LNX_Mirai_Oct10_2_RID2F3A",
            "groups" : [
              "yara"
            ],
            "id" : "108001"
          },
          "decoder" : {
            "name" : "yara_decoder"
          },
          "full_log" : "wazuh-yara: INFO - Scan result: MAL_ELF_LNX_Mirai_Oct10_2_RID2F3A /tmp/yara/malware/mirai",
          "input" : {
            "type" : "log"
          },
          "@timestamp" : "2022-05-04T20:39:43.849Z",
          "location" : "/var/ossec/logs/active-responses.log",
          "id" : "1651696783.1065908",
          "timestamp" : "2022-05-04T20:39:43.849+0000"
        }
      },
      {
        "_index" : "wazuh-alerts-4.x-2022.05.04",
        "_type" : "_doc",
        "_id" : "QjfMkIABBVef_0zH4qix",
        "_score" : 4.1083508,
        "_source" : {
          "agent" : {
            "ip" : "10.0.2.15",
            "name" : "agent-2860",
            "id" : "001"
          },
          "manager" : {
            "name" : "manager-2860"
          },
          "data" : {
            "log_type" : "INFO",
            "yara_rule" : "Mirai_Botnet_Malware_RID2EF6",
            "yara_scanned_file" : "/tmp/yara/malware/mirai"
          },
          "rule" : {
            "firedtimes" : 4,
            "mail" : true,
            "level" : 12,
            "description" : "File \"/tmp/yara/malware/mirai\" is a positive match. Yara rule: Mirai_Botnet_Malware_RID2EF6",
            "groups" : [
              "yara"
            ],
            "id" : "108001"
          },
          "decoder" : {
            "name" : "yara_decoder"
          },
          "full_log" : "wazuh-yara: INFO - Scan result: Mirai_Botnet_Malware_RID2EF6 /tmp/yara/malware/mirai",
          "input" : {
            "type" : "log"
          },
          "@timestamp" : "2022-05-04T20:39:43.849Z",
          "location" : "/var/ossec/logs/active-responses.log",
          "id" : "1651696783.1066347",
          "timestamp" : "2022-05-04T20:39:43.849+0000"
        }
      }
    ]
  }
}

Alert in alerts.json

{"timestamp":"2022-05-04T20:39:45.852+0000","rule":{"level":12,"description":"File \"/tmp/yara/malware/mirai\" is a positive match. Yara rule: Mirai_Botnet_Malware_RID2EF6","id":"108001","firedtimes":6,"mail":true,"groups":["yara"]},"agent":{"id":"001","name":"agent-2860","ip":"10.0.2.15"},"manager":{"name":"manager-2860"},"id":"1651696785.1074392","full_log":"wazuh-yara: INFO - Scan result: Mirai_Botnet_Malware_RID2EF6 /tmp/yara/malware/mirai","decoder":{"name":"yara_decoder"},"data":{"log_type":"INFO","yara_rule":"Mirai_Botnet_Malware_RID2EF6","yara_scanned_file":"/tmp/yara/malware/mirai"},"location":"/var/ossec/logs/active-responses.log"}