Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timeout and retries to VirusTotal integration #16893

Merged
merged 2 commits into from
Sep 8, 2023

Conversation

TomasTurina
Copy link
Member

@TomasTurina TomasTurina commented Apr 27, 2023

Related issue Documentation
#16488 wazuh/wazuh-documentation#6442

Description

This PR adds the ability to configure an integration with timeouts and retries to prevent API calls from hanging.

  • VirusTotal script uses these parameters to control the execution of the API call and retries the request in case of the timeout expires.

Also, the logging of VirusTotal script is improved.

Configuration options

Example:

  <integration>
    <name>virustotal</name>
    <api_key>xyz</api_key>
    <group>syscheck</group>
    <alert_format>json</alert_format>
    <retries>5</retries>
    <timeout>15</timeout>
  </integration>

Logs/Alerts example

Alert example:

** Alert 1692133899.695844: - virustotal,gdpr_IV_35.7.d,gdpr_IV_32.2,
2023 Aug 15 21:11:39 devel->virustotal
Rule: 87106 (level 3) -> 'VirusTotal: Error: API request timed out'
{"virustotal": {"error": 408, "description": "Error: API request timed out"}, "integration": "virustotal"}
virustotal.error: 408
virustotal.description: Error: API request timed out
integration: virustotal

Logs example:

/tmp/virustotal-1692133809-1810084328.alert cfa079dd7c7c8ba311638fda953f46597957318c999324bfd3582d5f1fd50e43  debug  15 5
# Running VirusTotal script
# Opening alert file at '/tmp/virustotal-1692133809-1810084328.alert' with '{'timestamp': '2023-08-15T21:10:08.712+0000', 'rule': {'level': 7, 'description': 'File deleted.', 'id': '553', 'mitre': {'id': ['T1070.004', 'T1485'], 'tactic': ['Defense Evasion', 'Impact'], 'technique': ['File Deletion', 'Data Destruction']}, 'firedtimes': 1, 'mail': False, 'groups': ['ossec', 'syscheck', 'syscheck_entry_deleted', 'syscheck_file'], 'pci_dss': ['11.5'], 'gpg13': ['4.11'], 'gdpr': ['II_5.1.f'], 'hipaa': ['164.312.c.1', '164.312.c.2'], 'nist_800_53': ['SI.7'], 'tsc': ['PI1.4', 'PI1.5', 'CC6.1', 'CC6.8', 'CC7.2', 'CC7.3']}, 'agent': {'id': '000', 'name': 'devel'}, 'manager': {'name': 'devel'}, 'id': '1692133808.695181', 'full_log': "File '/test/lala.txt' deleted\nMode: realtime\n", 'syscheck': {'path': '/test/lala.txt', 'mode': 'realtime', 'size_after': '69', 'perm_after': 'rw-r--r--', 'uid_after': '0', 'gid_after': '0', 'md5_after': '69630e4574ec6798239b091cda43dca0', 'sha1_after': 'cf8bd9dfddff007f75adf4c2be48005cea317c62', 'sha256_after': '131f95c51cc819465fa1797f6ccacf9d494aaaff46fa3eac73ae63ffbdfd8267', 'uname_after': 'root', 'gname_after': 'root', 'mtime_after': '2023-08-15T21:09:40', 'inode_after': 1835011, 'event': 'deleted'}, 'decoder': {'name': 'syscheck_deleted'}, 'location': 'syscheck'}'
# Requesting VirusTotal information
# Querying VirusTotal API
# Error: Request timed out. Remaining retries: 5
# Querying VirusTotal API
# Error: Request timed out. Remaining retries: 4
# Querying VirusTotal API
# Error: Request timed out. Remaining retries: 3
# Querying VirusTotal API
# Error: Request timed out. Remaining retries: 2
# Querying VirusTotal API
# Error: Request timed out. Remaining retries: 1
# Querying VirusTotal API
# Error: Request timed out. Remaining retries: 0
# Error: Request timed out and maximum number of retries was exceeded
# Request result from VT server: 1:virustotal:{"virustotal": {"error": 408, "description": "Error: API request timed out"}, "integration": "virustotal"}

Tests

  • Compilation without warnings in every supported platform
    • Linux
  • Source installation
  • Package installation
  • Source upgrade
  • Package upgrade
  • Review logs syntax and correct language

@TomasTurina TomasTurina self-assigned this Apr 27, 2023
@TomasTurina TomasTurina added the module/integration Issues related to the Integrator daemon label Apr 27, 2023
@TomasTurina TomasTurina linked an issue Apr 27, 2023 that may be closed by this pull request
sdvendramini
sdvendramini previously approved these changes Apr 27, 2023
Copy link
Member

@sdvendramini sdvendramini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

sdvendramini
sdvendramini previously approved these changes Apr 28, 2023
@vikman90 vikman90 marked this pull request as draft May 2, 2023 14:22
@TomasTurina TomasTurina force-pushed the 16488_improve_vt_script branch from 7274e9c to 983489c Compare August 15, 2023 21:13
@TomasTurina TomasTurina marked this pull request as ready for review August 15, 2023 21:15
sdvendramini
sdvendramini previously approved these changes Aug 22, 2023
@TomasTurina TomasTurina force-pushed the 16488_improve_vt_script branch from 6e560ee to 9a688eb Compare September 4, 2023 21:00
@TomasTurina TomasTurina merged commit 17469f8 into master Sep 8, 2023
59 checks passed
@TomasTurina TomasTurina deleted the 16488_improve_vt_script branch September 8, 2023 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module/integration Issues related to the Integrator daemon
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve the reliability of the Virustotal integration
2 participants