Malcolm v24.09.0

[mmguero]

Malcolm v24.09.0 contains minor improvements, some component version updates, and bug fixes.

idaholab/Malcolm@v24.08.0...v24.09.0

• Features and enhancements
  • When building Docker images and the Hedgehog Linux ISO, allow specifying alternate download URL for MaxMind GeoIP database files (allow specifying alternate download location for MaxMind GeoIP database files idaholab/Malcolm#565)
  • Allow total index size-based pruning for opensearch-remote and elasticsearch-remote database modes (allow total index size-based pruning for opensearch-remote and elasticsearch-remote database modes idaholab/Malcolm#446)
  • Allow splitting out indexes by other field values (allow splitting out indexes by other field values idaholab/Malcolm#450)
  • Allow users to use the Arkime Lua plugin without having to create new bind volume mounts manually (allow users to use the Arkime Lua plugin without having to create new bind volume mounts manually idaholab/Malcolm#533)
  • Automatically create empty document on startup to avoid "no data" message spamming by Dashboards (automatically create empty document on startup to avoid "no data" message spamming by Dashboards idaholab/Malcolm#527 and Create Initial Opensearch Indices idaholab/Malcolm#567)
  • Improvements to documentation and install.py for Linux performance tweaks (improvements to documentation and install.py for Linux performance tweaks idaholab/Malcolm#495)
  • Include netbox-topology-views plugin by default (include netbox-topology plugin by default idaholab/Malcolm#553)
  • Integrate HART-IP parser (integrate HART IP parser idaholab/Malcolm#561)
  • Add option to go backwards in Malcolm's dialog-based install.py installation and configuration script (No option to go backwards in Malcolm install tool idaholab/Malcolm#487)
  • Added Podman support (podman support idaholab/Malcolm#407)
  • Update EtherNet/IP and CIP to account for new packet correlation ID (update ethernet/IP and CIP to account for new packet correlation ID idaholab/Malcolm#558)
  • Update [Network Traffic Analysis with Malcolm slides](https://github.com/cisagov/Malcolm/blob/150674d428573e303dd6c039700df8b8ca5f05a8/docs/slides/Network Traffic Analysis with Malcolm.pdf)
• Component version updates
  • watchdog Python package to v5.0.x (update watchdog package to 5.0.0 idaholab/Malcolm#550)
  • supercronic to v0.2.32
  • osd_transform_vis to v2.16.0
  • Fluent Bit to v3.1.8
  • OpenSearch and OpenSearch Dashboards to v2.17.0
  • YARA to v4.5.2
  • Zeek to v7.0.1
  • Spicy to v1.11.1
  • elasticsearch-dsl Python package to v8.15.3
  • elasticsearch Python package to v8.15.1
  • Beats to v8.15.1
  • Logstath to 8.15.1
  • flask-cors Python package to v5.0.0 to address CVE-2024-6221
• Bug fixes
  • Filtering on hunt ID in Arkime not working (filtering on hunt ID in Arkime not working idaholab/Malcolm#554)
  • Hedgehog with OOB/VPN connection sets ARKIME_NODE_HOST incorrectly (Hedgehog with OOB/VPN connection sets ARKIME_NODE_HOST incorrectly idaholab/Malcolm#560 and Set ARKIME_NODE_HOST from OS_HOST interface rather than default route idaholab/Malcolm#559, thanks @divinehawk)
  • Offline suricata Docker container does not initialize suricata.yml config file (offline suricata Docker container does not initialize suricata config file idaholab/Malcolm#564)
• Configuration changes (in environment variables in ./config/) for Malcolm and in control_vars.conf for Hedgehog Linux
  • Malcolm
    • The MALCOLM_NETWORK_INDEX_SUFFIX and MALCOLM_OTHER_INDEX_SUFFIX variables in ./config/opensearch.env now also support expanding dot-delimited field names in ｛｛ ｝｝ (e.g., ｛｛event.provider｝｝％{％y％m％d}).
    • MALCOLM_CONTAINER_RUNTIME has been added to ./config/process.env to indicate docker, podman, or kubernetes. This value only currently used in the install, configuration, and control scripts, not inside the containers themselves.
    • ZEEK_DISABLE_ICS_HART_IP has been added to ./config/zeek.env and can be set to true to disable the new HART-IP protocol parser.
  • Hedgehog Linux
    • ZEEK_DISABLE_ICS_HART_IP has been added to control_vars.conf and can be set to true to disable the new HART-IP protocol parser.