Malcolm v23.02.0

Malcolm v23.02.0 is a feature release with new features and enhancements, component version updates and bug fixes.

v23.01.0...v23.02.0

• New features

  • Compare and highlight discrepancies between NetBox inventory and observed network traffic (idaholab#133)
    • Added Zeek Known Summary and Asset Interaction Analysis dashboards which include visualizations about uninventoried devices and services
    • Added Uninventoried Internal Assets and Uninventoried Observed Services views to Arkime
    • Documentation updates related to NetBox
  • Added default device roles and service templates for initial NetBox population
  • Added netbox-backup/netbox-restore scripts to control.py for NetBox database and media
  • Added zeek_script_to_malcolm_boilerplate.py script for automating some of the tasks involved with adding new Zeek logs to Malcolm

• Enhancements

  • configurable dark mode for OpenSearch Dashboards (idaholab#145)
  • added third-party OpenSearch Dashboards custom visualization component lguillaud/osd_transform_vis
  • modbus and modbus_detailed logs should be better normalized for event.action and event.result (idaholab#146)
  • Added -n argument to script/logs akin to tail -n (#234, thanks @Njinx)
  • Accounted for major additions to the OPCUA-Binary parser in both parsing and the corresponding dashboard
  • Set state:storeInSessionStorage to true for OpenSearch dashboards: this allows some complicated visualizations to be built with the Vega and Transform plugins, at the cost of having some URL bookmarks not contain every possible state the current dashboard has
  • Added related.device_name for normalization and pivoting
  • Removed related.segment in favor of ECS network.name
  • allow NetBox in Malcolm's "read-only" configuration

• Component version updates

  • Arkime to v4.1.0
  • OpenSearch and OpenSearch Dashboards to v2.5.0
  • Beats to v8.6.1
  • Zeek to v5.0.6
  • OpenSearch-Py to v2.1.1

• Fixes

  • failure to build logstash container due to illformed gem requirement (idaholab#144)
  • when running as UID/GID other than 1000, chown on dashboards and logstash containers takes a LONG time (idaholab#148)
  • Logs are being spammed with Suricata warnings pertaining to duplicate rules (#233)
  • Opensearch statistics are now parsed correctly when only a one node is present (#232, thanks @Njinx)
  • Explicitly check /usr/bin for docker-compose in case for some reason that's not in PATH (?) (#226)
  • Some refactoring of the Zeek pipeline in Logstash

Malcolm and Hedgehog Linux may be obtained by pulling or building the Docker images and/or building the ISO installer images as described in the documentation. Unofficial ISO installer images for Malcolm and Hedgehog Linux are not hosted on GitHub, but may be downloaded from https://malcolm.fyi/.