List of changes in Malcolm v3.0.0:

- Change base for Elasticsearch and Kibana Docker images (version 7.6.2) from Elastic.co to Open Distro for Elastic (based on Elastic 7.10.0); see #15. This is a major change which **breaks backwards compatibility** for several features (listed below). If you are using these features, you will need to back up the data and/or configuration associated with them and migrate them manually to the new tools. No automatic migration or upgrade of these features is performed. It's recommended that you re-run `install.py --configure` (see [System configuration and tuning](https://github.com/cisagov/Malcolm#ConfigAndTuning)) prior to running Malcolm v3.0.0.
  - Kibana [comments](https://github.com/gwintzer/kibana-comments-app-plugin) replaced with [Notebooks](https://opendistro.github.io/for-elasticsearch-docs/docs/notebooks/)
  - Kibana [elastalert](https://github.com/nsano-rururu/elastalert-kibana-plugin) plugin replaced with [Alerting](https://opendistro.github.io/for-elasticsearch-docs/docs/alerting/) plugin
  - Elasticsearch [curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html) replaced with [Index Management](https://opendistro.github.io/for-elasticsearch-docs/docs/ism/) plugin
  - The third-party [Sankey visualization plugin](https://github.com/mmguero-dev/kbn_sankey_vis] has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see uniberg/kbn_sankey_vis#15)
  - The third-party [Kibana drill-down plugin](https://github.com/mmguero-dev/kibana-plugin-drilldownmenu/) providing Kibana-to-Moloch pivoting has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see goodlabs-studio/kibana-plugin-drilldownmenu#5)
  - In addition to those replacements, the Real Time Anomaly Detection feature is now available:
    - [Real Time Anomaly Detection in Open Distro for Elasticsearch](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/real-time-anomaly-detection-in-open-distro-for-elasticsearch/) blog announcement
     - [Anomaly Detection](https://opendistro.github.io/for-elasticsearch-docs/docs/ad/) documentation and source code for [Elasticsearch](https://github.com/opendistro-for-elasticsearch/anomaly-detection) and [Kibana](https://github.com/opendistro-for-elasticsearch/anomaly-detection-kibana-plugin) components
     - [Random Cut Forests writeup](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/random-cut-forests/)

- Malcolm startup time (especially the Logstash container) has been reduced drastically

- Improvements to Malcolm's prebuilt Kibana dashboards

- Improvements to build scripts

- Minor tweaks and bugfixes for ISO-installed environments for Malcolm and Hedgehog Linux

- Minor other bug fixes and performance improvements

- Version bump
  - Yara [v4.0.5](https://github.com/VirusTotal/yara/releases/tag/v4.0.5)

Dockerfiles/elasticsearch.Dockerfile

@@ -3,10 +3,10 @@ FROM amazon/opendistro-for-elasticsearch:1.12.0
 # Copyright (c) 2021 Battelle Energy Alliance, LLC.  All rights reserved.
 LABEL maintainer="[email protected]"
 LABEL org.opencontainers.image.authors='[email protected]'
-LABEL org.opencontainers.image.url='https://github.com/idaholab/Malcolm'
-LABEL org.opencontainers.image.documentation='https://github.com/idaholab/Malcolm/blob/master/README.md'
-LABEL org.opencontainers.image.source='https://github.com/idaholab/Malcolm'
-LABEL org.opencontainers.image.vendor='Idaho National Laboratory'
+LABEL org.opencontainers.image.url='https://github.com/cisagov/Malcolm'
+LABEL org.opencontainers.image.documentation='https://github.com/cisagov/Malcolm/blob/master/README.md'
+LABEL org.opencontainers.image.source='https://github.com/cisagov/Malcolm'
+LABEL org.opencontainers.image.vendor='Cybersecurity and Infrastructure Security Agency'
 LABEL org.opencontainers.image.title='malcolmnetsec/elasticsearch-od'
 LABEL org.opencontainers.image.description='Malcolm container providing Elasticsearch (the Apache-licensed Open Distro variant)'
 

Dockerfiles/kibana-helper.Dockerfile

@@ -3,10 +3,10 @@ FROM alpine:3.12
 # Copyright (c) 2020 Battelle Energy Alliance, LLC.  All rights reserved.
 LABEL maintainer="[email protected]"
 LABEL org.opencontainers.image.authors='[email protected]'
-LABEL org.opencontainers.image.url='https://github.com/idaholab/Malcolm'
-LABEL org.opencontainers.image.documentation='https://github.com/idaholab/Malcolm/blob/master/README.md'
-LABEL org.opencontainers.image.source='https://github.com/idaholab/Malcolm'
-LABEL org.opencontainers.image.vendor='Idaho National Laboratory'
+LABEL org.opencontainers.image.url='https://github.com/cisagov/Malcolm'
+LABEL org.opencontainers.image.documentation='https://github.com/cisagov/Malcolm/blob/master/README.md'
+LABEL org.opencontainers.image.source='https://github.com/cisagov/Malcolm'
+LABEL org.opencontainers.image.vendor='Cybersecurity and Infrastructure Security Agency'
 LABEL org.opencontainers.image.title='malcolmnetsec/kibana-helper'
 LABEL org.opencontainers.image.description='Malcolm container providing Kibana support functions'
 

Dockerfiles/kibana.Dockerfile

@@ -3,10 +3,10 @@ FROM amazon/opendistro-for-elasticsearch-kibana:1.12.0
 # Copyright (c) 2021 Battelle Energy Alliance, LLC.  All rights reserved.
 LABEL maintainer="[email protected]"
 LABEL org.opencontainers.image.authors='[email protected]'
-LABEL org.opencontainers.image.url='https://github.com/idaholab/Malcolm'
-LABEL org.opencontainers.image.documentation='https://github.com/idaholab/Malcolm/blob/master/README.md'
-LABEL org.opencontainers.image.source='https://github.com/idaholab/Malcolm'
-LABEL org.opencontainers.image.vendor='Idaho National Laboratory'
+LABEL org.opencontainers.image.url='https://github.com/cisagov/Malcolm'
+LABEL org.opencontainers.image.documentation='https://github.com/cisagov/Malcolm/blob/master/README.md'
+LABEL org.opencontainers.image.source='https://github.com/cisagov/Malcolm'
+LABEL org.opencontainers.image.vendor='Cybersecurity and Infrastructure Security Agency'
 LABEL org.opencontainers.image.title='malcolmnetsec/kibana-od'
 LABEL org.opencontainers.image.description='Malcolm container providing Kibana (the Apache-licensed Open Distro variant)'
 

scripts/control.py

@@ -16,6 +16,7 @@
 import sys
 
 from malcolm_common import *
+from contextlib import nullcontext
 from collections import defaultdict, namedtuple
 from subprocess import (PIPE, STDOUT, Popen, check_call, CalledProcessError)
 try: