Skip to content

Releases: cisagov/Malcolm

Malcolm v24.12.0

19 Dec 15:28
@mmguero mmguero
v24.12.0
e1afaec
This commit was signed with the committer’s verified signature.
mmguero Seth Grover
GPG key ID: 986055F72F110479
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v24.12.0 Latest
Latest

Malcolm v24.12.0 contains several improvements to the Malcolm configuration script, the Malcolm user interface, and the Malcolm API, as well as component version updates and bug fixes. This release also corresponds with the release of malcolm-test (#486), a Malcolm system testing framework.

v24.11.0...v24.12.0

  • Features and enhancements
    • Creation of a Malcolm systems testing framework (#486)
    • Added a number of Zeek packages to detect various CVEs
    • Improvements to the Indices, Ready, and Document Ingest Statistics APIs
    • Use new arkime tag-hiding feature to hide netbox tag from UI (#495)
    • Provide configuration script options for pulling from threat intel feeds (#532)
    • Prompt during configuration whether to enable capture statistics (#504)
    • Add additional EVTX fields to index template (#525) and minor improvements to normalization
    • Add simple readiness indicator to upload page (#528)
    • Add option to upload page to disable NetBox enrichment for the currently-uploaded batch of PCAPs
    • Expose more of the Logstash API passthrough to the Malcolm API
  • Component version updates
  • Bug fixes
    • Zeek DNS records don't open correctly in Arkime sessions (#509)
    • Mandiant threat intel source doesn't get split correctly when using JSON zeek log format (#494)
    • Set indices.query.bool.max_clause_count to 8192 to reflect maximum number of fields
    • Increase Java stack size (-Xss) for Logstash from 1536k to 2048k
    • Minor fixes for parsing Zeek intel.log (some fields not named correctly with Zeek JSON-formatted logs)
    • Fixes to some Zeek dns.log parsing conflicts between ECS's DNS fields and what the Arkime schema is expecting
    • Fixed setting the Signature event severity tags
  • Code and project maintenance
    • Replaced hard-coded Malcolm version number in documentation markdown files with variable-based replacer populated during generation
    • Documentation and screenshot updates

Official ISO installer images for Malcolm and Hedgehog Linux can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split into 2GB chunks and can be reassembled with scripts provided for both Bash (release_cleaver.sh) and PowerShell (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

Assets 18
0 Join discussion

Malcolm v24.11.0

18 Nov 17:55
@mmguero mmguero
v24.11.0
891cb15
This commit was signed with the committer’s verified signature.
mmguero Seth Grover
GPG key ID: 986055F72F110479
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v24.11.0

Malcolm v24.11.0 contains a new threat intelligence feed integration, a few new API calls, other minor improvements, bug fixes, and component version updates.

v24.10.1...v24.11.0

Official ISO installer images for Malcolm and Hedgehog Linux can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split into 2GB chunks and can be reassembled with scripts provided for both Bash (release_cleaver.sh) and PowerShell (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

Contributors

robrui
Assets 18
Loading
0 Join discussion

Malcolm v24.10.1

24 Oct 16:05
@mmguero mmguero
v24.10.1
cb0d34e
This commit was signed with the committer’s verified signature.
mmguero Seth Grover
GPG key ID: 986055F72F110479
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v24.10.1

Malcolm v24.10.1 contains some minor improvements, a few component version updates, a fix for a regression bug, and a fair amount of code cleanup.

v24.10.0...v24.10.1

  • Features and enhancements
  • Component version updates
  • Bug fixes
    • Fixed OpenSearch anomaly detection default detectors not being created (regression, idaholab#596)
  • Configuration changes (in environment variables in ./config/) for Malcolm and in control_vars.conf for Hedgehog Linux
    • Malcolm
    • Hedgehog Linux
      • ZEEK_JA4SSH_PACKET_COUNT has been added to control_vars.conf for the same purpose as described above
  • Code and project maintenance
    • Examine distro hardening, fix and update documentation as needed for Malcolm and Hedgehog Linux ISO-installed environments (idaholab#328)
    • Refactoring and code cleanup in the Logstash Zeek pipeline (idaholab#592)
    • Logstash container initialization code now automatically ensures that the Zeek TSV log parsing filters (dissect and split filters) in these files are looking for TAB characters (i.e., automatically replace spaces with tabs in these filter files in case the author forgot to do so) (idaholab#592)
    • Did some code cleanup in the ./shared/bin directory, mostly moving things that were specific to either the Malcolm or Hedgehog Installer ISO environments out of shared and into their respective locations for the ISO installer build.
    • When doing the aquasecurity/trivy-action action, use TRIVY_DB_REPOSITORY: ghcr.io/aquasecurity/trivy-db,public.ecr.aws/aquasecurity/trivy-db to try to fall back to an alternative official location for the vulnerability database if the first one fails. Also, pin this action to the v0.28.0 release rather than setting it to master.
    • As it's used pretty ubiquitously in shared scripts by many of the Malcolm containers, the jq utility is now installed across the board during the container image build.
    • Added a script to gather GitHub API metrics for Malcolm downloads (idaholab#594)

Official ISO installer images for Malcolm and Hedgehog Linux can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split into 2GB chunks and can be reassembled with scripts provided for both Bash (release_cleaver.sh) and PowerShell (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

Assets 17
Loading
0 Join discussion

Malcolm v24.10.0

09 Oct 15:34
@mmguero mmguero
v24.10.0
556ac85
This commit was signed with the committer’s verified signature.
mmguero Seth Grover
GPG key ID: 986055F72F110479
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v24.10.0

Malcolm v24.10.0 contains fixes for a few regression bugs, minor improvements, and a few component updates.

v24.09.0...v24.10.0

  • Features and enhancements
    • Enable Zeek's parsing of HTTP server and client header names as zeek.http.client_header_names and zeek.http.server_header_names
    • Bumped maximum field limit in OpenSearch templates from 5000 to 6000
    • Some documentation improvements
    • Build improvement: fall back to alternative Zeek .deb download URL (idaholab#585)
    • Build improvement: limit threads for spicy build processes during Zeek package installation (idaholab#571)
  • Component version updates
  • Bug fixes
    • Fix broken dashboards regression from v24.09.0 (idaholab#588)
    • Fix Zeek-extracted files not getting saved to correct location for live Zeek capture (idaholab#590)
    • Fix for building Hedgehog Linux for Raspberry Pi 4 on an M2 MacBook

Official ISO installer images for Malcolm and Hedgehog Linux can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split into 2GB chunks and can be reassembled with scripts provided for both Bash (release_cleaver.sh) and PowerShell (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

Assets 17
Loading
0 Join discussion

Malcolm v24.09.0

19 Sep 19:57
@mmguero mmguero
v24.09.0
150674d
This commit was signed with the committer’s verified signature.
mmguero Seth Grover
GPG key ID: 986055F72F110479
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v24.09.0

Malcolm v24.09.0 contains new features and enhancements, component version updates, and bug fixes.

v24.08.0...v24.09.0

Official ISO installer images for Malcolm and Hedgehog Linux can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split into 2GB chunks and can be reassembled with scripts provided for both Bash (release_cleaver.sh) and PowerShell (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

Contributors

divinehawk
Assets 17
Loading
0 Join discussion

Mal.Con24 Presentations

19 Sep 18:58
@mmguero mmguero
Mal.Con24
9384e22
This commit was signed with the committer’s verified signature.
mmguero Seth Grover
GPG key ID: 986055F72F110479
Learn about vigilant mode.
Compare
Choose a tag to compare
Mal.Con24 Presentations

Here are the slide decks for the presentations that were given at Mal.Con24. Enjoy!

Assets 10
Loading

Malcolm v24.08.0

27 Aug 19:38
@mmguero mmguero
v24.08.0
8072201
This commit was signed with the committer’s verified signature.
mmguero Seth Grover
GPG key ID: 986055F72F110479
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v24.08.0

Malcolm v24.08.0 contains minor improvements, some component version updates, and bug fixes.

v24.07.0...v24.08.0

  • Features and enhancements
    • in ISO installer, prompt to format other drives for artifact storage rather than just doing it automatically (idaholab#529)
    • allow users to more easily add NetBox plugins (idaholab#530)
    • run netbox-initializers plugin on startup even if we're doing a netbox database backup preload (idaholab#531)
    • during auth_setup "all" operation, do required operations without prompting if the files don't already exist (idaholab#536)
    • some containers need resource request specified for Kubernetes (idaholab#539)
    • add "public" pseudo-segments for public IP addresses (idaholab#542)
    • reworked Windows Event dashboard
    • some documentation updates
    • added netbox tag to any logs that are passed into the netbox_enrich.rb script in the Logstash enrichment pipeline
  • Component version updates
  • Bug fixes
    • dashboards-helper container's use of curl fails internal container name resolution when host has invalid DNS settings, prevents Malcolm initialization (idaholab#499)
    • Netbox service templates not populating (idaholab#522)
    • kubernetes manifest for netbox refers to netbox-netmap-json configmap which no longer exists (idaholab#540)
    • don't try to expose the OpenSearch port 9200 in docker-compose.yml when the database mode is not opensearch-local
    • improved the liveness check for the offline Zeek container so that it returns "healthy" if the intel thread feeds are still pulling before the monitoring processes start up
    • missing cracklib-runtime package prevents ISO service account password from being updated by non-root user (idaholab#548)

Official ISO installer images for Malcolm and Hedgehog Linux can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split into 2GB chunks and can be reassembled with scripts provided for both Bash (release_cleaver.sh) and PowerShell (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

Assets 17
Loading

Malcolm v24.07.0

30 Jul 21:38
@mmguero mmguero
v24.07.0
7b27173
This commit was signed with the committer’s verified signature.
mmguero Seth Grover
GPG key ID: 986055F72F110479
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v24.07.0

Malcolm v24.07.0 contains minor improvements, some component version updates, and a few bug fixes.

v24.06.0...v24.07.0

  • Features and enhancements
    • integrated the ICSNPP GE SRTP network analyzer (idaholab#516)
    • Changed the way docker compose does bind mounts of files and directories to avoid creating empty directories when the source is missing, returning an error instead (idaholab#473)
      • This changed necessitated a switch from Python's built-in YAML library to ruamel.yaml
    • code to pull from MISP feeds should specify JSON as preferred format in HTTP headers (idaholab#520)
    • add optional service argument to restart script (idaholab#521)
    • replace API link on landing page with extracted-files (idaholab#524)
    • exclude private IP space Intel::ADDR items when populating Zeek intel (idaholab#528)
    • updated some screenshots for the documentation
  • Component version updates
  • Bug fixes
    • tarball-based installation should not depend on UID inside of tarball, prevents installation if UID with which tarball's contents were created don't match installing user's (idaholab#519)
    • bacnet discovery log not parsed correctly (idaholab#523)
    • resolved issue with the build.sh helper script when building non-AMD64 Docker images
  • Configuration changes (in environment variables in ./config/)
    • The variable ZEEK_DISABLE_ICS_GE_SRTP has been added to zeek.env and control_vars.conf to control enabling the network analyzer for the GE SRTP protocol. It's default value is true (indicating that the analyzer is disabled) as it is a somewhat uncommon OT protocol that likely won't be needed by most Malcolm users.
  • Other
    • Removed long-deprecated net-map.json file support (idaholab#517)

Official ISO installer images for Malcolm and Hedgehog Linux can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split into 2GB chunks and can be reassembled with scripts provided for both Bash (release_cleaver.sh) and PowerShell (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

Assets 17
Loading
0 Join discussion

Malcolm v24.06.0

27 Jun 01:18
@mmguero mmguero
v24.06.0
d745e24
This commit was signed with the committer’s verified signature.
mmguero Seth Grover
GPG key ID: 986055F72F110479
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v24.06.0

Malcolm v24.06.0 contains new features, improvements, component version updates, and a few bug fixes.

v24.05.0...v24.06.0

NetBox: backwards compatibility-breaking change: This release of Malcolm updates NetBox from v3.6.7 to v4.0.6, for bug fixes, security updates, and requirements for Malcolm to support enrichment with multiple NetBox sites. However, NetBox's built-in migrations do not appear to work handle going from v3.6.7 to v4.0.6. It is likely that if you are using NetBox that you will encounter errors upon updating to this release of Malcolm. Prior to upgrading it is recommended that you navigate to Sites, IPAM > Prefixes, DCIM > Devices, and anywhere else you've populated NetBox data and click Export > All Data (CSV) and save those in case you need to recreate your NetBox inventory after upgrading. Malcolm's NetBox backup and restore will not work in this case. If you find NetBox has data errors after upgrading Malcolm, stop Malcolm and clear the NetBox inventory from your Malcolm installation directory (e.g., rm -rf ./netbox/postgres/* ./netbox/redis/*), then start Malcolm and recreate your NetBox inventory.

  • Features and enhancements
    • Support for multiple NetBox sites (idaholab#449)
      • Malcolm now supports enrichment from a NetBox inventory for asset interaction analysis across multiple sites. The NetBox site can be specified for uploaded PCAP, for a Hedgehog Linux sensor, and for Malcolm live capture.
    • JA4+ replaces the JA3 TLS fingerprinting standard from 2017 (see also this blog post) (idaholab#419)
    • Support uploading Windows Event Log evtx files (idaholab#465) and update associated dashboard
    • Document using GitHub runners to build Malcolm images (for contributors' guide, idaholab#491)
    • Generate new forwarder SSL keys on-the-fly when transferring between Malcolm and Hedgehog Linux (idaholab#492)
    • Incorporate ATT&CK-based Control-system Indicator Detection for Zeek (ACID) (idaholab#489), a collection of Operational Techonology (OT) protocol indicators developed to alert on specific ATT&CK for ICS behaviors
    • Add platform architecture and machine boot time to Malcolm version API
    • Add links to the navigation pane of most dashboards to "other" dashboards for non-network log data (e.g., resource monitoring, Windows Event logs, etc.)
  • Component version updates
  • Bug fixes
    • Arkime viewer not rolling PCAPs (idaholab#484)
    • Free up space in GitHub runner environment building ISO images to avoid errors due to exhausted disk space
  • Configuration changes in environment variables
    • There are no significant changes or additions to the ./config/*.env environment variable files in Malcolm v24.06.0

Official ISO installer images for Malcolm and Hedgehog Linux can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split into 2GB chunks and can be reassembled with scripts provided for both Bash (release_cleaver.sh) and PowerShell (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

Assets 19
Loading

Malcolm v24.05.0

30 May 02:38
@mmguero mmguero
v24.05.0
f54cfd8
This commit was signed with the committer’s verified signature.
mmguero Seth Grover
GPG key ID: 986055F72F110479
Learn about vigilant mode.
Compare
Choose a tag to compare
Malcolm v24.05.0

Malcolm v24.05.0 contains new features, improvements, bug fixes and component version updates.

v24.04.0...v24.05.0

  • Features and enhancements
    • Added ARM64/AArch64 support. Malcolm can now run natively on ARM64 hardware. The ./scripts/configure script should detect the architecture and automatically adjust the image: names in the docker-compose.yml files in Docker deployments, or this can be changed manually by appending -arm64 to the tag for Malcolm's Docker images, e.g., ghcr.io/idaholab/malcolm/zeek:24.05.0-arm64. (idaholab#369)
    • Support for new environment variables added to Hedgehog Linux's control_vars.conf for tuning capture-related settings for Arkime on the sensor, bringing them into parity with those that were available in the arkime-live container in Malcolm. (idaholab#476)
    • Tweaked some of the default resource-related live capture settings for Suricata and Arkime.
    • Reworked the environment variables used for tuning Zeek live capture resource and performance on both Malcolm and Hedgehog Linux. An in-depth discussion of these tuning parameters can be found in the documentation. (idaholab#475)
    • Allow setting the spiDataMaxIndexes variable for Arkime's config.ini file via the ARKIME_SPI_DATA_MAX_INDICES environment variable. (idaholab#471)
    • Allow custom tags to be specified at the point of log file ingestion (i.e., FileBeat) on Malcolm and Hedgehog Linux. This makes it easier to specify custom tags used to group network traffic by sensor. (idaholab#463)
    • Handle invalid URLs made to the Malcolm web-based UIs better (with a custom 404/502 page). (idaholab#461)
    • Switched to official .deb packages for Arkime rather than building from source, reducing build times significantly. (Thanks @awick.)
  • Component version updates
    • Suricata to v7.0.5
      • Also, going forward Malcolm will track the latest Suricata release (from the Debian Stable Backports APT repository) rather than what's in the Debian Stable APT repository. (idaholab#462)
    • Arkime to v5.2.0
    • OpenSearch and OpenSearch Dashboards to v2.14.0
    • YARA to v4.5.1
    • Beats to v8.13.4
    • Logstash to v8.13.4
    • YQ to v4.44.1
    • Zeek to v6.2.1
    • Fluent Bit to v3.0.6
    • requests Python library to v2.32.0 for CVE-2024-35195
    • flask-cors Python library on Hedgehog Linux to v4.0.1 for CVE-2024-1681
    • Jinja Python library on Hedgehog Linux to v3.1.4 for CVE-2024-34064
    • Werkzeug Python library on Hedgehog linux to v3.0.3 for CVE-2024-34069
  • Bug fixes
    • The code that cleans up already-processed Zeek and Suricata logs after a defined period of time was out of date for the current FileBeat registry behavior and would potentially leave log files around longer than they needed to be. This has been remedied. (idaholab#479)
    • Fixed issue where the BPF capture filter was not passed to Zeek correctly. (idaholab#474)
    • The process which queries threat intelligence feeds and generates the corresponding Zeek intel files will no longer relpace existing intel definitions unless it succeeds in pulling definitions from at least one of the specified feeds. (idaholab#472)
    • Fixed calculation of memory and CPU resources used in ./scripts/status for Kubernetes deployment. (idaholab#467)
  • Configuration changes (in environment variables in ./config/) for Malcolm and in control_vars.conf for Hedgehog Linux
    • Malcolm
      • Added ARKIME_SPI_DATA_MAX_INDICES to arkime.env with a default value of 7, which manifests as spiDataMaxIndexes in Arkime's config.ini. If you are changing the Arkime index period from daily to weekly, hourly, etc., you may wish to adjust this value. (idaholab#471)
      • Added EXTRA_TAGS to upload-common.env for specifying custom tags to be associated with logs forwarded to Logstash by FileBeat. (idaholab#463)
      • A number of new and modified environment variables are available and can be added to zeek-live.env for tuning the resource utilization and performance of Zeek's live capture, see the documentation for details. (idaholab#475)
    • Hedgehog Linux

Official ISO installer images for Malcolm and Hedgehog Linux can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split into 2GB chunks and can be reassembled with scripts provided for both Bash (release_cleaver.sh) and PowerShell (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

Contributors

awick
Assets 19
Loading