Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Idea: Terminate ./script/start once Malcolm has started properly. #240

Closed
Njinx opened this issue Feb 21, 2023 · 2 comments
Closed

Idea: Terminate ./script/start once Malcolm has started properly. #240

Njinx opened this issue Feb 21, 2023 · 2 comments
Assignees
@mmguero
Labels
enhancement New feature or request

Comments

@Njinx
Copy link
Contributor

Njinx commented Feb 21, 2023

I find ./script/start a bit clunky because you never really know when Malcolm is finished starting and able to be interacted with. Would you be open to changing the default behavior (or creating a setting) so that ./scripts/start will terminate once Malcolm is fully started? I'd be happy to try and implement this myself, but I'd like to know if anybody is opposed to this beforehand.
@Njinx Njinx changed the title Terminate ./script/start once Malcolm has started properly. Idea: Terminate ./script/start once Malcolm has started properly. Feb 21, 2023
@mmguero
Copy link
Collaborator

mmguero commented Feb 22, 2023

yeah that's a cool idea. "Fully started" could depend on your opinion of what that means, but generally for me I look for:

malcolm-logstash-1  | [2022-07-27T20:27:52,056][INFO ][logstash.agent           ] Pipelines running {:count=>6, :running_pipelines=>[:"malcolm-input", :"malcolm-output", :"malcolm-beats", :"malcolm-suricata", :"malcolm-enrichment", :"malcolm-zeek"], :non_running_pipelines=>[]}

The "Pipelines running" script from logstash is usually the last indication that Malcolm is fully ready to process data.

You're welcome to take a stab at it, or I think I could take care of it this Friday once I get back from the travel I'm on.

Cheers

@mmguero mmguero self-assigned this Feb 22, 2023
@mmguero mmguero added the enhancement New feature or request label Feb 22, 2023
@Njinx Njinx mentioned this issue Feb 23, 2023
Closed
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Feb 27, 2023
@mmguero
minor tweaks for the changes for start/restart exiting logs after sta…
9248f78 
…rtup is "finished". for cisagov#241 and cisagov#240.
This was referenced Mar 8, 2023
Merged
Merged
@mmguero
Copy link
Collaborator

mmguero commented Mar 8, 2023

Going into v23.03.0 release forthcoming

@mmguero mmguero closed this as completed Mar 8, 2023
mmguero added a commit to idaholab/Malcolm that referenced this issue Mar 8, 2023
@mmguero
Merge pull request #156 from mmguero-dev/v23.03.0_merge_idaholab
4adf3c9 
Malcolm v23.03.0 is a release with enhancements, component version updates and bug fixes.

* Enhancements
    - Replace Zeek's [misc/scan.zeek with ncsa/bro-simple-scan](https://github.com/zeek/zeek/blob/cdadc329859810244244c8800f0102543e4f134f/NEWS#L540-L541)
    - terminate `start` and `restart` scripts once Malcolm has started properly (cisagov#240 and cisagov#241, thanks @Njinx)
    - minor usability improvements for ISO-installed Malcolm and Hedgehog (#155)
        + Added a "Configure Malcolm" menu item (under the "Internet" GTK menu with the other Malcolm stuff) and launcher on the top panel of icons in Malcolm. This runs `./scripts/install.py --configure` in full screen. May look at starting this automatically on first boot in the future. (Malcolm)
        + Added Malcolm shortcut to gtk-3.0/bookmarks so it shows up in Thunar sidebar (Malcolm)
        + Added /opt/sensor/sensor_ctl shortcut to gtk-3.0/bookmarks so it shows up in Thunar sidebar (Hedgehog)
        + Have tilix from launcher panel start in /opt/sensor/sensor_ctl (Hedgehog)
    - minor tweaks to defaults for `install.py --configure` (enable offline-capable file scanners by default)
    - interrupt NetBox startup import script when `netbox-restore` is run
    - added NetBox restore logic to `reset_and_auto_populate.sh` script (used mostly for demos and presentations)

* Component version updates
    - Arkime to [v4.2.0](https://github.com/arkime/arkime/blob/93c89d68b25a4a56f7a6b8099a2661af9648ebaf/CHANGELOG#L39-L66)
    - [OpenSearch](https://github.com/opensearch-project/OpenSearch/blob/bc50a2edcf29c3c41b7a777575c61e1874847d8a/release-notes/opensearch.release-notes-2.6.0.md) and [OpenSearch Dashboards](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/69bcbfeea9bb345364e47f048cd5bcfe64c9c242/release-notes/opensearch-dashboards.release-notes-2.6.0.md) to 2.6.0
    - [Logstash](https://www.elastic.co/guide/en/logstash/current/releasenotes.html) from v8.4.0 to v8.6.1
    - [Beats](https://www.elastic.co/guide/en/beats/libbeat/current/release-notes-8.6.2.html) to v8.6.2
    - Zeek to [v5.0.7](https://github.com/zeek/zeek/releases/tag/v5.0.7)
    - OpenSearch-Py to [v2.2.0](https://github.com/opensearch-project/opensearch-py/releases/tag/v2.2.0) (and remove opensearch-dsl which is now part of opensearch-py)
    - Supercronic to [v0.2.2](https://github.com/aptible/supercronic/releases/tag/v0.2.2)
    - Capa to [v5.0.0](https://github.com/mandiant/capa/releases/tag/v5.0.0)
    - Fluent Bit to [v2.0.9](https://github.com/fluent/fluent-bit/releases/tag/v2.0.9)
    - Version updates to various Python package dependencies

* Fixes
    - last few seconds' Zeek logs prior to log rotation may be lost (#151)
    - in ISO-packaged Malcolm installation `scripts` directory, symlink `netbox-backup` and `netbox-restore` to `control.py`
    - improve opensearchpy connect/health check logig in `pcap_watcher.py` in `pcap-monitor` container
mmguero added a commit that referenced this issue Mar 8, 2023
@mmguero
Merge pull request #243 from mmguero-dev/v23.03.0_merge_cisagov
50cf2eb 
Malcolm v23.03.0 is a release with enhancements, component version updates and bug fixes.

* Enhancements
    - Replace Zeek's [misc/scan.zeek with ncsa/bro-simple-scan](https://github.com/zeek/zeek/blob/cdadc329859810244244c8800f0102543e4f134f/NEWS#L540-L541)
    - terminate `start` and `restart` scripts once Malcolm has started properly (#240 and #241, thanks @Njinx)
    - minor usability improvements for ISO-installed Malcolm and Hedgehog (idaholab#155)
        + Added a "Configure Malcolm" menu item (under the "Internet" GTK menu with the other Malcolm stuff) and launcher on the top panel of icons in Malcolm. This runs `./scripts/install.py --configure` in full screen. May look at starting this automatically on first boot in the future. (Malcolm)
        + Added Malcolm shortcut to gtk-3.0/bookmarks so it shows up in Thunar sidebar (Malcolm)
        + Added /opt/sensor/sensor_ctl shortcut to gtk-3.0/bookmarks so it shows up in Thunar sidebar (Hedgehog)
        + Have tilix from launcher panel start in /opt/sensor/sensor_ctl (Hedgehog)
    - minor tweaks to defaults for `install.py --configure` (enable offline-capable file scanners by default)
    - interrupt NetBox startup import script when `netbox-restore` is run
    - added NetBox restore logic to `reset_and_auto_populate.sh` script (used mostly for demos and presentations)

* Component version updates
    - Arkime to [v4.2.0](https://github.com/arkime/arkime/blob/93c89d68b25a4a56f7a6b8099a2661af9648ebaf/CHANGELOG#L39-L66)
    - [OpenSearch](https://github.com/opensearch-project/OpenSearch/blob/bc50a2edcf29c3c41b7a777575c61e1874847d8a/release-notes/opensearch.release-notes-2.6.0.md) and [OpenSearch Dashboards](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/69bcbfeea9bb345364e47f048cd5bcfe64c9c242/release-notes/opensearch-dashboards.release-notes-2.6.0.md) to 2.6.0
    - [Logstash](https://www.elastic.co/guide/en/logstash/current/releasenotes.html) from v8.4.0 to v8.6.1
    - [Beats](https://www.elastic.co/guide/en/beats/libbeat/current/release-notes-8.6.2.html) to v8.6.2
    - Zeek to [v5.0.7](https://github.com/zeek/zeek/releases/tag/v5.0.7)
    - OpenSearch-Py to [v2.2.0](https://github.com/opensearch-project/opensearch-py/releases/tag/v2.2.0) (and remove opensearch-dsl which is now part of opensearch-py)
    - Supercronic to [v0.2.2](https://github.com/aptible/supercronic/releases/tag/v0.2.2)
    - Capa to [v5.0.0](https://github.com/mandiant/capa/releases/tag/v5.0.0)
    - Fluent Bit to [v2.0.9](https://github.com/fluent/fluent-bit/releases/tag/v2.0.9)
    - Version updates to various Python package dependencies

* Fixes
    - last few seconds' Zeek logs prior to log rotation may be lost (idaholab#151)
    - in ISO-packaged Malcolm installation `scripts` directory, symlink `netbox-backup` and `netbox-restore` to `control.py`
    - improve opensearchpy connect/health check logig in `pcap_watcher.py` in `pcap-monitor` container
mmguero added a commit that referenced this issue Feb 14, 2024
@mmguero
Merge pull request #240 from idaholab/v23.08.1_merge_idaholab
2175872 
Malcolm v23.08.1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mmguero mmguero

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Terminate ./script/start once Malcolm has started properly. Njinx/Malcolm
2 participants
@Njinx @mmguero