[HOT FIX] v0.1.7-alpha03042019

[HOT FIX] 03042019

fix https://github.com/Cyb3rWard0g/HELK/issues/215
- Logstash plugins offline install (default)
- Logstash mutate statements update
- ES Memory Calculation fix
- Compose files typo

[HOT FIX ] Logstash Pipeline

[HOT FIX] v0.1.7-alpha02262019 - Logstash Pipeline

helk-logstash
+ Added offline plugins file
+ Updated win security conversion
+ cleaned process-name filter & process-name-split configs
+ cleaned process-id filter & proces-id conversion configs
+ set kafka max poll records to 500
+ updated SOURCE_ & TARGET_ field names from process entity to be renamed process_source_ and process_target. Following the basic `entity_context_property` from OSSEM CIM

Winter is coming Release

v0.1.7-alpha02242019

[Alpha] v0.1.7-alpha02242019

[Hot Fix] 02022019

[HOT-FIX] 02022019

helk-Elasticsearch
- Adjusted ES JAVA OPTs (Heap size) calculations

helk-jupyter
+ Upgraded image to 0.1.0
+ Updated graphframes to 0.7.0
+ fix https://github.com/Cyb3rWard0g/HELK/issues/161
+ fix https://github.com/Cyb3rWard0g/HELK/issues/163

helk-logstash
+ fix https://github.com/Cyb3rWard0g/HELK/issues/162

[Hot Fix] 01312019

[HOT FIX] 01312019

helk ELK
Updated to version 6.5.4

helk-logstash
fix https://github.com/Cyb3rWard0g/HELK/issues/156
+ Pipeline Updated
++ More security events
++ Reduced regex complexity to split process paths to process names
++ Enabled Kafka output again for Win Security and Win Sysmon logs
++ Added more win security conversion events

helk-elastalert
fix https://github.com/Cyb3rWard0g/HELK/issues/157
fix https://github.com/Cyb3rWard0g/HELK/issues/159

ELK:
+ Consolidated ELK scripts to one per container instead of trial and basic

helk-sigma
+ Updated own fork

helk-jupyter
+ Updated Elastic ES-Hadoop to 6.5.4

helk-jupyter
+ jupyterlab-manager widgets
+ Updated pandas 0.24.0
+ Updated altair 2.3.0

CHRISTMAS RELEASE - v0.1.6-alpha12132018 - PRESENT #1

Updating README

Added Elastalert and Sigma information

v0.1.3-alpha08242018

Docker-compose Files Version
+ Updated version to 3.5

Base Docker Ubuntu Image
+ Updated to phusion/baseimage version 0.11 (https://github.com/phusion/baseimage-docker/releases/tag/0.11)

HELK base image
+ Updated to 0.0.2 due to Ubuntu upgrade

HELK ELK Version
+ Now using 6.4.0 official ELK Docker Images (https://www.elastic.co/blog/elastic-stack-6-4-0-released?blade=tw&hulk=social)

helk_install
+ Fixed https://github.com/Cyb3rWard0g/HELK/issues/99

helk-elasticsearch
+ Updated main yml config to set most of the settings via environment variables via docker-compose
+ Trial docker-compose file now has ELASTICSEARCH_PASSWORD environment variable set/available. Trial Dockerfile was deleted since the elasticsearch_password update is now taken care of by the internal elasticsearch docker script that is comes with the official elasticsearch docker image.
+ reduced the memory requirements from 4GB to 2GB

helk-logstash
+ entrypoint scripts remove kafka output plugin 7.1.2 and installs version 7.1.1 due to https://github.com/logstash-plugins/logstash-output-kafka/pull/198
++ this error happens right after upgrading ELK built from 6.3.2 to 6.4.0

helk-jupyter
+ Added Altair python package
+ updated Jupyterlab to 0.34.1
+ updated jupyterhub to 0.9.2
+ updated jupyterlab hub extension to 0.11.0
+ updated Spark config to use Graphframes 0.6.0 (https://graphframes.github.io/user-guide.html)
+ updated spark-kafka library to spark-sql-kafka-0-10_2.11:2.3.1

helk-kafka-base
+ updated Kafka to 2.0.0 (this affects Kafka brokers and zookeeper)
+ Created user kafkauser to run kafka containers as non-root

helk-kafka-broker
+ split entrypoint script to have topics creation separate
++ auomated the way how the container checks for the kafka broker port availability. If the port is open, then it attempts to create kafka topics
+ No need to tail kafka logs to keep the container alive after running the kafka start script. It now just starts the broker via Dockerfile CMD command and stays alive.

helk-zookeeper
+ updated entrypoint to only set the main server config
+ zookeeper is now started via Dockerfile CMD command

Las Vegas 2018-b

v0.1.2-alpha08062018

Updated Logstash output templates to replace _doc mappings to doc.

Las Vegas 2018-a

HELK v0.1.3-alpha08042018

Removed Zeppelin Folder and removed wrong password string displayed for Jupyterhub after installing HELK.

Las Vegas 2018

HELK v0.1.3-alpha08032018

All
+ Moved all to docker folder. Getting ready to start sharing other ways to deploy helk (terraform & Packer maybe)

Compose-files
+ Basic & Trial Elastic Subscriptions available now and can be automatically managed via the helk_install script

ELK Version : 6.3.2

Elasticsearch
+ Set 4GB for ES_JAVA_OPTS by default allowing the modification of it via docker-compose and calculating half of the host memory if it is not set
+ Added Entrypoint script and using docker-entrypoint to start ES

Logstash
+ Big Pipeline Update by Nate Guagenti (@neu5ron)
++better cli & file name searching
++”dst_ip_public:true” filter out all rfc1918/non-routable
++Geo ASName
++Identification of 16+ windows IP fields
++Arrayed IPs support
++IPv6&IPv4 differentiation
++removing “-“ values and MORE!!!
++ THANK YOU SO MUCH NATE!!!
++ PR: https://github.com/Cyb3rWard0g/HELK/pull/93
+ Added entrypoint script to push new output_templates straight to Elasticsearch per Nate's recommendation
+ Starting Logstash now with docker-entrypoint
+ "event_data" is now taken out of winlogbeat logs to allow integration with nxlog (sauce added by Nate Guagenti (@neu5ron)

Kibana
+ Kibana yml file updated to allow a longer time for timeout

Nginx:
+ it handles communications to Kibana and Jupyterhub via port 443 SSL
+ certificate and key get created at build time
+ Nate added several settings to improve the way how nginx operates

Jupyterhub
+ Multiple users and mulitple notebooks open at the same time are possible now
+ Jupytehub now has 3 users hunter1,hunter2.hunter3 and password patterh is <user>P@ssw0rd!
+ Every notebook created is also JupyterLab
+ Updated ES-Hadoop 6.3.2

Kafka Update
+ 1.1.1 Update

Spark Master + Brokers
+ reduce memory for brokers by default to 512m

Resources:
+ Added new images for Wiki