Test ISM rollover + alias initialization update #83

AlexRuiz7 · 2023-12-21T10:57:28Z

Description

Test the new wazuh-indexer packages for 4.8.0 containing the fix for the rollover + alias initialization bug found in #79.

The packages have been generated by @wazuh/qa team and their links are available here:

We need to test these packages following the installation and upgrade guides for 4.8.0 (master branch contains latest code).

Issues

The text was updated successfully, but these errors were encountered:

AlexRuiz7 · 2023-12-21T11:47:16Z

Upgrade

Vagrantfile

Vagrant.configure("2") do |config|

    config.vm.provider "virtualbox" do |vb|
        vb.memory = "4096"
        vb.cpus = "4"
    end

    config.vm.box = "generic/rhel7"
    config.vm.synced_folder ".", "/vagrant"
    config.vm.network "private_network", ip: "192.168.56.10"
    
    config.vm.provision "shell", inline: <<-SHELL
        # Disable firewall
        systemctl stop firewalld
        systemctl disable firewalld

        curl -sO https://packages.wazuh.com/4.7/wazuh-install.sh && bash ./wazuh-install.sh
    SHELL
end

Install [email protected] using the installation assistant: vagrant up
Follow the upgrade guide, but using a local RPM package for wazuh-indexer:

Clone wazuh-documentation repo. Prepare python's virtual env and build with make:

 git clone [email protected]:wazuh/wazuh-documentation.git
 cd wazuh-documentation
 git checkout 4.8.0
 python3 -m venv venv
 source venv/bin/activate
 pip -r install requirements.txt
 make html
 # Open build/html/index.html in your browser

Download new RPM package for wazuh-indexer. Place it in the same folder as the Vagrantfile.
Follow the upgrade guide:
Continued in the next comment.

AlexRuiz7 · 2023-12-21T11:47:24Z

Upgrade

Procedure (commands)

`wazuh-indexer`

rpm --import https://packages-dev.wazuh.com/key/GPG-KEY-WAZUH
echo -e '[wazuh]\ngpgcheck=1\ngpgkey=https://packages-dev.wazuh.com/key/GPG-KEY-WAZUH\nenabled=1\nname=EL-$releasever - Wazuh\nbaseurl=https://packages-dev.wazuh.com/pre-release/yum/\nprotect=1' | tee /etc/yum.repos.d/wazuh.repo

systemctl stop filebeat
systemctl stop wazuh-dashboard

curl -X PUT "https://0.0.0.0:9200/_cluster/settings"  -u admin:wiarUF8dnrUCiY*yHVIk*J+Z4bDIJdcH -k -H 'Content-Type: application/json' -d'
{
  "persistent": {
    "cluster.routing.allocation.enable": "primaries"
  }
}
'
curl -X POST "https://0.0.0.0:9200/_flush/synced" -u admin:wiarUF8dnrUCiY*yHVIk*J+Z4bDIJdcH -k

systemctl stop wazuh-indexer
yum localinstall /vagrant/wazuh-indexer-4.8.0-wp2697.x86_64.rpm

systemctl daemon-reload
systemctl enable wazuh-indexer
systemctl start wazuh-indexer

curl -k -u admin:wiarUF8dnrUCiY*yHVIk*J+Z4bDIJdcH https://0.0.0.0:9200/_cat/nodes?v
curl -X PUT "https://0.0.0.0:9200/_cluster/settings"  -u admin:wiarUF8dnrUCiY*yHVIk*J+Z4bDIJdcH -k -H 'Content-Type: application/json' -d'
{
  "persistent": {
    "cluster.routing.allocation.enable": "all"
  }
}
'
curl -k -u admin:wiarUF8dnrUCiY*yHVIk*J+Z4bDIJdcH https://0.0.0.0:9200/_cat/nodes?v
bash /usr/share/wazuh-indexer/bin/indexer-init.sh -i 0.0.0.0 -p wiarUF8dnrUCiY*yHVIk*J+Z4bDIJdcH

`wazuh-server`

yum upgrade wazuh-manager
curl -s https://packages-dev.wazuh.com/pre-release/filebeat/wazuh-filebeat-0.4.tar.gz | sudo tar -xvz -C /usr/share/filebeat/module
curl -so /etc/filebeat/wazuh-template.json https://raw.githubusercontent.com/wazuh/wazuh/4.8.0/extensions/elasticsearch/7.x/wazuh-template.json
chmod go+r /etc/filebeat/wazuh-template.json
systemctl daemon-reload
systemctl enable filebeat
systemctl start filebeat
filebeat setup --index-management -E output.logstash.enabled=false

`wazuh-dashboard`

yum upgrade wazuh-dashboard
systemctl daemon-reload
systemctl enable wazuh-dashboard
systemctl start wazuh-dashboard

Logs

[root@rhel7 vagrant]# yum localinstall /vagrant/wazuh-indexer-4.8.0-wp2697.x86_64.rpm 
Failed to set locale, defaulting to C
Loaded plugins: product-id, search-disabled-repos
Examining /vagrant/wazuh-indexer-4.8.0-wp2697.x86_64.rpm: wazuh-indexer-4.8.0-wp2697.x86_64
Marking /vagrant/wazuh-indexer-4.8.0-wp2697.x86_64.rpm as an update to wazuh-indexer-4.7.1-1.x86_64
Resolving Dependencies
--> Running transaction check
---> Package wazuh-indexer.x86_64 0:4.7.1-1 will be updated
---> Package wazuh-indexer.x86_64 0:4.8.0-wp2697 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=====================================================================================================================================================================================================
 Package                                     Arch                                 Version                                     Repository                                                        Size
=====================================================================================================================================================================================================
Updating:
 wazuh-indexer                               x86_64                               4.8.0-wp2697                                /wazuh-indexer-4.8.0-wp2697.x86_64                               1.0 G

Transaction Summary
=====================================================================================================================================================================================================
Upgrade  1 Package

Total size: 1.0 G
Is this ok [y/d/N]: y

Updated:
  wazuh-indexer.x86_64 0:4.8.0-wp2697                                                                                                                                                                

Complete!
[root@rhel7 vagrant]# cat /usr/share/wazuh-indexer/VERSION 
4.8.0

[root@rhel7 vagrant]# bash /usr/share/wazuh-indexer/bin/indexer-init.sh -i 0.0.0.0 -p wiarUF8dnrUCiY*yHVIk*J+Z4bDIJdcH
Executing Wazuh indexer security init script...
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
Security Admin v7
Will connect to 127.0.0.1:9200 ... done
Connected as "CN=admin,OU=Wazuh,O=Wazuh,L=California,C=US"
OpenSearch Version: 2.10.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: wazuh-cluster
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index already exists, so we do not need to create one.
Populate config from /etc/wazuh-indexer/opensearch-security/
Will update '/config' with /etc/wazuh-indexer/opensearch-security/config.yml 
   SUCC: Configuration for 'config' created or updated
Will update '/roles' with /etc/wazuh-indexer/opensearch-security/roles.yml 
   SUCC: Configuration for 'roles' created or updated
Will update '/rolesmapping' with /etc/wazuh-indexer/opensearch-security/roles_mapping.yml 
   SUCC: Configuration for 'rolesmapping' created or updated
Will update '/internalusers' with /etc/wazuh-indexer/opensearch-security/internal_users.yml 
   SUCC: Configuration for 'internalusers' created or updated
Will update '/actiongroups' with /etc/wazuh-indexer/opensearch-security/action_groups.yml 
   SUCC: Configuration for 'actiongroups' created or updated
Will update '/tenants' with /etc/wazuh-indexer/opensearch-security/tenants.yml 
   SUCC: Configuration for 'tenants' created or updated
Will update '/nodesdn' with /etc/wazuh-indexer/opensearch-security/nodes_dn.yml 
   SUCC: Configuration for 'nodesdn' created or updated
Will update '/whitelist' with /etc/wazuh-indexer/opensearch-security/whitelist.yml 
   SUCC: Configuration for 'whitelist' created or updated
Will update '/audit' with /etc/wazuh-indexer/opensearch-security/audit.yml 
   SUCC: Configuration for 'audit' created or updated
Will update '/allowlist' with /etc/wazuh-indexer/opensearch-security/allowlist.yml 
   SUCC: Configuration for 'allowlist' created or updated
SUCC: Expected 10 config types for node {"updated_config_types":["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"],"updated_config_size":10,"message":null} is 10 (["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"]) due to: null
Done with success
Executing Wazuh indexer ISM init script...
Will create 'wazuh' index template
 SUCC: 'wazuh' template created or updated
Will create index templates to configure the alias
 SUCC: 'wazuh-alerts' template created or updated
 SUCC: 'wazuh-archives' template created or updated
Will create the 'rollover_policy' policy
  SUCC: 'rollover_policy' policy created
Will create initial indices for the aliases
  SUCC: 'wazuh-alerts' write index created
  SUCC: 'wazuh-archives' write index created
SUCC: Indexer ISM initialization finished successfully.

AlexRuiz7 · 2023-12-21T12:59:28Z

🟢 The initial indices are created properly, using all the settings from the templates.

However, there are 2 problems:

🔴 Missing step in the upgrade guide to refresh Filebeat's ingest pipelines.
- Update upgrade guide wazuh-documentation#6879
🟠 The rollover policy is applied to wazuh-alerts-4.x-sample indices, but shouldn't.

Although the second one doesn't present any major issue, the first one does, as Filebeat continues to send events to the old indices (wazuh-alerts-4.x-2023.12.21), instead of the new ones (wazuh-alerts-4.x-2023.12.21-000001, aliased to wazuh-alerts). The ingest pipelines must be refreshed with filebeat setup pipelines. This step is missing in the upgrade guide.

AlexRuiz7 · 2023-12-21T13:47:15Z

Checking the DEB package

Having successfully completed an upgrade using the RPM package, there is no point on repeating the process for the deb package, as the process is the same. Instead, we'll check that the template is at the expected path /etc/wazuh-indexer/wazuh-template.json, which it is.

To extract the package, use:

ar x wazuh-indexer_4.8.0-wp2697_amd64.deb
tar xvf data.tar.xz

./etc/
./etc/wazuh-indexer/
./etc/wazuh-indexer/opensearch-reports-scheduler/
./etc/wazuh-indexer/opensearch-reports-scheduler/reports-scheduler.yml
./etc/wazuh-indexer/wazuh-template.json
./etc/wazuh-indexer/log4j2.properties
./etc/wazuh-indexer/opensearch-notifications/
[...]

Test result: 🟢

AlexRuiz7 · 2023-12-21T16:19:00Z

Testing completed. As a result, new issues have been opened.

AlexRuiz7 added level/task Task issue request/operational Operational requests type/test Test issue labels Dec 21, 2023

AlexRuiz7 self-assigned this Dec 21, 2023

wazuhci added this to Release 4.8.0 Dec 21, 2023

wazuhci moved this to In progress in Release 4.8.0 Dec 21, 2023

AlexRuiz7 mentioned this issue Dec 21, 2023

Update upgrade guide wazuh/wazuh-documentation#6879

Merged

7 tasks

AlexRuiz7 linked a pull request Dec 21, 2023 that will close this issue

Update upgrade guide wazuh/wazuh-documentation#6879

Merged

7 tasks

wazuhci moved this from In progress to Pending final review in Release 4.8.0 Dec 21, 2023

rauldpm mentioned this issue Dec 21, 2023

Added wazuh-filebeat template to Wazuh indexer wazuh/wazuh-packages#2712

Merged

11 tasks

AlexRuiz7 closed this as completed Dec 21, 2023

wazuhci moved this from Pending final review to Done in Release 4.8.0 Dec 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test ISM rollover + alias initialization update #83

Test ISM rollover + alias initialization update #83

AlexRuiz7 commented Dec 21, 2023 •

edited

Loading

AlexRuiz7 commented Dec 21, 2023 •

edited

Loading

AlexRuiz7 commented Dec 21, 2023

AlexRuiz7 commented Dec 21, 2023 •

edited

Loading

AlexRuiz7 commented Dec 21, 2023 •

edited

Loading

AlexRuiz7 commented Dec 21, 2023

Test ISM rollover + alias initialization update #83

Test ISM rollover + alias initialization update #83

Comments

AlexRuiz7 commented Dec 21, 2023 • edited Loading

Description

Issues

AlexRuiz7 commented Dec 21, 2023 • edited Loading

Upgrade

AlexRuiz7 commented Dec 21, 2023

Upgrade

Procedure (commands)

wazuh-indexer

wazuh-server

wazuh-dashboard

Logs

AlexRuiz7 commented Dec 21, 2023 • edited Loading

AlexRuiz7 commented Dec 21, 2023 • edited Loading

Checking the DEB package

AlexRuiz7 commented Dec 21, 2023

AlexRuiz7 commented Dec 21, 2023 •

edited

Loading

AlexRuiz7 commented Dec 21, 2023 •

edited

Loading

`wazuh-indexer`

`wazuh-server`

`wazuh-dashboard`

AlexRuiz7 commented Dec 21, 2023 •

edited

Loading

AlexRuiz7 commented Dec 21, 2023 •

edited

Loading