Add bash scripts for MVP validation tests #482
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
15 tasks
AlexRuiz7
approved these changes
Oct 22, 2024
mcasas993
reviewed
Oct 25, 2024
AlexRuiz7
requested changes
Oct 28, 2024
There was a problem hiding this comment.
The Vagrantfile fails to start due to missing certificates archive.
indexer_2: Running: inline script
indexer_2: Synchronizing state of ufw.service with SysV service script with /lib/systemd/systemd-sysv-install.
indexer_2: Executing: /lib/systemd/systemd-sysv-install disable ufw
indexer_2: Removed /etc/systemd/system/multi-user.target.wants/ufw.service.
indexer_2: cp: cannot stat '/vagrant/wazuh-certificates.tar': No such file or directory
==> indexer_2: An error occurred. The error will be shown after all tasks complete.
An error occurred while executing multiple actions in parallel.
Any errors that occurred are shown below.
An error occurred while executing the action on the 'indexer_1'
machine. Please handle this error then try again:
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.
An error occurred while executing the action on the 'indexer_2'
machine. Please handle this error then try again:
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.Aside from that, use ShellCheck to lint the scripts, as it's reporting some flaws / warnings.
QU3B1M
force-pushed
the
enhancement/478-validation-tests
branch
from
47c03c8 to
5a7f8ad
Compare
Copy the scripts to the VMs auto.
…artifacts Update the tests scripts README
|
The last commit adds a script to automate the testing process. The script prompts for the GitHub Token, the workflow run ID and the package name for its download. The package is only downloaded if needed, reusing it if it's already there. Execution reusing the package
[vagrant@node-1 scripts]$ sudo bash 00_run.sh
Enter GitHub Token:
Enter Artifact ID: 11523519950
Enter Artifact Name: wazuh-indexer-5.0.0-0.x86_64.rpm
Enter Node 2 (optional):
Enter IP of Node 2 (optional):
2024-10-31 10:44:10 - Starting the script execution
2024-10-31 10:44:10 - Executing: bash 01_download_and_install_package.sh -id 11523519950 -n wazuh-indexer-5.0.0-0.x86_64.rpm
Package wazuh-indexer-5.0.0-0.x86_64.rpm found locally. Reusing existing package.
Installing wazuh-indexer package...
### NOT starting on installation, please execute the following statements to configure wazuh-indexer service to start automatically using systemd
sudo systemctl daemon-reload
sudo systemctl enable wazuh-indexer.service
### You can start wazuh-indexer service by executing
sudo systemctl start wazuh-indexer.service
Couldn't write '64' to 'kernel/random/read_wakeup_threshold', ignoring: No such file or directory
Package installed successfully.
2024-10-31 10:44:17 - Successfully executed: bash 01_download_and_install_package.sh -id 11523519950 -n wazuh-indexer-5.0.0-0.x86_64.rpm
2024-10-31 10:44:17 - Executing: sudo bash 02_apply_certificates.sh -p /home/vagrant/wazuh-certificates.tar -n node-1 -nip 192.168.56.10
Creating a backup of the original config file...
Updating configuration...
Configuration updated successfully. Backup created at ./opensearch.yml.bak
Creating certificates directory and extracting certificates...
Moving and setting permissions for certificates...
Certificates configured successfully.
2024-10-31 10:44:17 - Successfully executed: sudo bash 02_apply_certificates.sh -p /home/vagrant/wazuh-certificates.tar -n node-1 -nip 192.168.56.10
2024-10-31 10:44:17 - Executing: sudo bash 03_manage_indexer_service.sh -a start
Starting wazuh-indexer service...
Created symlink /etc/systemd/system/multi-user.target.wants/wazuh-indexer.service → /usr/lib/systemd/system/wazuh-indexer.service.
wazuh-indexer service is running.
2024-10-31 10:44:25 - Successfully executed: sudo bash 03_manage_indexer_service.sh -a start
2024-10-31 10:44:25 - Executing: sudo bash 04_initialize_cluster.sh
Initializing wazuh-indexer cluster...
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755 **
**************************************************************************
Security Admin v7
Will connect to 127.0.0.1:9200 ... done
Connected as "CN=admin,OU=Wazuh,O=Wazuh,L=California,C=US"
OpenSearch Version: 2.16.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: wazuh-cluster
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index already exists, so we do not need to create one.
Populate config from /etc/wazuh-indexer/opensearch-security/
Will update '/config' with /etc/wazuh-indexer/opensearch-security/config.yml
SUCC: Configuration for 'config' created or updated
Will update '/roles' with /etc/wazuh-indexer/opensearch-security/roles.yml
SUCC: Configuration for 'roles' created or updated
Will update '/rolesmapping' with /etc/wazuh-indexer/opensearch-security/roles_mapping.yml
SUCC: Configuration for 'rolesmapping' created or updated
Will update '/internalusers' with /etc/wazuh-indexer/opensearch-security/internal_users.yml
SUCC: Configuration for 'internalusers' created or updated
Will update '/actiongroups' with /etc/wazuh-indexer/opensearch-security/action_groups.yml
SUCC: Configuration for 'actiongroups' created or updated
Will update '/tenants' with /etc/wazuh-indexer/opensearch-security/tenants.yml
SUCC: Configuration for 'tenants' created or updated
Will update '/nodesdn' with /etc/wazuh-indexer/opensearch-security/nodes_dn.yml
SUCC: Configuration for 'nodesdn' created or updated
Will update '/whitelist' with /etc/wazuh-indexer/opensearch-security/whitelist.yml
SUCC: Configuration for 'whitelist' created or updated
Will update '/audit' with /etc/wazuh-indexer/opensearch-security/audit.yml
SUCC: Configuration for 'audit' created or updated
Will update '/allowlist' with /etc/wazuh-indexer/opensearch-security/allowlist.yml
SUCC: Configuration for 'allowlist' created or updated
SUCC: Expected 10 config types for node {"updated_config_types":["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","actiongroups","config","internalusers"],"updated_config_size":10,"message":null} is 10 (["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","actiongroups","config","internalusers"]) due to: null
Done with success
Checking cluster status...
Indexer Status:
Node Name: node-1
Cluster Name: wazuh-cluster
Version Number: 7.10.2
Verifying the Wazuh indexer nodes...
Nodes:
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role node.roles cluster_manager name
192.168.56.10 48 94 43 0.27 0.13 0.05 dimr cluster_manager,data,ingest,remote_cluster_client * node-1
Initialization completed successfully.
2024-10-31 10:44:30 - Successfully executed: sudo bash 04_initialize_cluster.sh
2024-10-31 10:44:30 - Executing: bash 05_validate_installed_plugins.sh -n node-1
Checking installed plugins on Wazuh indexer nodes...
Checking node node-1...
wazuh-indexer-command-manager is installed on node-1.
wazuh-indexer-setup is installed on node-1.
All required plugins are installed on all nodes.
2024-10-31 10:44:30 - Successfully executed: bash 05_validate_installed_plugins.sh -n node-1
2024-10-31 10:44:30 - Executing: bash 06_validate_setup.sh
Fetching templates from Wazuh indexer cluster...
Validating templates...
Template index-template-agent is created correctly.
Template index-template-alerts is created correctly.
Template index-template-fim is created correctly.
Template index-template-packages is created correctly.
Template index-template-processes is created correctly.
Template index-template-system is created correctly.
Template index-template-vulnerabilities is created correctly.
All templates are correctly created.
Fetching indices from Wazuh indexer cluster...
Fetching protected indices from Wazuh indexer cluster...
Validating index patterns...
Index pattern wazuh-alerts-5.x-* is valid.
Index pattern wazuh-states-vulnerabilities* is valid.
Index pattern .commands* is valid.
Index pattern wazuh-states-inventory-system* is valid.
Index pattern wazuh-states-inventory-packages* is valid.
Index pattern .agents* is valid.
Index pattern wazuh-states-fim* is valid.
Index pattern wazuh-states-inventory-processes* is valid.
Index-patterns validated successfully.
2024-10-31 10:44:31 - Successfully executed: bash 06_validate_setup.sh
2024-10-31 10:44:31 - Executing: bash 07_validate_command_manager.sh
{"_index":".commands","_id":"j10t4pIBVw9ERPuTR_vn","result":"CREATED"}Command created successfully.
Validating .commands index is created...
Index created correctly.
Validate the command is created
Validation successful: The command was created and found in the search results.
2024-10-31 10:44:32 - Successfully executed: bash 07_validate_command_manager.sh
2024-10-31 10:44:32 - Running 08_uninstall_indexer.sh
2024-10-31 10:44:32 - Executing: sudo bash 08_uninstall_indexer.sh
Uninstalling Wazuh Indexer...
Validating Wazuh Indexer removal...
Wazuh Indexer packages removed.
Wazuh Indexer service removed.
Wazuh Indexer uninstallation and validation completed successfully.
2024-10-31 10:44:33 - Successfully executed: sudo bash 08_uninstall_indexer.sh
2024-10-31 10:44:33 - All tasks completed successfully.
[vagrant@node-1 scripts]$ Execution downloading the package
vagrant@node-2:~/scripts$ sudo bash 00_run.sh
Enter GitHub Token:
Enter Artifact ID: 11523519950
Enter Artifact Name: wazuh-indexer_5.0.0-0_amd64.deb
Enter Node 2 (optional):
Enter IP of Node 2 (optional):
2024-10-31 10:46:28 - Starting the script execution
2024-10-31 10:46:28 - Executing: bash 01_download_and_install_package.sh -id 11523519950 -n wazuh-indexer_5.0.0-0_amd64.deb
Fetching artifacts list...
Checking wazuh-indexer_5.0.0-0_amd64.deb package is generated for workflow run 11523519950
Wazuh indexer artifact detected. Artifact ID: 2105657424
Downloading wazuh-indexer package from GitHub artifactory...
(It could take a couple of minutes)
Package downloaded successfully
Decompressing wazuh-indexer package...
Archive: ./package.zip
inflating: wazuh-indexer_5.0.0-0_amd64.deb
Package decompressed
Installing wazuh-indexer package...
Selecting previously unselected package wazuh-indexer.
(Reading database ... 76250 files and directories currently installed.)
Preparing to unpack wazuh-indexer_5.0.0-0_amd64.deb ...
Running Wazuh Indexer Pre-Installation Script
Unpacking wazuh-indexer (5.0.0-0) ...
Setting up wazuh-indexer (5.0.0-0) ...
Running Wazuh Indexer Post-Installation Script
### NOT starting on installation, please execute the following statements to configure wazuh-indexer service to start automatically using systemd
sudo systemctl daemon-reload
sudo systemctl enable wazuh-indexer.service
### You can start wazuh-indexer service by executing
sudo systemctl start wazuh-indexer.service
Package installed successfully.
2024-10-31 10:54:55 - Successfully executed: bash 01_download_and_install_package.sh -id 11523519950 -n wazuh-indexer_5.0.0-0_amd64.deb
2024-10-31 10:54:55 - Executing: sudo bash 02_apply_certificates.sh -p /home/vagrant/wazuh-certificates.tar -n node-1 -nip 192.168.56.10
Creating a backup of the original config file...
Updating configuration...
Configuration updated successfully. Backup created at ./opensearch.yml.bak
Creating certificates directory and extracting certificates...
Moving and setting permissions for certificates...
Certificates configured successfully.
2024-10-31 10:54:55 - Successfully executed: sudo bash 02_apply_certificates.sh -p /home/vagrant/wazuh-certificates.tar -n node-1 -nip 192.168.56.10
2024-10-31 10:54:55 - Executing: sudo bash 03_manage_indexer_service.sh -a start
Starting wazuh-indexer service...
Synchronizing state of wazuh-indexer.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable wazuh-indexer
Created symlink /etc/systemd/system/multi-user.target.wants/wazuh-indexer.service -> /lib/systemd/system/wazuh-indexer.service.
wazuh-indexer service is running.
2024-10-31 10:55:06 - Successfully executed: sudo bash 03_manage_indexer_service.sh -a start
2024-10-31 10:55:06 - Executing: sudo bash 04_initialize_cluster.sh
Initializing wazuh-indexer cluster...
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755 **
**************************************************************************
Security Admin v7
Will connect to 127.0.0.1:9200 ... done
Connected as "CN=admin,OU=Wazuh,O=Wazuh,L=California,C=US"
OpenSearch Version: 2.16.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: wazuh-cluster
Clusterstate: YELLOW
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index does not exists, attempt to create it ... done (0-all replicas)
Populate config from /etc/wazuh-indexer/opensearch-security/
Will update '/config' with /etc/wazuh-indexer/opensearch-security/config.yml
SUCC: Configuration for 'config' created or updated
Will update '/roles' with /etc/wazuh-indexer/opensearch-security/roles.yml
SUCC: Configuration for 'roles' created or updated
Will update '/rolesmapping' with /etc/wazuh-indexer/opensearch-security/roles_mapping.yml
SUCC: Configuration for 'rolesmapping' created or updated
Will update '/internalusers' with /etc/wazuh-indexer/opensearch-security/internal_users.yml
SUCC: Configuration for 'internalusers' created or updated
Will update '/actiongroups' with /etc/wazuh-indexer/opensearch-security/action_groups.yml
SUCC: Configuration for 'actiongroups' created or updated
Will update '/tenants' with /etc/wazuh-indexer/opensearch-security/tenants.yml
SUCC: Configuration for 'tenants' created or updated
Will update '/nodesdn' with /etc/wazuh-indexer/opensearch-security/nodes_dn.yml
SUCC: Configuration for 'nodesdn' created or updated
Will update '/whitelist' with /etc/wazuh-indexer/opensearch-security/whitelist.yml
SUCC: Configuration for 'whitelist' created or updated
Will update '/audit' with /etc/wazuh-indexer/opensearch-security/audit.yml
SUCC: Configuration for 'audit' created or updated
Will update '/allowlist' with /etc/wazuh-indexer/opensearch-security/allowlist.yml
SUCC: Configuration for 'allowlist' created or updated
SUCC: Expected 10 config types for node {"updated_config_types":["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","actiongroups","config","internalusers"],"updated_config_size":10,"message":null} is 10 (["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","actiongroups","config","internalusers"]) due to: null
Done with success
Checking cluster status...
parse error: Invalid numeric literal at line 1, column 11
parse error: Invalid numeric literal at line 1, column 11
parse error: Invalid numeric literal at line 1, column 11
Indexer Status:
Node Name:
Cluster Name:
Version Number:
Verifying the Wazuh indexer nodes...
Nodes:
OpenSearch Security not initialized.
Initialization completed successfully.
2024-10-31 10:55:15 - Successfully executed: sudo bash 04_initialize_cluster.sh
2024-10-31 10:55:15 - Executing: bash 05_validate_installed_plugins.sh -n node-1
Checking installed plugins on Wazuh indexer nodes...
Checking node node-1...
Error: Failed to connect to Wazuh indexer.
2024-10-31 10:55:15 - Error executing: bash 05_validate_installed_plugins.sh -n node-1vagrant@node-2:~/scripts$ curl -k -u admin:admin https://node-1:9200
curl: (7) Failed to connect to node-1 port 9200 after 0 ms: Connection refused
vagrant@node-2:~/scripts$ curl -k -u admin:admin https://192.168.56.10:9200
curl: (7) Failed to connect to 192.168.56.10 port 9200 after 0 ms: Connection refused
vagrant@node-2:~/scripts$ curl -k -u admin:admin https://192.168.56.11:9200
{
"name" : "node-1",
"cluster_name" : "wazuh-cluster",
"cluster_uuid" : "_IgH4HX3RhyByxUC278XJg",
"version" : {
"number" : "7.10.2",
"build_type" : "deb",
"build_hash" : "3ab51a880debe26119c818e2b4e4db74ae9783fe",
"build_date" : "2024-10-25T18:35:17.158735352Z",
"build_snapshot" : false,
"lucene_version" : "9.11.1",
"minimum_wire_compatibility_version" : "7.10.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "The OpenSearch Project: https://opensearch.org/"
}
vagrant@node-2:~/scripts$ sudo head /etc/wazuh-indexer/opensearch.yml
network.host: "0.0.0.0"
node.name: "node-1"
cluster.initial_master_nodes:
- node-1
#- "node-2"
#- "node-3"
cluster.name: "wazuh-cluster"
discovery.seed_hosts:
- "192.168.56.10"
# - "node-1-ip" |
Fix script 00 to work on any node Remove unwanted outputs from executed commands
Update default IP detection
QU3B1M
force-pushed
the
enhancement/478-validation-tests
branch
from
e3bbeab to
1ceef3c
Compare
|
In the last update I've improved the all-at-once script to work on any node, and cleaned the output of the scripts. The scripts README is also updated adding a guide to run the validations on single node and multi node clusters scenarios. Execution on
|
AlexRuiz7
approved these changes
Nov 5, 2024
AlexRuiz7
added a commit
that referenced
this pull request
Nov 8, 2024
* Add MVP validation tests bash scripts * Add validations for generated index-patterns * Update scripts to support debian ARM * Update validations scripts to be able to use the generated package name * Add argument to define certificates path * Update OS detection on scripts * Add dependencies validations * Add usage description to each script and a simple README * Add dependencies validations * Fix typos * Apply SpellCheck linter recommendations * Skip checks related to SC2181 where the fix is not applicable * Remove unnecesary double quotes from certificates generation script * Update variable quoting * Provision VMs with dependencies for the testing scripts Copy the scripts to the VMs auto. * Merge scripts 00 and 01 making it easier to get the package from GHA artifacts Update the tests scripts README * Optimize test scripts * Add sleep after clister initialization * Update README and improve scripts output logs Fix script 00 to work on any node Remove unwanted outputs from executed commands * Update execution guide on README * Add conditional to remove certs directory if already exists Update default IP detection * Add sleep to avoid requesting to the API before cluster is initialized * Add index force merge for the command_manager plugin index * Avoid errors due to race conditions --------- Co-authored-by: Álex Ruiz <[email protected]>
AlexRuiz7
added a commit
that referenced
this pull request
Nov 18, 2024
* Add MVP validation tests bash scripts * Add validations for generated index-patterns * Update scripts to support debian ARM * Update validations scripts to be able to use the generated package name * Add argument to define certificates path * Update OS detection on scripts * Add dependencies validations * Add usage description to each script and a simple README * Add dependencies validations * Fix typos * Apply SpellCheck linter recommendations * Skip checks related to SC2181 where the fix is not applicable * Remove unnecesary double quotes from certificates generation script * Update variable quoting * Provision VMs with dependencies for the testing scripts Copy the scripts to the VMs auto. * Merge scripts 00 and 01 making it easier to get the package from GHA artifacts Update the tests scripts README * Optimize test scripts * Add sleep after clister initialization * Update README and improve scripts output logs Fix script 00 to work on any node Remove unwanted outputs from executed commands * Update execution guide on README * Add conditional to remove certs directory if already exists Update default IP detection * Add sleep to avoid requesting to the API before cluster is initialized * Add index force merge for the command_manager plugin index * Avoid errors due to race conditions --------- Co-authored-by: Álex Ruiz <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR implements a set of scripts used for the MVP validation tests.
Related Issues
Resolves #478
Check List