Release 4.6.0 - Beta 1 - E2E UX tests - Demo environment

[havidarou]

End-to-End (E2E) Testing Guideline

• Documentation: Always consult the development documentation for the current stage tag at this link. Be careful because some of the description steps might refer to a current version in production, always navigate using the current development documention for the stage under test.
• Test Requirements: Ensure your test comprehensively includes a full stack and agent/s deployment as per the Deployment requirements, detailing the machine OS, installed version, and revision.
• Deployment Options: While deployments can be local (using VMs, Vagrant, etc) or on the aws-dev account, opt for local deployments when feasible. For AWS access, coordinate with the CICD team through this link.
• External Accounts: If tests require third-party accounts (e.g., GitHub, Azure, AWS, GCP), request the necessary access through the CICD team here.
• Alerts: Every test should generate a minimum of one end-to-end alert, from the agent to the dashboard, irrespective of test type.
• Multi-node Testing: For multi-node wazuh-manager tests, ensure agents are connected to both workers and the master node.
• Package Verification: Use the pre-release package that matches the current TAG you're testing. Confirm its version and revision.
• Filebeat Errors: If you encounter errors with Filebeat during testing, refer to this Slack discussion for insights and resolutions.
• Known Issues: Familiarize yourself with previously reported issues in the Known Issues section. This helps in identifying already recognized errors during testing.
• Reporting New Issues: Any new errors discovered during testing that aren't listed under Known Issues should be reported. Assign the issue to the corresponding team (QA if unsure), add the Release testing/publication objective and Very high priority. Communicate these to the team and QA via the c-release Slack channel.
• Test Conduct: It's imperative to be thorough in your testing, offering enough detail for reviewers. Incomplete tests might necessitate a redo.
• Documentation Feedback: Encountering documentation gaps, unclear guidelines, or anything that disrupts the testing or UX? Open an issue, especially if it's not listed under Known Issues.
• Format: If this is your first time doing this, refer to the format (but not necessarily the content, as it may vary) of previous E2E tests, here you have an example Release 4.3.5 - Release Candidate 1 - E2E UX tests - Wazuh Indexer #13994.
• Status and completion: Change the issue status within your team project accordingly. Once you finish testing and write the conclusions, move it to Pending review and notify the @wazuh/cicd team via Slack using the c-release channel. Beware that the reviewers might request additional information or task repetitions.
• For reviewers: Please move the issue to Pending final review and notify via Slack using the same thread if everything is ok, otherwise, perform an issue update with the requested changes and move it to On hold, increase the review_cycles in the team project by one and notify the issue assignee via Slack using the same thread.

For the conclusions and the issue testing and updates, use the following legend:

Status legend

• 🟢 All checks passed
• 🟡 Found a known issue
• 🔴 Found a new error

Deployment requirements

Component	Installation	Type	OS
Indexer
Server
Dashboard		-
Agent		-

Test description

Test demo.wazuh.info environment:

• Check that there are no errors in the manager, agent, cluster, indexer, and dashboard logs.
• Check that wazuh daemons are running with the expected user.
• Check that the status of the indexer cluster is the expected.
• Check that there are no errors in the browser's developer console when browsing the App.
• Check that no warning symbols appear in the browser's developer console when browsing the App
• Check that there are alerts for each of the modules configured.
• Generate an alert and check that this alert appears in the dashboard (end to end)
• Check that the search engine works without specifying a field and using *

To access the demo environment, please contact @cicd-team.

Known issues

• Warnings and errors when logging out of the app wazuh-dashboard-plugins#4108
• Multiple unexpected warning during Wazuh app session wazuh-dashboard-plugins#4092
• Warnings messages when checking Wazuh-Indexer status wazuh-packages#1749
• Logcollector attempts messages when a file doesn't exist should be info until the last attempt #13253
• [Development] Windows 2012 R2 Wazuh agent can't subscribe to Windows Defender events #5214
• Reconnect time does not work when system time changed #8580
• Wazuh does not reconnect to Windows event log if the service is down at the start #8581
• ChunkLoadError, Loading chunk 4 failed. wazuh-dashboard-plugins#4406
• highlight.js is EOL wazuh-dashboard-plugins#4407
• Agent information cannot be parsed #14710
• Map rendering Uncaught TypeError: bounds is null wazuh-dashboard-plugins#4232
• Disabled plugins shouldn't appear in More menu wazuh-dashboard-plugins#4074
• Console error when I have an agent selected and environment change wazuh-dashboard-plugins#5745
• OpenSearch modifies log files permissions wazuh-packages#2139
• SUSE OVAL feeds download are too slow #18593
• Multiple unexpected Errors during Wazuh app session wazuh-dashboard-plugins#5821
• https://github.com/wazuh/wazuh-automation/issues/1113
• Does not show events in the 'Docker Listener', 'VirusTotal' and 'System Auditing' Modules when they are enabled wazuh-dashboard-plugins#5749
• The search with (*) does not return all the expected results. wazuh-dashboard-plugins#5750

Conclusions

Summarize the errors detected (Known Issues included). Illustrate using the table below, removing current examples:

Status	Test	Failure type	Notes
🔴	Check that there are no errors in the manager, agent, cluster, indexer, and dashboard logs.	Found issues in the Dashboard and on managers.	Known Issue Fix to truncate overflow attempt when parsing XML feed #19217 Known issue Found deleteOldIndices errors in indexer  wazuh-packages#2094 Known issue Reduce log level for informative message opensearch-project/security-analytics#203 New Issue Error ocurred in Delayed API Request on Demo Enviroment wazuh-dashboard-plugins#5958
🟢	The daemons are running with the correct user
🟢	The status of the Wazuh Indexer clusters is as expected
🟡	Check that there are no errors in the browser's developer console when browsing the App.	Found multiple known Issues	Comment about issues found: Multiple unexpected Errors during Wazuh app session wazuh-dashboard-plugins#5821 Console error when I have an agent selected and environment change wazuh-dashboard-plugins#5745 Multiple unexpected warning during Wazuh app session wazuh-dashboard-plugins#4092 Disabled plugins shouldn't appear in More menu wazuh-dashboard-plugins#4074
🟢	Check that no warning symbols appear in the browser's developer console when browsing the App
🟡	Alerts are being generated for each of the modules configured for this purpose	Some modules do not generate alerts, but it is known to happen from previous versions.	Known Issue found wazuh/wazuh-dashboard-plugins#5749
🟢	Generate an alert and check that this alert appears in the dashboard (end to end)
🟢	Search works without specifying a field and using *

Feedback

We value your feedback. Please provide insights on your testing experience.

• Was the testing guideline clear? Were there any ambiguities?
• Did you face any challenges not covered by the guideline?
• Suggestions for improvement:

Reviewers validation

The criteria for completing this task is based on the validation of the conclusions and the test results by all reviewers.

All the checkboxes below must be marked in order to close this issue.

• @havidarou
• @wazuh/cicd

[Deblintrake09]

T1 - No errors or warnings found in logs

Agents

Amazon Linux 2 🟢

1. Check agent's version

# /var/ossec/bin/wazuh-control info
WAZUH_VERSION="v4.6.0"
WAZUH_REVISION="40601"
WAZUH_TYPE="agent"

2. Check the general status: systemctl status wazuh-agent -l 🟢

● wazuh-agent.service - Wazuh agent
   Loaded: loaded (/usr/lib/systemd/system/wazuh-agent.service; enabled; vendor preset: disabled)
   Active: active (running) since lun 2023-09-25 14:12:41 UTC; 2 days ago
  Process: 5790 ExecStop=/usr/bin/env /var/ossec/bin/wazuh-control stop (code=exited, status=0/SUCCESS)
  Process: 5926 ExecStart=/usr/bin/env /var/ossec/bin/wazuh-control start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/wazuh-agent.service
           ├─3831 /var/ossec/bin/wazuh-execd
           ├─3843 /var/ossec/bin/wazuh-agentd
           ├─3858 /var/ossec/bin/wazuh-syscheckd
           ├─3874 /var/ossec/bin/wazuh-logcollector
           └─3891 /var/ossec/bin/wazuh-modulesd

sep 25 14:12:37 ip-10-0-1-199.us-west-1.compute.internal env[5926]: Started wazuh-syscheckd...
sep 25 14:12:38 ip-10-0-1-199.us-west-1.compute.internal env[5926]: Started wazuh-logcollector...
sep 25 14:12:39 ip-10-0-1-199.us-west-1.compute.internal env[5926]: Started wazuh-modulesd...
sep 25 14:12:41 ip-10-0-1-199.us-west-1.compute.internal env[5926]: Completed.
sep 25 14:12:41 ip-10-0-1-199.us-west-1.compute.internal systemd[1]: Started Wazuh agent.
sep 25 14:32:02 ip-10-0-1-199.us-west-1.compute.internal crontab[4019]: (root) LIST (root)

3. Check the modules status: /var/ossec/bin/wazuh-control status 🟢

root@xxxx# /var/ossec/bin/wazuh-control status
wazuh-modulesd is running...
wazuh-logcollector is running...
wazuh-syscheckd is running...
wazuh-agentd is running...
wazuh-execd is running...

4. Check the service status: journalctl -xe -u wazuh-agent.service 🟢

-- Subject: Unit wazuh-agent.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit wazuh-agent.service has begun starting up.
sep 25 14:12:34 ip-xx.xx.xx.xx.xxxxx.compute.internal env[5926]: Starting Wazuh v4.6.0...
sep 25 14:12:35 ip-xx.xx.xx.xx.xxxxx.compute.internal env[5926]: Started wazuh-execd...
sep 25 14:12:36 ip-xx.xx.xx.xx.xxxxx.compute.internal env[5926]: Started wazuh-agentd...
sep 25 14:12:37 ip-xx.xx.xx.xx.xxxxx.compute.internal env[5926]: Started wazuh-syscheckd...
sep 25 14:12:38 ip-xx.xx.xx.xx.xxxxx.compute.internal env[5926]: Started wazuh-logcollector...
sep 25 14:12:39 ip-xx.xx.xx.xx.xxxxx.compute.internal env[5926]: Started wazuh-modulesd...
sep 25 14:12:41 ip-xx.xx.xx.xx.xxxxx.compute.internal env[5926]: Completed.
sep 25 14:12:41 ip-xx.xx.xx.xx.xxxxx.compute.internal systemd[1]: Started Wazuh agent.
-- Subject: Unit wazuh-agent.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit wazuh-agent.service has finished starting up.
-- 
-- The start-up result is done.
sep 25 14:32:02 ip-xx.xx.xx.xx.xxxxx.compute.internal crontab[4019]: (root) LIST (root)
sep 26 02:32:01 ip-xx.xx.xx.xx.xxxxx.compute.internal crontab[9420]: (root) LIST (root)
sep 26 14:32:01 ip-xx.xx.xx.xx.xxxxx.compute.internal crontab[14416]: (root) LIST (root)
sep 27 02:32:01 ip-xx.xx.xx.xx.xxxxx.compute.internal crontab[20501]: (root) LIST (root)
sep 27 14:32:02 ip-xx.xx.xx.xx.xxxxx.compute.internal crontab[25704]: (root) LIST (root)

5. Check for errors in logs: egrep -i "ERROR|WARNING" /var/ossec/logs/ossec.log 🟢

CentOS 🟢

1. Check agent's version

WAZUH_VERSION="v4.6.0"
WAZUH_REVISION="40601"
WAZUH_TYPE="agent"

2. Check the general status: systemctl status wazuh-agent -l 🟢

● wazuh-agent.service - Wazuh agent
   Loaded: loaded (/usr/lib/systemd/system/wazuh-agent.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2023-09-25 14:13:56 UTC; 2 days ago
  Process: 8015 ExecStop=/usr/bin/env /var/ossec/bin/wazuh-control stop (code=exited, status=0/SUCCESS)
  Process: 8110 ExecStart=/usr/bin/env /var/ossec/bin/wazuh-control start (code=exited, status=0/SUCCESS)
    Tasks: 32 (limit: 4668)
   Memory: 218.0M
   CGroup: /system.slice/wazuh-agent.service
           ├─9449 /var/ossec/bin/wazuh-execd
           ├─9461 /var/ossec/bin/wazuh-agentd
           ├─9476 /var/ossec/bin/wazuh-syscheckd
           ├─9492 /var/ossec/bin/wazuh-logcollector
           └─9510 /var/ossec/bin/wazuh-modulesd

3. Check the modules status: /var/ossec/bin/wazuh-control status 🟢

root@xxxx# /var/ossec/bin/wazuh-control status
wazuh-modulesd is running...
wazuh-logcollector is running...
wazuh-syscheckd is running...
wazuh-agentd is running...
wazuh-execd is running...

4. Check the service status: journalctl -xe -u wazuh-agent.service 🟢

-- Support: https://access.redhat.com/support
-- 
-- Unit wazuh-agent.service has finished shutting down.
sep 25 14:13:49 ip-xx.xx.xx.xx.xxxxx.compute.internal systemd[1]: Starting Wazuh agent...
-- Subject: Unit wazuh-agent.service has begun start-up
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
-- 
-- Unit wazuh-agent.service has begun starting up.
sep 25 14:13:49 ip-xx.xx.xx.xx.xxxxx.compute.internal env[8110]: Starting Wazuh v4.6.0...
sep 25 14:13:50 ip-xx.xx.xx.xx.xxxxx.compute.internal env[8110]: Started wazuh-execd...
sep 25 14:13:51 ip-xx.xx.xx.xx.xxxxx.compute.internal env[8110]: Started wazuh-agentd...
sep 25 14:13:52 ip-xx.xx.xx.xx.xxxxx.compute.internal env[8110]: Started wazuh-syscheckd...
sep 25 14:13:53 ip-xx.xx.xx.xx.xxxxx.compute.internal env[8110]: Started wazuh-logcollector...
sep 25 14:13:54 ip-xx.xx.xx.xx.xxxxx.compute.internal env[8110]: Started wazuh-modulesd...
sep 25 14:13:56 ip-xx.xx.xx.xx.xxxxx.compute.internal env[8110]: Completed.
sep 25 14:13:56 ip-xx.xx.xx.xx.xxxxx.compute.internal systemd[1]: Started Wazuh agent.
-- Subject: Unit wazuh-agent.service has finished start-up
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
-- 
-- Unit wazuh-agent.service has finished starting up.
-- 
-- The start-up result is done.
sep 25 14:14:03 ip-xx.xx.xx.xx.xxxxx.compute.internal crontab[8567]: (root) LIST (root)

5. Check for errors in logs: egrep -i "ERROR|WARNING" /var/ossec/logs/ossec.log 🟢

Debian 🟢

1. Check agent's version

WAZUH_VERSION="v4.6.0"
WAZUH_REVISION="40601"
WAZUH_TYPE="agent"

2. Check the general status: systemctl status wazuh-agent -l 🟢

● wazuh-agent.service - Wazuh agent
     Loaded: loaded (/lib/systemd/system/wazuh-agent.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2023-09-25 14:13:01 UTC; 2 days ago
      Tasks: 32 (limit: 1123)
     Memory: 187.6M
        CPU: 3min 23.121s
     CGroup: /system.slice/wazuh-agent.service
             ├─9067 /var/ossec/bin/wazuh-execd
             ├─9078 /var/ossec/bin/wazuh-agentd
             ├─9092 /var/ossec/bin/wazuh-syscheckd
             ├─9107 /var/ossec/bin/wazuh-logcollector
             └─9126 /var/ossec/bin/wazuh-modulesd

Sep 25 14:12:54 ip-xx.xx.xx.xx systemd[1]: Starting Wazuh agent...
Sep 25 14:12:54 ip-xx.xx.xx.xx env[7037]: Starting Wazuh v4.6.0...
Sep 25 14:12:55 ip-xx.xx.xx.xx env[7037]: Started wazuh-execd...
Sep 25 14:12:56 ip-xx.xx.xx.xx env[7037]: Started wazuh-agentd...
Sep 25 14:12:57 ip-xx.xx.xx.xx env[7037]: Started wazuh-syscheckd...
Sep 25 14:12:58 ip-xx.xx.xx.xx env[7037]: Started wazuh-logcollector...
Sep 25 14:12:59 ip-xx.xx.xx.xx env[7037]: Started wazuh-modulesd...
Sep 25 14:13:01 ip-xx.xx.xx.xx env[7037]: Completed.

3. Check the modules status: /var/ossec/bin/wazuh-control status 🟢

root@xxxx# /var/ossec/bin/wazuh-control status
wazuh-modulesd is running...
wazuh-logcollector is running...
wazuh-syscheckd is running...
wazuh-agentd is running...
wazuh-execd is running...

4. Check the service status: journalctl -xe -u wazuh-agent.service 🟢

░░ The unit wazuh-agent.service completed and consumed the indicated resources.
Sep 25 14:12:54 ip-xx.xx.xx.xx systemd[1]: Starting Wazuh agent...
░░ Subject: A start job for unit wazuh-agent.service has begun execution
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit wazuh-agent.service has begun execution.
░░ 
░░ The job identifier is 3250.
Sep 25 14:12:54 ip-xx.xx.xx.xx env[7037]: Starting Wazuh v4.6.0...
Sep 25 14:12:55 ip-xx.xx.xx.xx env[7037]: Started wazuh-execd...
Sep 25 14:12:56 ip-xx.xx.xx.xx env[7037]: Started wazuh-agentd...
Sep 25 14:12:57 ip-xx.xx.xx.xx env[7037]: Started wazuh-syscheckd...
Sep 25 14:12:58 ip-xx.xx.xx.xx env[7037]: Started wazuh-logcollector...
Sep 25 14:12:59 ip-xx.xx.xx.xx env[7037]: Started wazuh-modulesd...
Sep 25 14:13:01 ip-xx.xx.xx.xx env[7037]: Completed.
Sep 25 14:13:01 ip-xx.xx.xx.xx systemd[1]: Started Wazuh agent.
░░ Subject: A start job for unit wazuh-agent.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit wazuh-agent.service has finished successfully.
░░ 
░░ The job identifier is 3250.

5. Check for errors in logs: egrep -i "ERROR|WARNING" /var/ossec/logs/ossec.log 🟢

Ubuntu 🟢

1. Check agent's version

WAZUH_VERSION="v4.6.0"
WAZUH_REVISION="40601"
WAZUH_TYPE="agent"

2. Check the general status: systemctl status wazuh-agent -l 🟢

● wazuh-agent.service - Wazuh agent
     Loaded: loaded (/lib/systemd/system/wazuh-agent.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2023-09-25 14:14:06 UTC; 2 days ago
      Tasks: 32 (limit: 1116)
     Memory: 137.2M
        CPU: 4min 14.950s
     CGroup: /system.slice/wazuh-agent.service
             ├─9303 /var/ossec/bin/wazuh-execd
             ├─9314 /var/ossec/bin/wazuh-agentd
             ├─9328 /var/ossec/bin/wazuh-syscheckd
             ├─9343 /var/ossec/bin/wazuh-logcollector
             └─9360 /var/ossec/bin/wazuh-modulesd

Sep 25 14:13:59 ip-xx.xx.xx.xx systemd[1]: Starting Wazuh agent...
Sep 25 14:13:59 ip-xx.xx.xx.xx env[8250]: Starting Wazuh v4.6.0...
Sep 25 14:14:00 ip-xx.xx.xx.xx env[8250]: Started wazuh-execd...
Sep 25 14:14:01 ip-xx.xx.xx.xx env[8250]: Started wazuh-agentd...
Sep 25 14:14:02 ip-xx.xx.xx.xx env[8250]: Started wazuh-syscheckd...
Sep 25 14:14:03 ip-xx.xx.xx.xx env[8250]: Started wazuh-logcollector...
Sep 25 14:14:04 ip-xx.xx.xx.xx env[8250]: Started wazuh-modulesd...
Sep 25 14:14:06 ip-xx.xx.xx.xx env[8250]: Completed.
Sep 25 14:14:06 ip-xx.xx.xx.xx systemd[1]: Started Wazuh agent.

3. Check the modules status: /var/ossec/bin/wazuh-control status 🟢

root@xxxx# /var/ossec/bin/wazuh-control status
wazuh-modulesd is running...
wazuh-logcollector is running...
wazuh-syscheckd is running...
wazuh-agentd is running...
wazuh-execd is running...

4. Check the service status: journalctl -xe -u wazuh-agent.service 🟢

░░ The unit wazuh-agent.service completed and consumed the indicated resources.
Sep 25 14:13:59 ip-xx.xx.xx.xx systemd[1]: Starting Wazuh agent...
░░ Subject: A start job for unit wazuh-agent.service has begun execution
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ A start job for unit wazuh-agent.service has begun execution.
░░ 
░░ The job identifier is 6663.
Sep 25 14:13:59 ip-xx.xx.xx.xx env[8250]: Starting Wazuh v4.6.0...
Sep 25 14:14:00 ip-xx.xx.xx.xx env[8250]: Started wazuh-execd...
Sep 25 14:14:01 ip-xx.xx.xx.xx env[8250]: Started wazuh-agentd...
Sep 25 14:14:02 ip-xx.xx.xx.xx env[8250]: Started wazuh-syscheckd...
Sep 25 14:14:03 ip-xx.xx.xx.xx env[8250]: Started wazuh-logcollector...
Sep 25 14:14:04 ip-xx.xx.xx.xx env[8250]: Started wazuh-modulesd...
Sep 25 14:14:06 ip-xx.xx.xx.xx env[8250]: Completed.
Sep 25 14:14:06 ip-xx.xx.xx.xx systemd[1]: Started Wazuh agent.
░░ Subject: A start job for unit wazuh-agent.service has finished successfully
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ A start job for unit wazuh-agent.service has finished successfully.
░░ 
░░ The job identifier is 6663.

5. Check for errors in logs: egrep -i "ERROR|WARNING" /var/ossec/logs/ossec.log 🟢

RHEL 🟢

1. Check agent's version

# /var/ossec/bin/wazuh-control info
WAZUH_VERSION="v4.6.0"
WAZUH_REVISION="40601"
WAZUH_TYPE="agent"

2. Check the general status: systemctl status wazuh-agent -l 🟢

● wazuh-agent.service - Wazuh agent
     Loaded: loaded (/usr/lib/systemd/system/wazuh-agent.service; enabled; preset: disabled)
     Active: active (running) since Mon 2023-09-25 14:51:30 UTC; 2 days ago
    Process: 60152 ExecStart=/usr/bin/env /var/ossec/bin/wazuh-control start (code=exited, status=0/SUCCESS)
      Tasks: 54 (limit: 22632)
     Memory: 333.8M
        CPU: 21min 50.640s
     CGroup: /system.slice/wazuh-agent.service
             ├─60179 /var/ossec/bin/wazuh-execd
             ├─60191 /var/ossec/bin/wazuh-agentd
             ├─60206 /var/ossec/bin/wazuh-syscheckd
             ├─60226 /var/ossec/bin/wazuh-logcollector
             ├─60243 /var/ossec/bin/wazuh-modulesd
             ├─60254 python3 wodles/docker/DockerListener
             ├─60259 /usr/bin/osqueryd --config_path=/etc/osquery/osquery.conf
             └─60263 /usr/bin/osqueryd

Sep 25 14:51:23 ip-xx.xx.xx.xx.compute.internal systemd[1]: Starting Wazuh agent...
Sep 25 14:51:23 ip-xx.xx.xx.xx.compute.internal env[60152]: Starting Wazuh v4.6.0...

3. Check the modules status: /var/ossec/bin/wazuh-control status 🟢

root@xxxx# /var/ossec/bin/wazuh-control status
wazuh-modulesd is running...
wazuh-logcollector is running...
wazuh-syscheckd is running...
wazuh-agentd is running...
wazuh-execd is running...

4. Check the service status: journalctl -xe -u wazuh-agent.service 🟢

 Subject: A start job for unit wazuh-agent.service has begun execution
░░ Defined-By: systemd
░░ Support: https://access.redhat.com/support
░░ 
░░ A start job for unit wazuh-agent.service has begun execution.
░░ 
░░ The job identifier is 25295.
Sep 25 14:51:23 ip-xx.xx.xx.xx.compute.internal env[60152]: Starting Wazuh v4.6.0...
Sep 25 14:51:24 ip-xx.xx.xx.xx.compute.internal env[60152]: Started wazuh-execd...
Sep 25 14:51:25 ip-xx.xx.xx.xx.compute.internal env[60152]: Started wazuh-agentd...
Sep 25 14:51:26 ip-xx.xx.xx.xx.compute.internal env[60152]: Started wazuh-syscheckd...
Sep 25 14:51:27 ip-xx.xx.xx.xx.compute.internal env[60152]: Started wazuh-logcollector...
Sep 25 14:51:27 ip-xx.xx.xx.xx.compute.internal osqueryd[60259]: osqueryd started [version=4.4.0]
Sep 25 14:51:28 ip-xx.xx.xx.xx.compute.internal env[60152]: Started wazuh-modulesd...
Sep 25 14:51:30 ip-xx.xx.xx.xx.compute.internal env[60152]: Completed.
Sep 25 14:51:30 ip-xx.xx.xx.xx.compute.internal systemd[1]: Started Wazuh agent.
░░ Subject: A start job for unit wazuh-agent.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://access.redhat.com/support
░░ 
░░ A start job for unit wazuh-agent.service has finished successfully.
░░ 
░░ The job identifier is 25295.

5. Check for errors in logs: egrep -i "ERROR|WARNING" /var/ossec/logs/ossec.log 🟢

Windows 🟢

1. Check agent's version and status
   

2. Check event in EventViewer: 🟢
   

[-](https://github.com/wazuh/wazuh/issues/19157#) <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
[-](https://github.com/wazuh/wazuh/issues/19157#) <System>
  <Provider Name="Service Control Manager" Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}" EventSourceName="Service Control Manager" /> 
  <EventID Qualifiers="16384">7036</EventID> 
  <Version>0</Version> 
  <Level>4</Level> 
  <Task>0</Task> 
  <Opcode>0</Opcode> 
  <Keywords>0x8080000000000000</Keywords> 
  <TimeCreated SystemTime="2023-09-25T14:40:37.161807300Z" /> 
  <EventRecordID>109146</EventRecordID> 
  <Correlation /> 
  <Execution ProcessID="596" ThreadID="4020" /> 
  <Channel>System</Channel> 
  <Computer>EC2AMAZ-PR2CC4P</Computer> 
  <Security /> 
  </System>
[-](https://github.com/wazuh/wazuh/issues/19157#) <EventData>
  <Data Name="param1">Wazuh</Data> 
  <Data Name="param2">running</Data> 
  <Binary>570061007A00750068005300760063002F0034000000</Binary> 
  </EventData>
  </Event>

Managers

Master env 1 🟡

1. Check the general status: systemctl status wazuh-manager -l:🟢

● wazuh-manager.service - Wazuh manager
   Loaded: loaded (/usr/lib/systemd/system/wazuh-manager.service; enabled; vendor preset: disabled)
   Active: active (exited) since Mon 2023-09-25 14:05:09 UTC; 2 days ago
  Process: 10952 ExecStop=/usr/bin/env /var/ossec/bin/wazuh-control stop (code=exited, status=0/SUCCESS)
  Process: 11107 ExecStart=/usr/bin/env /var/ossec/bin/wazuh-control start (code=exited, status=0/SUCCESS)

Sep 25 14:05:00 wazuh-manager-master-0 env[11107]: Started wazuh-analysisd...
Sep 25 14:05:01 wazuh-manager-master-0 env[11107]: Started wazuh-syscheckd...
Sep 25 14:05:02 wazuh-manager-master-0 env[11107]: Started wazuh-remoted...
Sep 25 14:05:03 wazuh-manager-master-0 env[11107]: Started wazuh-logcollector...
Sep 25 14:05:04 wazuh-manager-master-0 env[11107]: Started wazuh-monitord...
Sep 25 14:05:05 wazuh-manager-master-0 env[11107]: Started wazuh-modulesd...
Sep 25 14:05:06 wazuh-manager-master-0 crontab[11523]: (root) LIST (root)
Sep 25 14:05:07 wazuh-manager-master-0 env[11107]: Started wazuh-clusterd...
Sep 25 14:05:09 wazuh-manager-master-0 env[11107]: Completed.
Sep 25 14:05:09 wazuh-manager-master-0 systemd[1]: Started Wazuh manager.

2. Check the modules status: /var/ossec/bin/wazuh-control status: 🟢

wazuh-clusterd is running...
wazuh-modulesd is running...
wazuh-monitord is running...
wazuh-logcollector is running...
wazuh-remoted is running...
wazuh-syscheckd is running...
wazuh-analysisd is running...
wazuh-maild not running...
wazuh-execd is running...
wazuh-db is running...
wazuh-authd is running...
wazuh-agentlessd not running...
wazuh-integratord is running...
wazuh-dbd not running...
wazuh-csyslogd not running...

3. Check the service status: journalctl -xe -u wazuh-manager.service: 🟢

-- Logs begin at Mon 2023-09-25 13:40:40 UTC, end at Wed 2023-09-27 20:51:21 UTC. --
Sep 25 14:03:08 wazuh-manager-master-0 systemd[1]: Starting Wazuh manager...
-- Subject: Unit wazuh-manager.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-manager.service has begun starting up.
Sep 25 14:03:10 wazuh-manager-master-0 env[7314]: Starting Wazuh v4.6.0...
Sep 25 14:03:14 wazuh-manager-master-0 env[7314]: Started wazuh-apid...
Sep 25 14:03:14 wazuh-manager-master-0 env[7314]: Started wazuh-csyslogd...
Sep 25 14:03:14 wazuh-manager-master-0 env[7314]: Started wazuh-dbd...
Sep 25 14:03:14 wazuh-manager-master-0 env[7314]: 2023/09/25 14:03:14 wazuh-integratord: INFO: Remote integrations not configured. Clean exit.
Sep 25 14:03:14 wazuh-manager-master-0 env[7314]: Started wazuh-integratord...
Sep 25 14:03:14 wazuh-manager-master-0 env[7314]: Started wazuh-agentlessd...
Sep 25 14:03:15 wazuh-manager-master-0 env[7314]: Started wazuh-authd...
Sep 25 14:03:16 wazuh-manager-master-0 env[7314]: Started wazuh-db...
Sep 25 14:03:17 wazuh-manager-master-0 env[7314]: Started wazuh-execd...
Sep 25 14:03:18 wazuh-manager-master-0 env[7314]: Started wazuh-analysisd...
Sep 25 14:03:19 wazuh-manager-master-0 env[7314]: Started wazuh-syscheckd...
Sep 25 14:03:20 wazuh-manager-master-0 env[7314]: Started wazuh-remoted...
Sep 25 14:03:22 wazuh-manager-master-0 env[7314]: Started wazuh-logcollector...
Sep 25 14:03:23 wazuh-manager-master-0 env[7314]: Started wazuh-monitord...
Sep 25 14:03:24 wazuh-manager-master-0 crontab[7704]: (root) LIST (root)
Sep 25 14:03:24 wazuh-manager-master-0 env[7314]: Started wazuh-modulesd...
Sep 25 14:03:25 wazuh-manager-master-0 env[7314]: Started wazuh-clusterd...
Sep 25 14:03:27 wazuh-manager-master-0 env[7314]: Completed.
Sep 25 14:03:27 wazuh-manager-master-0 systemd[1]: Started Wazuh manager.
-- Subject: Unit wazuh-manager.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-manager.service has finished starting up.
--
-- The start-up result is done.

4. Check for errors in ossec.log: egrep -i "ERROR|WARNING" /var/ossec/logs/ossec.log: 🟡

2023/09/27 20:17:57 wazuh-modulesd:vulnerability-detector: ERROR: (5502): Could not load the CVE OVAL for 'SLES15'. 'XMLERR: Overflow attempt at attribute 'comment'.'
2023/09/27 20:19:20 wazuh-modulesd:vulnerability-detector: ERROR: (5502): Could not load the CVE OVAL for 'SLED15'. 'XMLERR: Overflow attempt at attribute 'comment'.'
2023/09/27 20:19:21 wazuh-modulesd:vulnerability-detector: ERROR: (5513): CVE database could not be updated.

Note: Known issue #19127 - Fixed for 4.5.3

5. Check for errors in cluster.log: egrep -i "ERROR|WARNING" /var/ossec/logs/cluster.log: 🟢

6. Check the filebeat output: filebeat test output:🟢

elasticsearch: https://10.0.2.172:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 10.0.2.172
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2
elasticsearch: https://10.0.2.191:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 10.0.2.191
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2
elasticsearch: https://10.0.2.20:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 10.0.2.20
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2

Worker env 1 🟡

1. Check the general status: systemctl status wazuh-manager -l:🟢

● wazuh-manager.service - Wazuh manager
   Loaded: loaded (/usr/lib/systemd/system/wazuh-manager.service; enabled; vendor preset: disabled)
   Active: active (exited) since Mon 2023-09-25 14:09:07 UTC; 2 days ago
  Process: 10718 ExecStop=/usr/bin/env /var/ossec/bin/wazuh-control stop (code=exited, status=0/SUCCESS)
  Process: 10857 ExecStart=/usr/bin/env /var/ossec/bin/wazuh-control start (code=exited, status=0/SUCCESS)

Sep 25 14:08:58 wazuh-manager-worker-0 env[10857]: Started wazuh-analysisd...
Sep 25 14:08:59 wazuh-manager-worker-0 env[10857]: Started wazuh-syscheckd...
Sep 25 14:09:00 wazuh-manager-worker-0 env[10857]: Started wazuh-remoted...
Sep 25 14:09:01 wazuh-manager-worker-0 env[10857]: Started wazuh-logcollector...
Sep 25 14:09:02 wazuh-manager-worker-0 env[10857]: Started wazuh-monitord...
Sep 25 14:09:03 wazuh-manager-worker-0 env[10857]: Started wazuh-modulesd...
Sep 25 14:09:04 wazuh-manager-worker-0 crontab[11253]: (root) LIST (root)
Sep 25 14:09:05 wazuh-manager-worker-0 env[10857]: Started wazuh-clusterd...
Sep 25 14:09:07 wazuh-manager-worker-0 env[10857]: Completed.
Sep 25 14:09:07 wazuh-manager-worker-0 systemd[1]: Started Wazuh manager.

2. Check the modules status: /var/ossec/bin/wazuh-control status: 🟢

3. Check the service status: journalctl -xe -u wazuh-manager.service: 🟢

Sep 25 14:08:49 wazuh-manager-worker-0 systemd[1]: Starting Wazuh manager...
-- Subject: Unit wazuh-manager.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-manager.service has begun starting up.
Sep 25 14:08:52 wazuh-manager-worker-0 env[10857]: Starting Wazuh v4.6.0...
Sep 25 14:08:54 wazuh-manager-worker-0 env[10857]: Started wazuh-apid...
Sep 25 14:08:54 wazuh-manager-worker-0 env[10857]: Started wazuh-csyslogd...
Sep 25 14:08:54 wazuh-manager-worker-0 env[10857]: Started wazuh-dbd...
Sep 25 14:08:54 wazuh-manager-worker-0 env[10857]: 2023/09/25 14:08:54 wazuh-integratord: INFO: Remote integrations not configured. Clean exit.
Sep 25 14:08:54 wazuh-manager-worker-0 env[10857]: Started wazuh-integratord...
Sep 25 14:08:54 wazuh-manager-worker-0 env[10857]: Started wazuh-agentlessd...
Sep 25 14:08:56 wazuh-manager-worker-0 env[10857]: Started wazuh-db...
Sep 25 14:08:57 wazuh-manager-worker-0 env[10857]: Started wazuh-execd...
Sep 25 14:08:58 wazuh-manager-worker-0 env[10857]: Started wazuh-analysisd...
Sep 25 14:08:59 wazuh-manager-worker-0 env[10857]: Started wazuh-syscheckd...
Sep 25 14:09:00 wazuh-manager-worker-0 env[10857]: Started wazuh-remoted...
Sep 25 14:09:01 wazuh-manager-worker-0 env[10857]: Started wazuh-logcollector...
Sep 25 14:09:02 wazuh-manager-worker-0 env[10857]: Started wazuh-monitord...
Sep 25 14:09:03 wazuh-manager-worker-0 env[10857]: Started wazuh-modulesd...
Sep 25 14:09:04 wazuh-manager-worker-0 crontab[11253]: (root) LIST (root)
Sep 25 14:09:05 wazuh-manager-worker-0 env[10857]: Started wazuh-clusterd...
Sep 25 14:09:07 wazuh-manager-worker-0 env[10857]: Completed.
Sep 25 14:09:07 wazuh-manager-worker-0 systemd[1]: Started Wazuh manager.
-- Subject: Unit wazuh-manager.service has finished start-up

4. Check for errors in ossec.log: egrep -i "ERROR|WARNING" /var/ossec/logs/ossec.log: 🟡

2023/09/27 20:00:24 wazuh-modulesd:vulnerability-detector: ERROR: (5502): Could not load the CVE OVAL for 'SLES15'. 'XMLERR: Overflow attempt at attribute 'comment'.'
2023/09/27 20:01:42 wazuh-modulesd:vulnerability-detector: ERROR: (5502): Could not load the CVE OVAL for 'SLED15'. 'XMLERR: Overflow attempt at attribute 'comment'.'
2023/09/27 20:01:43 wazuh-modulesd:vulnerability-detector: ERROR: (5513): CVE database could not be updated.

Note: Known issue #19127 - Fixed for 4.5.3

5. Check for errors in cluster.log: egrep -i "ERROR|WARNING" /var/ossec/logs/cluster.log: 🟢

6. Check the filebeat output: filebeat test output:🟢

.elasticsearch: https://10.0.2.172:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 10.0.2.172
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2
elasticsearch: https://10.0.2.191:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 10.0.2.191
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2
elasticsearch: https://10.0.2.20:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 10.0.2.20
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2

Master env 2 🟡

1. Check the general status: systemctl status wazuh-manager -l:🟢

● wazuh-manager.service - Wazuh manager
   Loaded: loaded (/usr/lib/systemd/system/wazuh-manager.service; enabled; vendor preset: disabled)
   Active: active (exited) since Mon 2023-09-25 14:05:36 UTC; 2 days ago
  Process: 10962 ExecStop=/usr/bin/env /var/ossec/bin/wazuh-control stop (code=exited, status=0/SUCCESS)
  Process: 11116 ExecStart=/usr/bin/env /var/ossec/bin/wazuh-control start (code=exited, status=0/SUCCESS)

Sep 25 14:05:27 wazuh-manager-master-0 env[11116]: Started wazuh-analysisd...
Sep 25 14:05:28 wazuh-manager-master-0 env[11116]: Started wazuh-syscheckd...
Sep 25 14:05:29 wazuh-manager-master-0 env[11116]: Started wazuh-remoted...
Sep 25 14:05:30 wazuh-manager-master-0 env[11116]: Started wazuh-logcollector...
Sep 25 14:05:32 wazuh-manager-master-0 env[11116]: Started wazuh-monitord...
Sep 25 14:05:33 wazuh-manager-master-0 env[11116]: Started wazuh-modulesd...
Sep 25 14:05:33 wazuh-manager-master-0 crontab[11531]: (root) LIST (root)
Sep 25 14:05:34 wazuh-manager-master-0 env[11116]: Started wazuh-clusterd...
Sep 25 14:05:36 wazuh-manager-master-0 env[11116]: Completed.
Sep 25 14:05:36 wazuh-manager-master-0 systemd[1]: Started Wazuh manager.

2. Check the modules status: /var/ossec/bin/wazuh-control status: 🟢

wazuh-clusterd is running...
wazuh-modulesd is running...
wazuh-monitord is running...
wazuh-logcollector is running...
wazuh-remoted is running...
wazuh-syscheckd is running...
wazuh-analysisd is running...
wazuh-maild not running...
wazuh-execd is running...
wazuh-db is running...
wazuh-authd is running...
wazuh-agentlessd not running...
wazuh-integratord is running...
wazuh-dbd not running...
wazuh-csyslogd not running...
wazuh-apid is running...

3. Check the service status: journalctl -xe -u wazuh-manager.service: 🟢

Sep 25 14:05:17 wazuh-manager-master-0 systemd[1]: Starting Wazuh manager...
-- Subject: Unit wazuh-manager.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-manager.service has begun starting up.
Sep 25 14:05:19 wazuh-manager-master-0 env[11116]: Starting Wazuh v4.6.0...
Sep 25 14:05:23 wazuh-manager-master-0 env[11116]: Started wazuh-apid...
Sep 25 14:05:23 wazuh-manager-master-0 env[11116]: Started wazuh-csyslogd...
Sep 25 14:05:23 wazuh-manager-master-0 env[11116]: Started wazuh-dbd...
Sep 25 14:05:23 wazuh-manager-master-0 env[11116]: 2023/09/25 14:05:23 wazuh-integratord: INFO: Remote integrations not configured. Clean exit.
Sep 25 14:05:23 wazuh-manager-master-0 env[11116]: Started wazuh-integratord...
Sep 25 14:05:23 wazuh-manager-master-0 env[11116]: Started wazuh-agentlessd...
Sep 25 14:05:24 wazuh-manager-master-0 env[11116]: Started wazuh-authd...
Sep 25 14:05:25 wazuh-manager-master-0 env[11116]: Started wazuh-db...
Sep 25 14:05:26 wazuh-manager-master-0 env[11116]: Started wazuh-execd...
Sep 25 14:05:27 wazuh-manager-master-0 env[11116]: Started wazuh-analysisd...
Sep 25 14:05:28 wazuh-manager-master-0 env[11116]: Started wazuh-syscheckd...
Sep 25 14:05:29 wazuh-manager-master-0 env[11116]: Started wazuh-remoted...
Sep 25 14:05:30 wazuh-manager-master-0 env[11116]: Started wazuh-logcollector...
Sep 25 14:05:32 wazuh-manager-master-0 env[11116]: Started wazuh-monitord...
Sep 25 14:05:33 wazuh-manager-master-0 env[11116]: Started wazuh-modulesd...
Sep 25 14:05:33 wazuh-manager-master-0 crontab[11531]: (root) LIST (root)
Sep 25 14:05:34 wazuh-manager-master-0 env[11116]: Started wazuh-clusterd...
Sep 25 14:05:36 wazuh-manager-master-0 env[11116]: Completed.
Sep 25 14:05:36 wazuh-manager-master-0 systemd[1]: Started Wazuh manager.
-- Subject: Unit wazuh-manager.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit wazuh-manager.service has finished starting up.
--
-- The start-up result is done.

4. Check for errors in ossec.log: egrep -i "ERROR|WARNING" /var/ossec/logs/ossec.log: 🟡

2023/09/27 20:59:30 wazuh-modulesd:vulnerability-detector: ERROR: (5502): Could not load the CVE OVAL for 'SLES15'. 'XMLERR: Overflow attempt at attribute 'comment'.'
2023/09/27 21:01:36 wazuh-modulesd:vulnerability-detector: ERROR: (5502): Could not load the CVE OVAL for 'SLED15'. 'XMLERR: Overflow attempt at attribute 'comment'.'
2023/09/27 21:01:36 wazuh-modulesd:vulnerability-detector: ERROR: (5513): CVE database could not be updated.

Note: Known issue #19127 - Fixed for 4.5.3

5. Check for errors in cluster.log: egrep -i "ERROR|WARNING" /var/ossec/logs/cluster.log: 🟢

6. Check the filebeat output: filebeat test output:🟢

elasticsearch: https://10.0.2.172:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 10.0.2.172
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2
elasticsearch: https://10.0.2.191:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 10.0.2.191
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2
elasticsearch: https://10.0.2.20:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 10.0.2.20
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2

Wazuh Indexer

Node 1 🟢

1. Check the general status: systemctl status wazuh-indexer -l: 🟡

● wazuh-indexer.service - Wazuh-indexer
   Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2023-09-25 13:54:37 UTC; 2 days ago
     Docs: https://documentation.wazuh.com
 Main PID: 8592 (java)
   CGroup: /system.slice/wazuh-indexer.service
           └─8592 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms3928m -Xmx3928m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-5854299814286760887 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=2059403264 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:593)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:561)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:484)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:186)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:747)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedOpenSearchThreadPoolExecutor.java:282)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedOpenSearchThreadPoolExecutor.java:245)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at java.base/java.lang.Thread.run(Thread.java:833)

2. Check the service status: journalctl -xe -u wazuh-indexer.service: 🟢

Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:485)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at java.base/java.security.AccessController.checkPermission(AccessController.java:1068)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:416)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.checkWriteExtended(UnixFileAttributeViews.java:195)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.setMode(UnixFileAttributeViews.java:264)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.setPermissions(UnixFileAttributeViews.java:299)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.util.FileUtils.defineFilePosixAttributeView(FileUtils.java:177)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.appender.FileManager.defineAttributeView(FileManager.java:215)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.appender.FileManager.createOutputStream(FileManager.java:202)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.createFileAfterRollover(RollingFileManager.java:419
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.rollover(RollingFileManager.java:396)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.checkRollover(RollingFileManager.java:308)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.appender.RollingFileAppender.append(RollingFileAppender.java:311)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:161)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:134)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:125)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:89)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:542)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:500)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:483)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:8
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.core.Logger.log(Logger.java:161)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2205)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2159)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2142)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2017)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1983)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.apache.logging.log4j.spi.AbstractLogger.info(AbstractLogger.java:1320)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.plugins.PluginsService.onIndexModule(PluginsService.java:308)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.indices.IndicesService.createIndexService(IndicesService.java:811)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.indices.IndicesService.createIndex(IndicesService.java:702)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.indices.IndicesService.createIndex(IndicesService.java:210)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.indices.cluster.IndicesClusterStateService.createIndices(IndicesClusterStateService.java:547)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.indices.cluster.IndicesClusterStateService.applyClusterState(IndicesClusterStateService.java:294)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:606)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:593)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:561)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:484)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:186)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:747)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndCl
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(Prio
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
Sep 27 00:00:03 ip-10-0-2-20.us-west-1.compute.internal systemd-entrypoint[8592]: at java.base/java.lang.Thread.run(Thread.java:833)

3. Check for errors in wazuh.log: egrep -i "ERROR|WARNING" /var/log/wazuh-indexer/wazuh.log: 🟢

Node 2 🟡

1. Check the general status: systemctl status wazuh-indexer -l: 🟡

● wazuh-indexer.service - Wazuh-indexer
   Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2023-09-25 13:54:07 UTC; 2 days ago
     Docs: https://documentation.wazuh.com
 Main PID: 8594 (java)
   CGroup: /system.slice/wazuh-indexer.service
           └─8594 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms3928m -Xmx3928m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-8351906198100010787 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=2059403264 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.cluster.service.MasterService.runTasks(MasterService.java:295)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.cluster.service.MasterService$Batcher.run(MasterService.java:206)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:204)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:242)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:747)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedOpenSearchThreadPoolExecutor.java:282)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedOpenSearchThreadPoolExecutor.java:245)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/java.lang.Thread.run(Thread.java:833)

2. Check the service status: journalctl -xe -u wazuh-indexer.service: 🟢

Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/java.lang.Thread.run(Thread.java:833)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: 2023-09-27 00:00:03,096 opensearch[node-1][clusterManagerService#updateTask][T#1] ERROR Could not define attribute view
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:485)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/java.security.AccessController.checkPermission(AccessController.java:1068)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:416)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.checkWriteExtended(UnixFileAttributeViews.java:195)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.setMode(UnixFileAttributeViews.java:264)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.setPermissions(UnixFileAttributeViews.java:299)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.util.FileUtils.defineFilePosixAttributeView(FileUtils.java:177)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.appender.FileManager.defineAttributeView(FileManager.java:215)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.appender.FileManager.createOutputStream(FileManager.java:202)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.createFileAfterRollover(RollingFileManager.java:41
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.rollover(RollingFileManager.java:396)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.checkRollover(RollingFileManager.java:308)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.appender.RollingFileAppender.append(RollingFileAppender.java:311)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:161)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:134)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:125)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:89)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:542)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:500)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:483)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.core.Logger.log(Logger.java:161)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2205)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2159)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2142)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2040)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1907)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.apache.logging.log4j.spi.AbstractLogger.info(AbstractLogger.java:1449)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.cluster.metadata.MetadataUpdateSettingsService$1.execute(MetadataUpdateSettingsService.java:247)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:65)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.cluster.service.MasterService.executeTasks(MasterService.java:874)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:424)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.cluster.service.MasterService.runTasks(MasterService.java:295)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.cluster.service.MasterService$Batcher.run(MasterService.java:206)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:204)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:242)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:747)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndC
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(Pri
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
Sep 27 00:00:03 ip-10-0-2-172.us-west-1.compute.internal systemd-entrypoint[8594]: at java.base/java.lang.Thread.run(Thread.java:833)

3. Check for errors in wazuh.log: egrep -i "ERROR|WARNING" /var/log/wazuh-indexer/wazuh.log: 🟡

[2023-09-27T01:54:12,841][ERROR][o.o.a.a.AlertIndices     ] [node-1] info deleteOldIndices
[2023-09-27T01:54:12,841][ERROR][o.o.a.a.AlertIndices     ] [node-1] info deleteOldIndices
[2023-09-27T01:54:12,912][ERROR][o.o.s.i.DetectorIndexManagementService] [node-1] info deleteOldIndices
[2023-09-27T01:54:12,912][ERROR][o.o.s.i.DetectorIndexManagementService] [node-1] info deleteOldIndices
[2023-09-27T13:54:12,843][ERROR][o.o.a.a.AlertIndices     ] [node-1] info deleteOldIndices
[2023-09-27T13:54:12,843][ERROR][o.o.a.a.AlertIndices     ] [node-1] info deleteOldIndices
[2023-09-27T13:54:12,912][ERROR][o.o.s.i.DetectorIndexManagementService] [node-1] info deleteOldIndices
[2023-09-27T13:54:12,913][ERROR][o.o.s.i.DetectorIndexManagementService] [node-1] info deleteOldIndices

Note: known issues: wazuh/wazuh-packages#2094 - opensearch-project/security-analytics#203

Node 3 🟢

1. Check the general status: systemctl status wazuh-indexer -l: 🟡

● wazuh-indexer.service - Wazuh-indexer
   Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2023-09-25 13:53:35 UTC; 2 days ago
     Docs: https://documentation.wazuh.com
 Main PID: 8357 (java)
   CGroup: /system.slice/wazuh-indexer.service
           └─8357 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms3928m -Xmx3928m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-4074345481855212232 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=2059403264 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:593)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:561)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:484)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:186)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:747)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedOpenSearchThreadPoolExecutor.java:282)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedOpenSearchThreadPoolExecutor.java:245)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at java.base/java.lang.Thread.run(Thread.java:833)

2. Check the service status: journalctl -xe -u wazuh-indexer.service: 🟢

Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:485)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at java.base/java.security.AccessController.checkPermission(AccessController.java:1068)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:416)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.checkWriteExtended(UnixFileAttributeViews.java:195)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.setMode(UnixFileAttributeViews.java:264)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.setPermissions(UnixFileAttributeViews.java:299)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.util.FileUtils.defineFilePosixAttributeView(FileUtils.java:177)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.appender.FileManager.defineAttributeView(FileManager.java:215)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.appender.FileManager.createOutputStream(FileManager.java:202)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.createFileAfterRollover(RollingFileManager.java:41
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.rollover(RollingFileManager.java:396)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.checkRollover(RollingFileManager.java:308)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.appender.RollingFileAppender.append(RollingFileAppender.java:311)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:161)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:134)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:125)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:89)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:542)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:500)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:483)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.core.Logger.log(Logger.java:161)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2205)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2159)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2142)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2017)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1983)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.apache.logging.log4j.spi.AbstractLogger.info(AbstractLogger.java:1320)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.plugins.PluginsService.onIndexModule(PluginsService.java:308)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.indices.IndicesService.createIndexService(IndicesService.java:811)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.indices.IndicesService.createIndex(IndicesService.java:702)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.indices.IndicesService.createIndex(IndicesService.java:210)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.indices.cluster.IndicesClusterStateService.createIndices(IndicesClusterStateService.java:547)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.indices.cluster.IndicesClusterStateService.applyClusterState(IndicesClusterStateService.java:294)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:606)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:593)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:561)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:484)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:186)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:747)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndC
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(Pri
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
Sep 27 00:00:03 ip-10-0-2-191.us-west-1.compute.internal systemd-entrypoint[8357]: at java.base/java.lang.Thread.run(Thread.java:833)

3. Check for errors in wazuh.log: egrep -i "ERROR|WARNING" /var/log/wazuh-indexer/wazuh.log: 🟢

Wazuh Dashboard

wazuh-indexer 🟢

1. Check the general status: systemctl status wazuh-indexer -l: 🟢

 wazuh-indexer.service - Wazuh-indexer
   Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2023-09-25 14:01:12 UTC; 2 days ago
     Docs: https://documentation.wazuh.com
 Main PID: 10879 (java)
   CGroup: /system.slice/wazuh-indexer.service
           └─10879 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms2560m -Xmx2560m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-9966713264233166992 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=1342177280 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:593)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:561)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:484)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:186)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:747)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedOpenSearchThreadPoolExecutor.java:282)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedOpenSearchThreadPoolExecutor.java:245)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at java.base/java.lang.Thread.run(Thread.java:833)

2. Check the service status: journalctl -xe -u wazuh-indexer.service 🟢

Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:485)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at java.base/java.security.AccessController.checkPermission(AccessController.java:1068)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:416)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.checkWriteExtended(UnixFileAttributeViews.java:195)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.setMode(UnixFileAttributeViews.java:264)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.setPermissions(UnixFileAttributeViews.java:299)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.util.FileUtils.defineFilePosixAttributeView(FileUtils.java:177)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.appender.FileManager.defineAttributeView(FileManager.java:215)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.appender.FileManager.createOutputStream(FileManager.java:202)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.createFileAfterRollover(RollingFileManager.java:4
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.rollover(RollingFileManager.java:396)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.checkRollover(RollingFileManager.java:308)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.appender.RollingFileAppender.append(RollingFileAppender.java:311)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:161)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:134)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:125)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:89)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:542)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:500)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:483)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.core.Logger.log(Logger.java:161)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2205)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2159)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2142)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2017)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1983)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.apache.logging.log4j.spi.AbstractLogger.info(AbstractLogger.java:1320)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.plugins.PluginsService.onIndexModule(PluginsService.java:308)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.indices.IndicesService.createIndexService(IndicesService.java:811)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.indices.IndicesService.createIndex(IndicesService.java:702)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.indices.IndicesService.createIndex(IndicesService.java:210)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.indices.cluster.IndicesClusterStateService.createIndices(IndicesClusterStateService.java:547)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.indices.cluster.IndicesClusterStateService.applyClusterState(IndicesClusterStateService.java:294)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:606)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:593)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:561)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:484)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:186)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:747)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAnd
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at org.opensearch.common.util.concurrent.PrioritizedOpenSearchThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(Pr
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
Sep 28 00:00:02 ip-xx-xx-xx-xx.us-west-1.compute.internal systemd-entrypoint[10879]: at java.base/java.lang.Thread.run(Thread.java:833)

3. Check for errors in wazuh.log: egrep -i "ERROR|WARNING" /var/log/wazuh-indexer/wazuh.log: 🟢

wazuh-dashboard 🔴

1. Check the general status: systemctl status wazuh-dashboard -l 🟢

● wazuh-dashboard.service - wazuh-dashboard
   Loaded: loaded (/etc/systemd/system/wazuh-dashboard.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2023-09-25 14:20:41 UTC; 2 days ago
 Main PID: 12675 (node)
   CGroup: /system.slice/wazuh-dashboard.service
           └─12675 /usr/share/wazuh-dashboard/node/bin/node --no-warnings --max-http-header-size=65536 --unhandled-rejections=warn /usr/share/wazuh-dashboard/src/cli/dist -c /etc/wazuh-dashboard/opensearch_dashboards.yml

Sep 28 12:32:48 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:32:47Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":{"url":"/46001/bundles/plugin/discover/discover.chunk.2.js","method":"get","headers":{"host":"xx-xx-xx-xx:5601","connection":"close","user-agent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0","accept":"*/*","accept-language":"es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3","accept-encoding":"gzip, deflate, br","referer":"https://demo-460-b1-wazuh-c38e9cc4db05142b.elb.us-west-1.amazonaws.com/app/discover","sec-fetch-dest":"script","sec-fetch-mode":"no-cors","sec-fetch-site":"same-origin","pragma":"no-cache","cache-control":"no-cache"},"remoteAddress":"xx-xx-xx-xx","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0","referer":"https://demo-460-b1-wazuh-c38e9cc4db05142b.elb.us-west-1.amazonaws.com/app/discover"},"res":{"statusCode":200,"responseTime":15,"contentLength":9},"message":"GET /46001/bundles/plugin/discover/discover.chunk.2.js 200 15ms - 9.0B"}
Sep 28 12:32:48 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:32:48Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":{"url":"/46001/bundles/plugin/discover/discover.chunk.4.js","method":"get","headers":{"host":"xx-xx-xx-xx:5601","connection":"close","user-agent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0","accept":"*/*","accept-language":"es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3","accept-encoding":"gzip, deflate, br","referer":"https://demo-460-b1-wazuh-c38e9cc4db05142b.elb.us-west-1.amazonaws.com/app/discover","sec-fetch-dest":"script","sec-fetch-mode":"no-cors","sec-fetch-site":"same-origin","pragma":"no-cache","cache-control":"no-cache"},"remoteAddress":"xx-xx-xx-xx","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0","referer":"https://demo-460-b1-wazuh-c38e9cc4db05142b.elb.us-west-1.amazonaws.com/app/discover"},"res":{"statusCode":200,"responseTime":3,"contentLength":9},"message":"GET /46001/bundles/plugin/discover/discover.chunk.4.js 200 3ms - 9.0B"}
Sep 28 12:32:48 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:32:48Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":{"url":"/46001/bundles/plugin/discover/discover.chunk.9.js","method":"get","headers":{"host":"xx-xx-xx-xx:5601","connection":"close","user-agent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0","accept":"*/*","accept-language":"es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3","accept-encoding":"gzip, deflate, br","referer":"https://demo-460-b1-wazuh-c38e9cc4db05142b.elb.us-west-1.amazonaws.com/app/discover","sec-fetch-dest":"script","sec-fetch-mode":"no-cors","sec-fetch-site":"same-origin","pragma":"no-cache","cache-control":"no-cache"},"remoteAddress":"xx-xx-xx-xx","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0","referer":"https://demo-460-b1-wazuh-c38e9cc4db05142b.elb.us-west-1.amazonaws.com/app/discover"},"res":{"statusCode":200,"responseTime":3,"contentLength":9},"message":"GET /46001/bundles/plugin/discover/discover.chunk.9.js 200 3ms - 9.0B"}
Sep 28 12:32:48 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:32:48Z","tags":[],"pid":12675,"method":"post","statusCode":200,"req":{"url":"/api/request","method":"post","headers":{"host":"xx-xx-xx-xx:5601","connection":"close","content-length":"56","user-agent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0","accept":"application/json, text/plain, */*","accept-language":"es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3","accept-encoding":"gzip, deflate, br","content-type":"application/json","osd-xsrf":"kibana","origin":"https://demo-460-b1-wazuh-c38e9cc4db05142b.elb.us-west-1.amazonaws.com","referer":"https://demo-460-b1-wazuh-c38e9cc4db05142b.elb.us-west-1.amazonaws.com/app/discover","sec-fetch-dest":"empty","sec-fetch-mode":"cors","sec-fetch-site":"same-origin","pragma":"no-cache","cache-control":"no-cache"},"remoteAddress":"xx-xx-xx-xx","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/117.0","referer":"https://demo-460-b1-wazuh-c38e9cc4db05142b.elb.us-west-1.amazonaws.com/app/discover"},"res":{"statusCode":200,"responseTime":39,"contentLength":9},"message":"POST /api/request 200 39ms - 9.0B"}

2. Check the service status: journalctl -xe -u wazuh-dashboard.service: 🟢

Sep 28 12:28:56 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:28:56Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":
Sep 28 12:28:57 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:28:57Z","tags":[],"pid":12675,"method":"post","statusCode":200,"req"
Sep 28 12:28:57 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:28:57Z","tags":[],"pid":12675,"method":"post","statusCode":200,"req"
Sep 28 12:28:57 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:28:57Z","tags":[],"pid":12675,"method":"post","statusCode":200,"req"
Sep 28 12:29:00 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:00Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":
Sep 28 12:29:00 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:00Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":
Sep 28 12:29:04 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:04Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":
Sep 28 12:29:04 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:04Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":
Sep 28 12:29:04 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:04Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":
Sep 28 12:29:04 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:04Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":
Sep 28 12:29:05 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:05Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":
Sep 28 12:29:05 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:05Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":
Sep 28 12:29:05 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:05Z","tags":[],"pid":12675,"method":"post","statusCode":200,"req"
Sep 28 12:29:06 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:05Z","tags":[],"pid":12675,"method":"post","statusCode":200,"req"
Sep 28 12:29:07 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:07Z","tags":[],"pid":12675,"method":"put","statusCode":200,"req":
Sep 28 12:29:08 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:07Z","tags":[],"pid":12675,"method":"post","statusCode":200,"req"
Sep 28 12:29:08 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:08Z","tags":[],"pid":12675,"method":"put","statusCode":200,"req":
Sep 28 12:29:08 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:08Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":
Sep 28 12:29:10 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:09Z","tags":[],"pid":12675,"method":"post","statusCode":200,"req"
Sep 28 12:29:10 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:29:10Z","tags":[],"pid":12675,"method":"put","statusCode":200,"req":
Sep 28 12:32:46 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:32:46Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":
Sep 28 12:32:46 ip-xx-xx-xx-xx.us-west-1.compute.internal opensearch-dashboards[12675]: {"type":"response","@timestamp":"2023-09-28T12:32:46Z","tags":[],"pid":12675,"method":"get","statusCode":200,"req":

3. Check for errors in wazuhapp.log: egrep -i "err|warn" /usr/share/wazuh-dashboard/data/wazuh/logs/wazuhapp.log: 🔴

{"date":"2023-09-28T12:26:15.066Z","level":"error","location":"queue:delayApiRequest","message":"An error ocurred in the delayed request: \"DELETE /security/user/authenticate\": Request failed with status code 401"}

• New Issue Error ocurred in Delayed API Request on Demo Enviroment wazuh-dashboard-plugins#5958

[Deblintrake09]

T2 - The daemons are running with the correct user

Agents

Amazon Linux 2 🟢

1. Check the user: ps -aux | grep wazuh 🟢

root      3831  0.0  0.3  38384  3068 ?        Sl   sep25   0:07 /var/ossec/bin/wazuh-execd
wazuh     3843  0.0  0.5 260204  4848 ?        Sl   sep25   0:35 /var/ossec/bin/wazuh-agentd
root      3858  0.0  0.9 230648  8608 ?        SNl  sep25   1:08 /var/ossec/bin/wazuh-syscheckd
root      3874  0.0  0.4 480836  4316 ?        Sl   sep25   0:26 /var/ossec/bin/wazuh-logcollector
root      3891  0.0  1.2 749056 11532 ?        Sl   sep25   0:14 /var/ossec/bin/wazuh-modulesd

CentOS 🟢

1. Check the user: ps -aux | grep wazuh 🟢

root        9449  0.0  0.2  43444  1920 ?        Sl   sep25   0:04 /var/ossec/bin/wazuh-execd
wazuh       9461  0.0  0.6 274248  5168 ?        Sl   sep25   0:29 /var/ossec/bin/wazuh-agentd
root        9476  0.0  1.0 373340  8316 ?        SNl  sep25   1:20 /var/ossec/bin/wazuh-syscheckd
root        9492  0.0  0.3 486000  3064 ?        Sl   sep25   0:20 /var/ossec/bin/wazuh-logcollector
root        9510  0.0  2.6 759144 21488 ?        Sl   sep25   0:14 /var/ossec/bin/wazuh-modulesd

Debian 🟢

1. Check the user: ps -aux | grep wazuh 🟢

root        9067  0.0  0.2  24220  2052 ?        Sl   Sep25   0:07 /var/ossec/bin/wazuh-execd
wazuh       9078  0.0  0.9 245980  9744 ?        Sl   Sep25   0:35 /var/ossec/bin/wazuh-agentd
root        9092  0.0  0.6 212024  6556 ?        SNl  Sep25   1:00 /var/ossec/bin/wazuh-syscheckd
root        9107  0.0  0.2 466776  2668 ?        Sl   Sep25   0:26 /var/ossec/bin/wazuh-logcollector
root        9126  0.0  1.4 728836 14504 ?        Sl   Sep25   0:13 /var/ossec/bin/wazuh-modulesd

Ubuntu 🟢

1. Check the user: ps -aux | grep wazuh 🟢

root        9303  0.0  0.2  24060  2324 ?        Sl   Sep25   0:09 /var/ossec/bin/wazuh-execd
wazuh       9314  0.0  0.3 245832  3812 ?        Sl   Sep25   0:49 /var/ossec/bin/wazuh-agentd
root        9328  0.0  0.6 212064  6496 ?        SNl  Sep25   1:32 /var/ossec/bin/wazuh-syscheckd
root        9343  0.0  0.2 466528  2212 ?        Sl   Sep25   0:34 /var/ossec/bin/wazuh-logcollector
root        9360  0.0  1.2 728628 11768 ?        Sl   Sep25   0:20 /var/ossec/bin/wazuh-modulesd

RHEL🟢

1. Check the user: ps -aux | grep wazuh 🟢

root       60179  0.0  0.1  24000  6212 ?        Sl   Sep25   0:04 /var/ossec/bin/wazuh-execd
wazuh      60191  0.0  0.2 245648  7564 ?        Sl   Sep25   1:22 /var/ossec/bin/wazuh-agentd
root       60206  0.0  0.4 424964 15888 ?        SNl  Sep25   1:54 /var/ossec/bin/wazuh-syscheckd
root       60226  0.0  0.2 466512  7708 ?        Sl   Sep25   0:37 /var/ossec/bin/wazuh-logcollector
root       60243  0.0  1.0 1022384 40256 ?       Sl   Sep25   0:47 /var/ossec/bin/wazuh-modulesd

Windows 🟢

1. Check the user: tasklist /svc | Select-String "wazuh" 🟢
   

Managers

Master env 1 🟢

1. Check the user: ps -aux | grep wazuh 🟢

wazuh    18357  0.0  3.0 998108 120728 ?       Sl   Sep25   1:58 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    18358  0.0  1.8 333284 72172 ?        S    Sep25   0:05 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    18361  0.0  1.9 418152 75192 ?        S    Sep25   3:04 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    18364  0.0  1.5 617760 63060 ?        S    Sep25   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    18390  0.0  0.1  39200  4368 ?        Sl   Sep25   0:22 /var/ossec/bin/wazuh-integratord
root     18411  0.2  0.1 260492  4512 ?        Sl   Sep25   7:45 /var/ossec/bin/wazuh-authd
wazuh    18427  0.1  0.6 850472 27376 ?        Sl   Sep25   4:02 /var/ossec/bin/wazuh-db
root     18452  0.0  0.0  39244  3460 ?        Sl   Sep25   0:06 /var/ossec/bin/wazuh-execd
wazuh    18467  0.9  4.2 1308804 168404 ?      Sl   Sep25  31:12 /var/ossec/bin/wazuh-analysisd
root     18481  0.0  0.2 423840 11064 ?        SNl  Sep25   1:16 /var/ossec/bin/wazuh-syscheckd
wazuh    18501  0.2  0.2 1122192 9736 ?        Sl   Sep25   8:58 /var/ossec/bin/wazuh-remoted
root     18535  0.0  0.1 481644  5056 ?        Sl   Sep25   0:24 /var/ossec/bin/wazuh-logcollector
wazuh    18555  0.0  0.1  39216  4448 ?        Sl   Sep25   1:21 /var/ossec/bin/wazuh-monitord
root     18603 11.5  0.8 1119672 32316 ?       Sl   Sep25 377:39 /var/ossec/bin/wazuh-modulesd
wazuh    18740  0.1  1.6 467772 65844 ?        Sl   Sep25   5:10 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    18768  0.0  1.3 300156 53720 ?        S    Sep25   0:47 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    18769  0.0  1.4 301672 55672 ?        S    Sep25   0:48 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py

Worker env 1 🟢

1. Check the user: ps -aux | grep wazuh 🟢

wazuh    11826  0.0  2.6 907088 103052 ?       Sl   Sep25   0:11 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    11827  0.0  1.5 322840 61756 ?        S    Sep25   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    11830  0.0  1.5 404768 61896 ?        S    Sep25   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    11833  0.0  1.5 552232 61864 ?        S    Sep25   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    11859  0.0  0.0  39164  3404 ?        Sl   Sep25   0:07 /var/ossec/bin/wazuh-integratord
wazuh    11878  0.0  0.3 784900 13728 ?        Sl   Sep25   1:56 /var/ossec/bin/wazuh-db
root     11903  0.0  0.0  39200  3344 ?        Sl   Sep25   0:06 /var/ossec/bin/wazuh-execd
wazuh    11918  0.0  0.8 1294672 33116 ?       Sl   Sep25   0:23 /var/ossec/bin/wazuh-analysisd
root     11931  0.0  0.2 358204  9764 ?        SNl  Sep25   1:10 /var/ossec/bin/wazuh-syscheckd
wazuh    11951  0.1  0.1 466396  4156 ?        Sl   Sep25   5:34 /var/ossec/bin/wazuh-remoted
root     11984  0.0  0.1 481604  4744 ?        Sl   Sep25   0:21 /var/ossec/bin/wazuh-logcollector
wazuh    12005  0.0  0.1  39172  4432 ?        Sl   Sep25   0:08 /var/ossec/bin/wazuh-monitord
root     12053  7.5  7.4 1078692 292752 ?      Sl   Sep25 247:44 /var/ossec/bin/wazuh-modulesd
wazuh    12197  0.1  1.4 608416 58516 ?        Sl   Sep25   4:51 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    12288  0.0  1.4 307940 55400 ?        S    Sep25   1:57 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    13292  0.0  1.3 460540 54484 ?        S    Sep25   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py

Master env 2 🟢

1. Check the user: ps -aux | grep wazuh 🟢

wazuh    21246  0.0  2.7 986748 109440 ?       Sl   Sep25   1:21 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    21247  0.0  1.6 330968 66716 ?        S    Sep25   0:03 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    21250  0.0  1.9 418280 75404 ?        S    Sep25   2:46 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    21253  0.0  1.5 552224 62948 ?        S    Sep25   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    21279  0.0  0.1  39200  4052 ?        Sl   Sep25   0:20 /var/ossec/bin/wazuh-integratord
root     21299  0.2  0.1 194960  4656 ?        Sl   Sep25   7:49 /var/ossec/bin/wazuh-authd
wazuh    21317  0.0  0.5 784936 21940 ?        Sl   Sep25   2:35 /var/ossec/bin/wazuh-db
root     21342  0.0  0.0  39240  3352 ?        Sl   Sep25   0:06 /var/ossec/bin/wazuh-execd
wazuh    21357  0.7  3.4 1294748 135124 ?      Sl   Sep25  23:19 /var/ossec/bin/wazuh-analysisd
root     21370  0.0  0.2 423844 11168 ?        SNl  Sep25   1:15 /var/ossec/bin/wazuh-syscheckd
wazuh    21390  0.1  0.2 925112  8900 ?        Sl   Sep25   5:38 /var/ossec/bin/wazuh-remoted
root     21424  0.0  0.1 481644  5000 ?        Sl   Sep25   0:24 /var/ossec/bin/wazuh-logcollector
wazuh    21444  0.0  0.1  39212  4488 ?        Sl   Sep25   1:15 /var/ossec/bin/wazuh-monitord
root     21493  8.0  5.7 1239480 227804 ?      Sl   Sep25 263:00 /var/ossec/bin/wazuh-modulesd
wazuh    21621  0.0  1.4 448360 57780 ?        Sl   Sep25   0:59 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    21629  0.0  1.3 300156 53680 ?        S    Sep25   0:43 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    21630  0.0  1.3 300288 53780 ?        S    Sep25   0:43 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py

Indexers

Node 1 🟢

1. Check the user: ps -aux | grep wazuh 🟢

wazuh-i+  8592  0.9 56.8 7146516 4572968 ?     Ssl  Sep25  33:06 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms3928m -Xmx3928m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-5854299814286760887 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=2059403264 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

Node 2 🟢

1. Check the user: ps -aux | grep wazuh 🟢

wazuh-i+  8594  1.0 57.1 7157204 4597380 ?     Ssl  Sep25  36:21 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms3928m -Xmx3928m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-8351906198100010787 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=2059403264 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

Node 3 🟢

1. Check the user: ps -aux | grep wazuh 🟢

wazuh-i+  8357  1.0 56.8 7091520 4577708 ?     Ssl  Sep25  33:33 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms3928m -Xmx3928m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-4074345481855212232 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=2059403264 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

[Deblintrake09]

T3 - The status of the Wazuh Indexer clusters is as expected

wazuh-indexer 🟢

• Check nodes: curl -k -u ADMIN_USER:PASS https://indexer_IP:9200/_cat/nodes?v

ip         heap.percent ram.percent cpu load_1m load_5m load_15m node.role node.roles                                        cluster_manager name
X.X.X.X            29          89   0    0.00    0.00     0.00 dimr      cluster_manager,data,ingest,remote_cluster_client -               node-3
Y.Y.Y.Y           54          90   0    0.00    0.00     0.00 dimr      cluster_manager,data,ingest,remote_cluster_client *               node-1
Z.Z.Z.Z           21          90   0    0.00    0.00     0.00 dimr      cluster_manager,data,ingest,remote_cluster_client -               node-2
T.T.T.T           15          92   0    0.00    0.00     0.00 dimr      cluster_manager,data,ingest,remote_cluster_client -               node-7

[Deblintrake09]

T4 - Check that there are no errors in the browser's developer console when browsing the App. 🟡

Bugs reported previously:

• Multiple unexpected Errors during Wazuh app session wazuh-dashboard-plugins#5821
• Console error when I have an agent selected and environment change wazuh-dashboard-plugins#5745
• Multiple unexpected warning during Wazuh app session wazuh-dashboard-plugins#4092
• Disabled plugins shouldn't appear in More menu wazuh-dashboard-plugins#4074

[Deblintrake09]

T5 - Check that no warning symbols appear in the browser's developer console when browsing the App 🟢

After performing several tests both in Discover and in different modules, we have not been able to find any warning.

[Deblintrake09]

T6 - Alerts are being generated for each of the modules configured for this purpose

Modules in ENV 1 🟢

Alerts

1. The modules show events when they are activated.

   

2. GitHub and Office365 are not activated, so they have not generated alerts.
   Note: Amazon AWS is configured. I attach details.

   
   

Modules in ENV 2 🟡

Alerts

1. The modules show events except Docker Listener, VirusTotal, and System Auditing.

   

   

• Note: Expected behavior Does not show events in the 'Docker Listener', 'VirusTotal' and 'System Auditing' Modules when they are enabled wazuh-dashboard-plugins#5749

2. GitHub, Office365, Google Cloud Platform and Osquery are not activated, so they have not generated alerts.

   

   

[Deblintrake09]

T7: Generate an alert and check that this alert appears in the dashboard (end to end) 🟢

1. Generate alert

ssh fake-user@amazon-agent-ip
fake-user@amazon-agent-ip's password: 
dasdasdPermission denied, please try again.
fake-user@amazon-agent-ip's password: 
Permission denied, please try again.
fake-user@amazon-agent-ip's password: 
fake-user@amazon-agent-ip: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

2. Check if the alerts are in the Dashboard

   

Alert information

{
  "_index": "wazuh-alerts-4.x-env-2-2023.09.27",
  "_id": "o1ev2IoBhyp-9nnP8PlE",
  "_version": 1,
  "_score": null,
  "_source": {
    "predecoder": {
      "hostname": "ip-10-0-1-199",
      "program_name": "sshd",
      "timestamp": "Sep 27 22:08:39"
    },
    "cluster": {
      "node": "master",
      "name": "wazuh2"
    },
    "agent": {
      "ip": "10.0.1.199",
      "name": "Amazon",
      "id": "001"
    },
    "data": {
      "srcuser": "fake-user",
      "srcip": "45.178.1.193"
    },
    "manager": {
      "name": "wazuh-manager-master-0"
    },
    "rule": {
      "mail": false,
      "level": 5,
      "hipaa": [
        "164.312.b"
      ],
      "pci_dss": [
        "10.2.4",
        "10.2.5",
        "10.6.1"
      ],
      "tsc": [
        "CC6.1",
        "CC6.8",
        "CC7.2",
        "CC7.3"
      ],
      "description": "sshd: Attempt to login using a non-existent user",
      "groups": [
        "syslog",
        "sshd",
        "authentication_failed",
        "invalid_login"
      ],
      "nist_800_53": [
        "AU.14",
        "AC.7",
        "AU.6"
      ],
      "gdpr": [
        "IV_35.7.d",
        "IV_32.2"
      ],
      "firedtimes": 4,
      "mitre": {
        "technique": [
          "Password Guessing",
          "SSH"
        ],
        "id": [
          "T1110.001",
          "T1021.004"
        ],
        "tactic": [
          "Credential Access",
          "Lateral Movement"
        ]
      },
      "id": "5710",
      "gpg13": [
        "7.1"
      ]
    },
    "decoder": {
      "parent": "sshd",
      "name": "sshd"
    },
    "full_log": "Sep 27 22:08:39 ip-10-0-1-199 sshd[29313]: Failed password for invalid user fake-user from 45.178.1.193 port 11115 ssh2",
    "input": {
      "type": "log"
    },
    "location": "/var/log/secure",
    "id": "1695852521.169752175",
    "GeoLocation": {
      "city_name": "San Luis",
      "country_name": "Argentina",
      "region_name": "San Luis",
      "location": {
        "lon": -66.354,
        "lat": -33.299
      }
    },
    "timestamp": "2023-09-27T22:08:41.854+0000"
  },
  "fields": {
    "timestamp": [
      "2023-09-27T22:08:41.854Z"
    ]
  },
  "highlight": {
    "cluster.name": [
      "@opensearch-dashboards-highlighted-field@wazuh2@/opensearch-dashboards-highlighted-field@"
    ],
    "GeoLocation.city_name": [
      "@opensearch-dashboards-highlighted-field@San Luis@/opensearch-dashboards-highlighted-field@"
    ],
    "agent.name": [
      "@opensearch-dashboards-highlighted-field@Amazon@/opensearch-dashboards-highlighted-field@"
    ]
  },
  "sort": [
    1695852521854
  ]
}

[Deblintrake09]

T8: Search works without specifying a field and using * 🟢

• *
  

• aw*
  

• *squer*
  

• *shd
  

[fcaffieri]

Review notes

Some notes, great job

• Fix all the status of test, some inconsistencies, for example:

• Could you take a look at this error, we must create an issue?

• This is an expected behaviour

[fcaffieri]

LGTM

[davidcr01]

LGTM

[juliamagan]

LGTM