Release 4.6.0 - Alpha 1 - E2E UX tests - Demo environment #18843

davidjiglesias · 2023-09-07T10:21:14Z

End-to-End (E2E) Testing Guideline

Documentation: Always consult the development documentation for the current stage tag at this link. Be careful because some of the description steps might refer to a current version in production, always navigate using the current development documentation for the stage under test.
Test Requirements: Ensure your test comprehensively includes a full stack and agent/s deployment as per the Deployment requirements, detailing the machine OS, installed version, and revision.
Deployment Options: While deployments can be local (using VMs, Vagrant, or Docker) or on the aws-dev account, opt for local deployments when feasible. For AWS access, coordinate with the CICD team through this link.
External Accounts: If tests require third-party accounts (e.g., GitHub, Azure, AWS, GCP), request the necessary access through the CICD team here.
Alerts: Every test should generate a minimum of one end-to-end alert, from the agent to the dashboard, irrespective of test type.
Multi-node Testing: For multi-node wazuh-manager tests, ensure agents are connected to both workers and the master node.
Package Verification: Use the pre-release package that matches the current TAG you're testing. Confirm its version and revision.
Filebeat Errors: If you encounter errors with Filebeat during testing, refer to this Slack discussion for insights and resolutions.
Known Issues: Familiarize yourself with previously reported issues in the Known Issues section. This helps in identifying already recognized errors during testing.
Reporting New Issues: Any new errors discovered during testing that aren't listed under Known Issues should be reported. Communicate these to QA via the c-release Slack channel.
Test Conduct: It's imperative to be thorough in your testing, offering enough detail for reviewers. Incomplete tests might necessitate a redo.
Documentation Feedback: Encountering documentation gaps, unclear guidelines, or anything that disrupts the testing or UX? Open an issue, especially if it's not listed under Known Issues.
Format: If this is your first time doing this, refer to the format (but not necessarily the content, as it may vary) of previous E2E tests, here you have an example Release 4.3.5 - Release Candidate 1 - E2E UX tests - Wazuh Indexer #13994.
Status and completion: Change the issue status within your team project accordingly. Once you finish testing and write the conclusions, move it to Pending review and notify the @wazuh/cicd team via Slack using the c-release channel. Beware that the reviewers might request additional information or task repetitions.
For reviewers: Please move the issue to Pending final review and notify via Slack using the same thread if everything is ok, otherwise, perform an issue update with the requested changes and move it to On hold, increase the review_cycles in the team project by one and notify the issue assignee via Slack using the same thread.

For the conclusions and the issue testing and updates, use the following legend:

Status legend

🟢 All checks passed
🟡 Found a known issue
🔴 Found a new error

Deployment requirements

Component	Installation	Type	OS
Indexer
Server
Dashboard		-
Agent		-

Test description

Test demo.wazuh.info environment:

Check that there are no errors in the manager, agent, cluster, indexer, and dashboard logs.
Check that wazuh daemons are running with the expected user.
Check that the status of the indexer cluster is the expected.
Check that there are no errors in the browser's developer console when browsing the App.
Check that there are alerts for each of the modules configured.
Check that no warning symbols appear in the browser's developer console when browsing the App
Generate an alert and check that this alert appears in the dashboard (end to end)
Check that the search engine works without specifying a field and using *
The following tests could not be completed
- Generate an alert and check that this alert appears in the dashboard (end to end)

To access the demo environment, please contact @cicd-team.

Known issues

Conclusions

Summarize the errors detected (Known Issues included). Illustrate using the table below, removing current examples:

Status	Test	Failure type	Notes
🔴	Check that there are no errors in manager, agent, cluster, indexer and dashboard logs - Red Hat/Windows	Missing files	New issue opened: https://github.com/wazuh/wazuh-automation/issues/1284
🔴	Check that there are no errors in manager, agent, cluster, indexer and dashboard logs - Red Hat	Add audit rule	New issue opened: https://github.com/wazuh/wazuh-automation/issues/1285
🔴	Check that there are no errors in manager, agent, cluster, indexer and dashboard logs - Windows	Log category	New issue opened: #18957 (added)
🟡	Check that there are no errors in manager, agent, cluster, indexer and dashboard logs - Windows	Failed to move shared file	Known issue: #17087
🟡	Check that there are no errors in manager, agent, cluster, indexer and dashboard logs - Windows	Log file not found in inetpub	Known issue: https://github.com/wazuh/wazuh-automation/issues/802 (added) and #13253
🔴	Check that there are no errors in manager, agent, cluster, indexer and dashboard logs - Windows	Create Auditpol backup	New issue opened: #18959
🔴	Check that there are no errors in manager, agent, cluster, indexer and dashboard logs - Wazuh dashboard	Index already exist	New issue opened: https://github.com/wazuh/wazuh-automation/issues/1286
🔴	Check that there are no errors in manager, agent, cluster, indexer and dashboard logs - Wazuh dashboard	Response error in Wazuh dashboard	New issue opened: wazuh/wazuh-packages#2442 (added to qa_known)
🟡	Check that there are no errors in manager, agent, cluster, indexer and dashboard logs - Wazuh dashboard	ECONNREFUSED in Wazuh dashboard app	Known issue: wazuh/wazuh-dashboard-plugins#4133 (added)
🟡	Check that there are no errors in manager, agent, cluster, indexer and dashboard logs - Wazuh indexer	Default endpoint could not be created	Known issue: wazuh/wazuh-packages#1511 (comment) (added)
🟡	Check that there are no errors in manager, agent, cluster, indexer and dashboard logs - Wazuh indexer	Index not found for security config	Known issue: https://github.com/wazuh/wazuh-jenkins/issues/4513 (added)
🟡	Check that there are no errors in manager, agent, cluster, indexer and dashboard logs - Wazuh manager master	Connection error	Known issue: https://github.com/wazuh/wazuh-jenkins/issues/4867
🟡	Check that there are no errors in manager, agent, cluster, indexer and dashboard logs - Wazuh manager master	SUSE feed could not be fetched	Known issue: #18593 and wazuh/wazuh-qa#4370 (comment)
🟡	Check that there are no errors in the browser's developer console when browsing the App	Refused script	Known issue: wazuh/wazuh-dashboard-plugins#4121 (added)
🟡	Check that there are no errors in the browser's developer console when browsing the App	Script not firing due to content security policy	Known issue: wazuh/wazuh-dashboard-plugins#5821
🟡	Check that there are no errors in the browser's developer console when browsing the App	Detected an unhandled Promise rejection	Known issue: wazuh/wazuh-dashboard-plugins#5821
🟡	Check that there are no errors in the browser's developer console when browsing the App	Uncaught (in promise) TypeError: Cannot read properties of undefined	Known issue: wazuh/wazuh-dashboard-plugins#5332 (added)
🔴	Check that there are no errors in the browser's developer console when browsing the App	Missing indices	New issue opened: wazuh/wazuh-dashboard-plugins#5869
🔴	Check that there are no errors in the browser's developer console when browsing the App	401 error	New issue opened: wazuh/wazuh-dashboard-plugins#5871
🔴	Check that there are no errors in the browser's developer console when browsing the App	Password field is not contained in a form	New issue opened: wazuh/wazuh-dashboard-plugins#5873
🔴	Check that there are no errors in the browser's developer console when browsing the App	RBAC database	New issue opened: #18900
🔴	Check that there are alerts for each of the modules configured	AWS wodle fills the disk with alerts	New issue opened: #18901
🔴	Check that there are alerts for each of the modules configured	Possible segfault in wazuh-analysisd	New issue opened: #18946
🟢	Check that wazuh daemons are running with the expected user.	--	--
🟢	Check that the status of the indexer cluster is the expected.	--	--
🟢	Check that there are alerts for each of the modules configured.	--	--
🟢	Check that the search engine works without specifying a field and using *	--	--

Feedback

We value your feedback. Please provide insights on your testing experience.

Was the testing guideline clear? Were there any ambiguities?
- For this issue, it seemed clear to me
Did you face any challenges not covered by the guideline?
- Understand the composition of a demo environment and its characteristics
Suggestions for improvement:
- We should indicate the nodes of which a demo environment is made up and what each node has installed, in addition to clarifying that the demo environment already incorporates the use cases to carry out the PoC, for example, there are 3 Wazuh indexer nodes and one node of Wazuh dashboard, but the Wazuh dashboard node also includes a Wazuh indexer

Reviewers validation

The criteria for completing this task is based on the validation of the conclusions and the test results by all reviewers.

All the checkboxes below must be marked in order to close this issue.

@davidjiglesias
@wazuh/cicd

The text was updated successfully, but these errors were encountered:

rauldpm · 2023-09-07T12:18:43Z

Update report

Blocked: Waiting for the Demo environment to be deployed and the Release 4.6.0 - Alpha 1 - Demo use cases #18871 issue to be completed

Too many error messages are being found in the instances, this could be due to the use case tests that have been carried out on them, with this it is not possible to determine if an error is derived from the previous or new test, an example would be the following:

Red Hat Wazuh agent

[root@ip-10-0-1-46 wazuh-user]# grep -i -E "error|critical|fatal|warning" /var/ossec/logs/ossec.log
2023/09/07 13:42:05 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/suricata/eve.json' due to [(2)-(No such file or directory)].
2023/09/07 13:42:05 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/httpd/error_log' due to [(2)-(No such file or directory)].
2023/09/07 13:42:05 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/httpd/error_log'.
2023/09/07 13:42:05 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/httpd/access_log' due to [(2)-(No such file or directory)].
2023/09/07 13:42:07 wazuh-syscheckd: WARNING: (6925): Unable to add audit rule for '/etc/httpd'
2023/09/07 13:42:29 wazuh-syscheckd: WARNING: (6925): Unable to add audit rule for '/etc/httpd'
2023/09/07 13:42:29 wazuh-syscheckd: WARNING: (6925): Unable to add audit rule for '/var/lib/mysql'
2023/09/07 13:42:29 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/suricata/eve.json' due to [(2)-(No such file or directory)].
2023/09/07 13:42:29 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/httpd/error_log' due to [(2)-(No such file or directory)].
2023/09/07 13:42:29 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/httpd/error_log'.
2023/09/07 13:42:29 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/httpd/access_log' due to [(2)-(No such file or directory)].
2023/09/07 13:42:30 wazuh-modulesd:docker-listener: ERROR: 'docker' module needs to be installed. Execute 'pip3 install docker' to do it.
2023/09/07 13:42:30 wazuh-modulesd:docker-listener: WARNING: Docker-listener finished unexpectedly (code 1). Retrying to run in next scheduled time...
2023/09/07 13:42:43 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/suricata/eve.json' due to [(2)-(No such file or directory)].
2023/09/07 13:42:43 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/httpd/error_log' due to [(2)-(No such file or directory)].
2023/09/07 13:42:43 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/httpd/error_log'.
2023/09/07 13:42:43 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/httpd/access_log' due to [(2)-(No such file or directory)].
2023/09/07 13:42:44 wazuh-syscheckd: WARNING: (6925): Unable to add audit rule for '/etc/httpd'
2023/09/07 13:42:44 wazuh-syscheckd: WARNING: (6925): Unable to add audit rule for '/var/lib/mysql'
2023/09/07 13:42:44 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 1 sec.
2023/09/07 13:42:44 wazuh-modulesd:osquery: WARNING: The configuration file '/etc/osquery/osquery.conf' is not accessible: No such file or directory (2)
2023/09/07 13:42:44 wazuh-modulesd:docker-listener: ERROR: 'docker' module needs to be installed. Execute 'pip3 install docker' to do it.
2023/09/07 13:42:44 wazuh-modulesd:docker-listener: WARNING: Docker-listener finished unexpectedly (code 1). Retrying to run in next scheduled time...
2023/09/07 13:42:45 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 2 sec.
2023/09/07 13:42:47 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 3 sec.
2023/09/07 13:42:50 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 4 sec.
2023/09/07 13:42:54 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 5 sec.
2023/09/07 13:42:59 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 6 sec.
2023/09/07 13:43:05 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 7 sec.
2023/09/07 13:43:12 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 8 sec.
2023/09/07 13:43:20 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 9 sec.
2023/09/07 13:43:29 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 10 sec.
2023/09/07 13:43:39 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 11 sec.
2023/09/07 13:43:50 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 12 sec.
2023/09/07 13:44:02 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 13 sec.
2023/09/07 13:44:15 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 14 sec.
2023/09/07 13:44:29 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 15 sec.
2023/09/07 13:44:39 wazuh-agentd: ERROR: (1137): Lost connection with manager. Setting lock.
2023/09/07 13:44:39 wazuh-agentd: ERROR: (1216): Unable to connect to '[xxxxxx]:1514/tcp': 'Connection refused'.
2023/09/07 13:44:44 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 16 sec.
2023/09/07 13:44:45 wazuh-logcollector: WARNING: Process locked due to agent is offline. Waiting for connection...
2023/09/07 13:45:00 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 17 sec.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.
2023/09/07 13:45:17 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 18 sec.
2023/09/07 13:45:24 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/suricata/eve.json' due to [(2)-(No such file or directory)].
2023/09/07 13:45:24 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/httpd/error_log' due to [(2)-(No such file or directory)].
2023/09/07 13:45:24 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/httpd/error_log'.
2023/09/07 13:45:24 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/httpd/access_log' due to [(2)-(No such file or directory)].
2023/09/07 13:45:25 wazuh-syscheckd: WARNING: (6925): Unable to add audit rule for '/etc/httpd'
2023/09/07 13:45:25 wazuh-syscheckd: WARNING: (6925): Unable to add audit rule for '/var/lib/mysql'
2023/09/07 13:45:25 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 1 sec.
2023/09/07 13:45:25 wazuh-modulesd:osquery: WARNING: The configuration file '/etc/osquery/osquery.conf' is not accessible: No such file or directory (2)
2023/09/07 13:45:25 wazuh-modulesd:docker-listener: ERROR: 'docker' module needs to be installed. Execute 'pip3 install docker' to do it.
2023/09/07 13:45:25 wazuh-modulesd:docker-listener: WARNING: Docker-listener finished unexpectedly (code 1). Retrying to run in next scheduled time...
2023/09/07 13:45:26 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 2 sec.
2023/09/07 13:45:28 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 3 sec.
2023/09/07 13:45:31 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 4 sec.
2023/09/07 13:45:35 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 5 sec.
2023/09/07 13:45:40 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 6 sec.
2023/09/07 13:45:46 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 7 sec.
2023/09/07 13:45:53 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 8 sec.
2023/09/07 13:46:01 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 9 sec.
2023/09/07 13:46:10 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 10 sec.
2023/09/07 13:46:20 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 11 sec.
2023/09/07 13:46:31 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 12 sec.
2023/09/07 13:46:43 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 13 sec.
2023/09/07 13:46:56 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 14 sec.
2023/09/07 13:47:10 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 15 sec.
2023/09/07 13:47:25 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 16 sec.
2023/09/07 13:47:41 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 17 sec.
2023/09/07 13:47:58 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 18 sec.
2023/09/07 13:48:16 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 19 sec.
2023/09/07 13:48:19 wazuh-logcollector: WARNING: Target 'agent' message queue is full (1024). Log lines may be lost.
2023/09/07 13:48:21 wazuh-agentd: WARNING: Agent buffer at 90 %.
2023/09/07 13:48:35 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 20 sec.
2023/09/07 13:48:55 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 21 sec.
2023/09/07 13:49:16 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 22 sec.
2023/09/07 13:49:38 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 23 sec.
2023/09/07 13:50:01 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 24 sec.
2023/09/07 13:50:25 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 25 sec.
2023/09/07 13:50:50 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 26 sec.
2023/09/07 13:51:16 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 27 sec.
2023/09/07 13:51:43 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 28 sec.
2023/09/07 13:52:11 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 29 sec.
2023/09/07 13:52:15 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/httpd/error_log'.
2023/09/07 13:52:16 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 1 sec.
2023/09/07 14:03:12 wazuh-agentd: ERROR: (1137): Lost connection with manager. Setting lock.
2023/09/07 14:03:12 wazuh-agentd: ERROR: (1216): Unable to connect to '[xxxxx]:1514/tcp': 'Connection refused'.
2023/09/07 14:03:12 wazuh-modulesd: WARNING: Process locked due to agent is offline. Waiting for connection...
2023/09/07 14:03:17 wazuh-logcollector: WARNING: Process locked due to agent is offline. Waiting for connection...
2023/09/07 14:03:22 wazuh-agentd: ERROR: (1216): Unable to connect to '[xxxx]:1514/tcp': 'Connection refused'.
2023/09/07 14:05:02 wazuh-syscheckd: ERROR: in w_compress_gzfile(): fopen error /tmp/ansible_copy_payload_stj_xk0o/ansible_copy_payload.zip (2):'No such file or directory'
2023/09/07 14:05:02 wazuh-syscheckd: WARNING: (6914): Cannot create a snapshot of file '/tmp/ansible_copy_payload_stj_xk0o/ansible_copy_payload.zip'
2023/09/07 14:07:37 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/httpd/error_log'.
2023/09/07 14:13:21 wazuh-logcollector: INFO: (1950): Analyzing file: '/var/log/httpd/error_log'.

It is necessary that a new, unmodified environment be provided
After speaking with @teddytpc1, it was decided to continue since apparently, the demo deployment process makes modifications for the use cases, this makes it necessary to consider whether the current E2E test should be carried out on a new environment without said modifications.

rauldpm · 2023-09-07T20:35:50Z

Analysis report

Check that there are no errors in manager, agent, cluster, indexer and dashboard logs

Wazuh agent - Ubuntu 22.04.2 LTS

root@ip-10-0-1-190:/home/wazuh-user# /var/ossec/bin/wazuh-control info
WAZUH_VERSION="v4.6.0"
WAZUH_REVISION="40600"
WAZUH_TYPE="agent"

root@ip-10-0-1-190:/home/wazuh-user# grep -i -E "error|critical|fatal|warning" /var/ossec/logs/ossec.log 
2023/09/07 13:44:39 wazuh-agentd: ERROR: (1137): Lost connection with manager. Setting lock.
2023/09/07 13:44:39 wazuh-agentd: ERROR: (1216): Unable to connect to '[xxxxxx]:1514/tcp': 'Connection refused'.
2023/09/07 13:44:49 wazuh-agentd: ERROR: (1216): Unable to connect to '[xxxxxx]:1514/tcp': 'Connection refused'.

Connection errors can be expected due to some network connection problem, just like what happens in footprint tests https://github.com/wazuh/wazuh-jenkins/issues/4867

Wazuh agent - Red Hat 9

[root@ip-10-0-1-46 wazuh-user]# /var/ossec/bin/wazuh-control info
WAZUH_VERSION="v4.6.0"
WAZUH_REVISION="40600"
WAZUH_TYPE="agent"

Reported at: https://github.com/wazuh/wazuh-automation/issues/1284

2023/09/07 13:42:05 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/suricata/eve.json' due to [(2)-(No such file or directory)].
2023/09/07 13:42:05 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/httpd/error_log' due to [(2)-(No such file or directory)].
2023/09/07 13:42:05 wazuh-logcollector: ERROR: (1103): Could not open file '/var/log/httpd/access_log' due to [(2)-(No such file or directory)].
2023/09/07 13:42:44 wazuh-modulesd:osquery: WARNING: Results file '/var/log/osquery/osqueryd.results.log' not available: No such file or directory (2). Retrying in 1 sec.
2023/09/07 13:42:44 wazuh-modulesd:osquery: WARNING: The configuration file '/etc/osquery/osquery.conf' is not accessible: No such file or directory (2)
2023/09/07 13:42:30 wazuh-modulesd:docker-listener: ERROR: 'docker' module needs to be installed. Execute 'pip3 install docker' to do it.
2023/09/07 13:42:30 wazuh-modulesd:docker-listener: WARNING: Docker-listener finished unexpectedly (code 1). Retrying to run in next scheduled time...
2023/09/07 14:05:02 wazuh-syscheckd: ERROR: in w_compress_gzfile(): fopen error /tmp/ansible_copy_payload_stj_xk0o/ansible_copy_payload.zip (2):'No such file or directory'
2023/09/07 14:05:02 wazuh-syscheckd: WARNING: (6914): Cannot create a snapshot of file '/tmp/ansible_copy_payload_stj_xk0o/ansible_copy_payload.zip'

Reported at: https://github.com/wazuh/wazuh-automation/issues/1285

2023/09/07 13:42:07 wazuh-syscheckd: WARNING: (6925): Unable to add audit rule for '/etc/httpd'
2023/09/07 13:42:29 wazuh-syscheckd: WARNING: (6925): Unable to add audit rule for '/var/lib/mysql'
2023/09/07 13:45:14 wazuh-syscheckd: WARNING: (6911): Detected Audit rules manipulation: Audit rules removed.

Wazuh agent - CentOS 8

[root@ip-10-0-1-191 wazuh-user]# /var/ossec/bin/wazuh-control info
WAZUH_VERSION="v4.6.0"
WAZUH_REVISION="40600"
WAZUH_TYPE="agent"

[root@ip-10-0-1-191 wazuh-user]# grep -i -E "error|critical|fatal|warning" /var/ossec/logs/ossec.log 
[root@ip-10-0-1-191 wazuh-user]# grep -i -E "error|critical|fatal|warning" /var/ossec/logs/ossec.log  | wc -l
0

Wazuh agent - Windows Server 2019 Datacenter

Reported at Message classified as INFO informs of an error when not finding a DLL file of the agent #18957

2023/09/07 13:36:56 wazuh-agent: INFO: Trust verification of a module failed by using the hash method. CryptCATAdminEnumCatalogFromHash failed with error 1168 for 'C:\Program Files (x86)\ossec-agent\libfimdb.dll': Element not found.
2023/09/07 13:36:56 wazuh-agent: WARNING: The file 'C:\Program Files (x86)\ossec-agent\libfimdb.dll' is not signed or its signature is invalid.

Reported at Windows agent throws an error message when receiving a file through the centralized configuration due to a lack of permissions #17087 (added to qa_known)

2023/09/07 13:37:17 wazuh-agent: ERROR: Could not move (shared/win_applications_rcl.txta02468) to (shared/win_applications_rcl.txt) which returned (5)

Related to https://github.com/wazuh/wazuh-automation/issues/1284 (files exist in Windows instance)

2023/09/07 13:43:01 wazuh-modulesd:osquery: WARNING: Results file 'C:\Program Files\osquery\log\osqueryd.results.log' not available: No error (0). Retrying in 1 sec.
2023/09/07 13:43:01 wazuh-modulesd:osquery: WARNING: The configuration file 'C:\Program Files\osquery\osquery.conf' is not accessible: No such file or directory (2)
2023/09/07 13:44:34 wazuh-modulesd:osquery: WARNING: Results file 'C:\Program Files\osquery\log\osqueryd.results.log' not available: Bad file descriptor (9). Retrying in 12 sec.

Related to https://github.com/wazuh/wazuh-automation/issues/802 (added to qa_known) and Logcollector attempts messages when a file doesn't exist should be info until the last attempt #13253

2023/09/07 13:45:22 wazuh-agent: ERROR: (1103): Could not open file 'C:\inetpub\logs\LogFiles\W3SVC1\u_ex230907.log' due to [(3)-(The system cannot find the path specified.)].

Reported at Error message when performing Auditpol backup #18959

2023/09/07 14:02:28 wazuh-agent: ERROR: (6635): Auditpol backup error: 'time overtaken while running the command'.

Wazuh agent - Debian 11

root@ip-10-0-1-185:/home/wazuh-user# /var/ossec/bin/wazuh-control info
WAZUH_VERSION="v4.6.0"
WAZUH_REVISION="40600"
WAZUH_TYPE="agent"

Expected connection error

root@ip-10-0-1-185:/home/wazuh-user# grep -i -E "error|critical|fatal|warning" /var/ossec/logs/ossec.log 
2023/09/07 14:03:12 wazuh-agentd: ERROR: (1137): Lost connection with manager. Setting lock.
root@ip-10-0-1-185:/home/wazuh-user# grep -i -E "error|critical|fatal|warning" /var/ossec/logs/ossec.log |wc -l
1
root@ip-10-0-1-185:/home/wazuh-user#

Wazuh agent - Amazon Linux 2

[root@ip-10-0-1-231 wazuh-user]# /var/ossec/bin/wazuh-control info
WAZUH_VERSION="v4.6.0"
WAZUH_REVISION="40600"
WAZUH_TYPE="agent"

Expected connection error

[root@ip-10-0-1-231 wazuh-user]# grep -i -E "error|critical|fatal|warning" /var/ossec/logs/ossec.log 
2023/09/07 13:44:39 wazuh-agentd: ERROR: (1137): Lost connection with manager. Setting lock.
2023/09/07 13:44:39 wazuh-agentd: ERROR: (1216): Unable to connect to '[xxxx]:1514/tcp': 'Connection refused'.
2023/09/07 13:44:49 wazuh-agentd: ERROR: (1216): Unable to connect to '[xxxx]:1514/tcp': 'Connection refused'.
[root@ip-10-0-1-231 wazuh-user]# grep -i -E "error|critical|fatal|warning" /var/ossec/logs/ossec.log  | wc -l
3

Wazuh dashboard - Amazon Linux 2

[root@ip-10-0-0-232 wazuh-user]# cat /usr/share/wazuh-dashboard/VERSION 
4.6.0

journalctl

[root@ip-10-0-0-232 wazuh-user]# journalctl -r -u wazuh-dashboard.service  | grep -i -E "error|critical|warning|fatal"
Sep 07 13:45:01 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[20957]: {"type":"log","@timestamp":"2023-09-07T13:45:01Z","tags":["error","opensearch","data"],"pid":20957,"message":"[resource_already_exists_exception]: index [wazuh-statistics-2023.36w/EN8sLU9HTtu5M6krwLHxTg] already exists"}
Sep 07 13:41:16 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[20957]: {"type":"log","@timestamp":"2023-09-07T13:41:16Z","tags":["warning","config","deprecation"],"pid":20957,"message":"\"opensearch.requestHeadersWhitelist\" is deprecated and has been replaced by \"opensearch.requestHeadersAllowlist\""}
Sep 07 13:22:18 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[17233]: {"type":"log","@timestamp":"2023-09-07T13:22:18Z","tags":["error","opensearch","data"],"pid":17233,"message":"[ResponseError]: Response Error"}

warning deprecation message related to OpenSearch Dashboard
resource already exist error reported at https://github.com/wazuh/wazuh-automation/issues/1286
ResponseError: ResponseError messages found in the Wazuh dashboard logs wazuh-packages#2442

wazuhapp.log

[root@ip-10-0-0-232 wazuh-user]# grep -i -E "error|critical|fatal|warning" /usr/share/wazuh-dashboard/data/wazuh/logs/wazuhapp.log 
{"date":"2023-09-07T13:22:19.627Z","level":"error","location":"monitoring:getApiInfo","message":"connect ECONNREFUSED xx.xx.xx.xx:55000"}

Related to Error logs found in wazuh-dashboard wazuh-dashboard-plugins#4133 (added to qa_known)

systemctl

[root@ip-10-0-0-232 wazuh-user]# systemctl status wazuh-dashboard.service -l
● wazuh-dashboard.service - wazuh-dashboard
Loaded: loaded (/etc/systemd/system/wazuh-dashboard.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2023-09-07 13:41:11 UTC; 6h ago
Main PID: 20957 (node)
CGroup: /system.slice/wazuh-dashboard.service
        └─20957 /usr/share/wazuh-dashboard/node/bin/node --no-warnings --max-http-header-size=65536 --unhandled-rejections=warn /usr/share/wazuh-dashboard/src/cli/dist -c /etc/wazuh-dashboard/opensearch_dashboards.yml

Sep 07 18:40:38 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[20957]: {"type":"response","@timestamp":"2023-09-07T18:40:38Z","tags":[],"pid":20957,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"xx.xx.xx.xx:5601","connection":"close","user-agent":"Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers&#39; presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: [email protected]"},"remoteAddress":"xx.xx.xx.xx","userAgent":"Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers&#39; presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: [email protected]"},"res":{"statusCode":302,"responseTime":8,"contentLength":9},"message":"GET / 302 8ms - 9.0B"}
Sep 07 18:40:39 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[20957]: {"type":"response","@timestamp":"2023-09-07T18:40:39Z","tags":[],"pid":20957,"method":"get","statusCode":200,"req":{"url":"/app/login","method":"get","headers":{"host":"xx.xx.xx.xx:5601","connection":"close","user-agent":"Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers&#39; presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: [email protected]"},"remoteAddress":"xx.xx.xx.xx","userAgent":"Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers&#39; presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: [email protected]"},"res":{"statusCode":200,"responseTime":19,"contentLength":9},"message":"GET /app/login 200 19ms - 9.0B"}
Sep 07 19:47:45 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[20957]: {"type":"response","@timestamp":"2023-09-07T19:47:45Z","tags":[],"pid":20957,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"xx.xx.xx.xx:5601","connection":"close"},"remoteAddress":"xx.xx.xx.xx"},"res":{"statusCode":302,"responseTime":10,"contentLength":9},"message":"GET / 302 10ms - 9.0B"}
Sep 07 19:47:46 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[20957]: {"type":"response","@timestamp":"2023-09-07T19:47:45Z","tags":[],"pid":20957,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"xx.xx.xx.xx:5601","connection":"close","user-agent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)","accept":"*/*","accept-encoding":"gzip"},"remoteAddress":"xx.xx.xx.xx","userAgent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)"},"res":{"statusCode":302,"responseTime":1,"contentLength":9},"message":"GET / 302 1ms - 9.0B"}
Sep 07 19:47:46 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[20957]: {"type":"response","@timestamp":"2023-09-07T19:47:46Z","tags":[],"pid":20957,"method":"get","statusCode":200,"req":{"url":"/app/login","method":"get","headers":{"host":"xx.xx.xx.xx:5601","connection":"close","user-agent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)","accept":"*/*","accept-encoding":"gzip"},"remoteAddress":"xx.xx.xx.xx","userAgent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)"},"res":{"statusCode":200,"responseTime":37,"contentLength":9},"message":"GET /app/login 200 37ms - 9.0B"}
Sep 07 19:47:47 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[20957]: {"type":"response","@timestamp":"2023-09-07T19:47:47Z","tags":[],"pid":20957,"method":"get","statusCode":200,"req":{"url":"/ui/favicons/apple-touch-icon.png","method":"get","headers":{"host":"xx.xx.xx.xx:5601","connection":"close","user-agent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)","accept-encoding":"gzip"},"remoteAddress":"xx.xx.xx.xx","userAgent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)"},"res":{"statusCode":200,"responseTime":3,"contentLength":9},"message":"GET /ui/favicons/apple-touch-icon.png 200 3ms - 9.0B"}
Sep 07 19:47:47 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[20957]: {"type":"response","@timestamp":"2023-09-07T19:47:47Z","tags":[],"pid":20957,"method":"get","statusCode":200,"req":{"url":"/ui/favicons/favicon-32x32.png","method":"get","headers":{"host":"xx.xx.xx.xx:5601","connection":"close","user-agent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)","accept-encoding":"gzip"},"remoteAddress":"xx.xx.xx.xx","userAgent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)"},"res":{"statusCode":200,"responseTime":5,"contentLength":9},"message":"GET /ui/favicons/favicon-32x32.png 200 5ms - 9.0B"}
Sep 07 19:47:48 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[20957]: {"type":"response","@timestamp":"2023-09-07T19:47:48Z","tags":[],"pid":20957,"method":"get","statusCode":200,"req":{"url":"/ui/favicons/favicon-16x16.png","method":"get","headers":{"host":"xx.xx.xx.xx:5601","connection":"close","user-agent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)","accept-encoding":"gzip"},"remoteAddress":"xx.xx.xx.xx","userAgent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)"},"res":{"statusCode":200,"responseTime":2,"contentLength":9},"message":"GET /ui/favicons/favicon-16x16.png 200 2ms - 9.0B"}
Sep 07 19:47:48 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[20957]: {"type":"response","@timestamp":"2023-09-07T19:47:48Z","tags":[],"pid":20957,"method":"get","statusCode":200,"req":{"url":"/ui/favicons/favicon.ico","method":"get","headers":{"host":"xx.xx.xx.xx:5601","connection":"close","user-agent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)","accept-encoding":"gzip"},"remoteAddress":"xx.xx.xx.xx","userAgent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)"},"res":{"statusCode":200,"responseTime":3,"contentLength":9},"message":"GET /ui/favicons/favicon.ico 200 3ms - 9.0B"}
Sep 07 19:47:48 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[20957]: {"type":"response","@timestamp":"2023-09-07T19:47:48Z","tags":[],"pid":20957,"method":"get","statusCode":401,"req":{"url":"/favicon.ico","method":"get","headers":{"host":"xx.xx.xx.xx:5601","connection":"close","user-agent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)","accept-encoding":"gzip"},"remoteAddress":"xx.xx.xx.xx","userAgent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)"},"res":{"statusCode":401,"responseTime":3,"contentLength":9},"message":"GET /favicon.ico 401 3ms - 9.0B"}

Amazon Linux 2 - Wazuh indexer - node3

[root@ip-10-0-2-13 wazuh-user]# cat /usr/share/wazuh-indexer/VERSION 
4.6.0

journalctl

Related to OpenSearch deprecation

[root@ip-10-0-2-13 wazuh-user]# journalctl -r -u wazuh-indexer.service | grep -i -E "error|critical|fatal|warning"
Sep 07 13:15:46 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:15:46 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Sep 07 13:15:46 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:15:46 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:15:44 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:15:44 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Sep 07 13:15:44 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:15:44 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:14:12 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[14441]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:14:12 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[14441]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Sep 07 13:14:12 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[14441]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:14:12 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[14441]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:14:10 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[14441]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:14:10 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[14441]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Sep 07 13:14:10 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[14441]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:14:10 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[14441]: WARNING: A terminally deprecated method in java.lang.System has been called
[root@ip-10-0-2-13 wazuh-user]#

systemctl

[root@ip-10-0-2-13 wazuh-user]# systemctl status wazuh-indexer -l
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2023-09-07 13:16:04 UTC; 6h ago
Docs: https://documentation.wazuh.com
Main PID: 15794 (java)
CGroup: /system.slice/wazuh-indexer.service
        └─15794 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms3948m -Xmx3948m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-14948807468172416516 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=2069889024 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

Sep 07 13:15:41 ip-10-0-2-13.us-west-1.compute.internal systemd[1]: Starting Wazuh-indexer...
Sep 07 13:15:44 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:15:44 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:15:44 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Sep 07 13:15:44 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:15:46 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:15:46 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:15:46 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Sep 07 13:15:46 ip-10-0-2-13.us-west-1.compute.internal systemd-entrypoint[15794]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:16:04 ip-10-0-2-13.us-west-1.compute.internal systemd[1]: Started Wazuh-indexer.

/var/log/

Related to: Warning and error logs found in wazuh-indexer in demo environment wazuh-packages#1511 (comment)

/var/log/wazuh-indexer/wazuh.log:[2023-09-07T13:14:23,977][ERROR][o.o.s.a.s.SinkProvider   ] [node-3] Default endpoint could not be created, auditlog will not work properly.

Related to: https://github.com/wazuh/wazuh-jenkins/issues/4513 (added to qa_known)

/var/log/wazuh-indexer/wazuh.log:[2023-09-07T13:14:30,818][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [node-3] Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)

Related to OpenSearch

/var/log/wazuh-indexer/wazuh.log:[2023-09-07T13:16:06,719][ERROR][o.o.s.a.m.AsynchronousSearchManagementService] [node-3] Exception executing action [indices:data/read/opendistro/asynchronous_search/response_cleanup]

Amazon Linuz 2 - Wazuh indexer - node2

[root@ip-10-0-2-117 wazuh-user]# cat /usr/share/wazuh-indexer/VERSION 
4.6.0

journalctl

Related to OpenSearch deprecation

[root@ip-10-0-2-117 wazuh-user]# journalctl -r -u wazuh-indexer.service | grep -i -E "error|critical|fatal|warning"
Sep 07 13:16:11 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:16:11 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Sep 07 13:16:11 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:16:11 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:16:09 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:16:09 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Sep 07 13:16:09 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:16:09 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:14:08 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[14435]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:14:08 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[14435]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Sep 07 13:14:08 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[14435]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:14:08 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[14435]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:14:06 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[14435]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:14:06 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[14435]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Sep 07 13:14:06 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[14435]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:14:06 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[14435]: WARNING: A terminally deprecated method in java.lang.System has been called

systemctl

[root@ip-10-0-2-117 wazuh-user]# systemctl status wazuh-indexer -l
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2023-09-07 13:16:28 UTC; 6h ago
Docs: https://documentation.wazuh.com
Main PID: 15764 (java)
CGroup: /system.slice/wazuh-indexer.service
        └─15764 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms3948m -Xmx3948m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-1755572708142381348 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=2069889024 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

Sep 07 13:16:06 ip-10-0-2-117.us-west-1.compute.internal systemd[1]: Starting Wazuh-indexer...
Sep 07 13:16:09 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:16:09 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:16:09 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Sep 07 13:16:09 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:16:11 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:16:11 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:16:11 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Sep 07 13:16:11 ip-10-0-2-117.us-west-1.compute.internal systemd-entrypoint[15764]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:16:28 ip-10-0-2-117.us-west-1.compute.internal systemd[1]: Started Wazuh-indexer.

/var/log/

Related to: Warning and error logs found in wazuh-indexer in demo environment wazuh-packages#1511 (comment)

/var/log/wazuh-indexer/wazuh.log:[2023-09-07T13:14:19,689][ERROR][o.o.s.a.s.SinkProvider   ] [node-2] Default endpoint could not be created, auditlog will not work properly.

Related to: https://github.com/wazuh/wazuh-jenkins/issues/4513 (added to qa_known)

/var/log/wazuh-indexer/wazuh.log:[2023-09-07T13:14:27,125][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [node-2] Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)

Related to OpenSearch

/var/log/wazuh-indexer/wazuh_server.json:{"type": "server", "timestamp": "2023-09-07T13:16:22,042Z", "level": "ERROR", "component": "o.o.s.a.s.SinkProvider", "cluster.name": "wazuh", "node.name": "node-2", "message": "Default endpoint could not be created, auditlog will not work properly." }

Amazon Linux 2 - Wazuh indexer - node1

[root@ip-10-0-2-239 wazuh-user]#  cat /usr/share/wazuh-indexer/VERSION
4.6.0

journalctl

Related to OpenSearch deprecation

[root@ip-10-0-2-239 wazuh-user]# journalctl -r -u wazuh-indexer.service | grep -i -E "error|critical|fatal|warning"
Sep 07 13:15:19 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:15:19 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Sep 07 13:15:19 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:15:19 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:15:17 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:15:17 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Sep 07 13:15:17 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:15:17 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:14:02 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[14567]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:14:02 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[14567]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Sep 07 13:14:02 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[14567]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:14:02 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[14567]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:14:00 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[14567]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:14:00 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[14567]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Sep 07 13:14:00 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[14567]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:14:00 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[14567]: WARNING: A terminally deprecated method in java.lang.System has been called

systemctl

[root@ip-10-0-2-239 wazuh-user]# systemctl status wazuh-indexer -l
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2023-09-07 13:15:36 UTC; 6h ago
Docs: https://documentation.wazuh.com
Main PID: 15829 (java)
CGroup: /system.slice/wazuh-indexer.service
        └─15829 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms3948m -Xmx3948m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-999654823370953880 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=2069889024 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

Sep 07 13:15:14 ip-10-0-2-239.us-west-1.compute.internal systemd[1]: Starting Wazuh-indexer...
Sep 07 13:15:17 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:15:17 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:15:17 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Sep 07 13:15:17 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:15:19 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: A terminally deprecated method in java.lang.System has been called
Sep 07 13:15:19 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.8.0.jar)
Sep 07 13:15:19 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Sep 07 13:15:19 ip-10-0-2-239.us-west-1.compute.internal systemd-entrypoint[15829]: WARNING: System::setSecurityManager will be removed in a future release
Sep 07 13:15:36 ip-10-0-2-239.us-west-1.compute.internal systemd[1]: Started Wazuh-indexer.

/var/log/

Related to: Warning and error logs found in wazuh-indexer in demo environment wazuh-packages#1511 (comment)

/var/log/wazuh-indexer/wazuh.log:[2023-09-07T13:14:12,706][ERROR][o.o.s.a.s.SinkProvider   ] [node-1] Default endpoint could not be created, auditlog will not work properly.

Related to: https://github.com/wazuh/wazuh-jenkins/issues/4513 (added to qa_known)

/var/log/wazuh-indexer/wazuh.log:[2023-09-07T13:14:27,267][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [node-1] Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, ALLOWLIST, AUDIT] (index=.opendistro_security)

Related to OpenSearch

/var/log/wazuh-indexer/wazuh_server.json:{"type": "server", "timestamp": "2023-09-07T13:15:30,210Z", "level": "ERROR", "component": "o.o.s.a.s.SinkProvider", "cluster.name": "wazuh", "node.name": "node-1", "message": "Default endpoint could not be created, auditlog will not work properly." }

Wazuh server - master - env1

[root@wazuh-manager-master-0 wazuh-user]# /var/ossec/bin/wazuh-control info
WAZUH_VERSION="v4.6.0"
WAZUH_REVISION="40600"
WAZUH_TYPE="server"

ossec.log

Related (same principle) https://github.com/wazuh/wazuh-jenkins/issues/4867

/var/ossec/logs/ossec.log:2023/09/07 13:37:19 wazuh-remoted: WARNING: Agent key already in use: agent ID '004'

Related to SUSE OVAL feeds download are too slow #18593 and Tests fail during feed downloads and others finished with setup errors in the VD suite wazuh-qa#4370 (comment)

/var/ossec/logs/ossec.log:2023/09/07 14:19:16 wazuh-modulesd:vulnerability-detector: WARNING: (5500): The 'SUSE Linux Enterprise Server 15' database could not be fetched.
/var/ossec/logs/ossec.log:2023/09/07 14:49:26 wazuh-modulesd:vulnerability-detector: WARNING: (5500): The 'SUSE Linux Enterprise Desktop 15' database could not be fetched.
/var/ossec/logs/ossec.log:2023/09/07 15:19:36 wazuh-modulesd:vulnerability-detector: WARNING: (5500): The 'SUSE Linux Enterprise Server 12' database could not be fetched.
/var/ossec/logs/ossec.log:2023/09/07 15:54:08 wazuh-modulesd:vulnerability-detector: WARNING: (5500): The 'SUSE Linux Enterprise Server 11' database could not be fetched.
/var/ossec/logs/ossec.log:2023/09/07 16:00:44 wazuh-modulesd:vulnerability-detector: ERROR: (5513): CVE database could not be updated.

Wazuh server - master - env2

[root@wazuh-manager-master-0 wazuh-user]# /var/ossec/bin/wazuh-control info
WAZUH_VERSION="v4.6.0"
WAZUH_REVISION="40600"
WAZUH_TYPE="server"

ossec.log

Related to SUSE OVAL feeds download are too slow #18593 and Tests fail during feed downloads and others finished with setup errors in the VD suite wazuh-qa#4370 (comment)

/var/ossec/logs/ossec.log:2023/09/07 14:19:10 wazuh-modulesd:vulnerability-detector: WARNING: (5500): The 'SUSE Linux Enterprise Server 15' database could not be fetched.
/var/ossec/logs/ossec.log:2023/09/07 14:49:20 wazuh-modulesd:vulnerability-detector: WARNING: (5500): The 'SUSE Linux Enterprise Desktop 15' database could not be fetched.
/var/ossec/logs/ossec.log:2023/09/07 15:19:30 wazuh-modulesd:vulnerability-detector: WARNING: (5500): The 'SUSE Linux Enterprise Server 12' database could not be fetched.
/var/ossec/logs/ossec.log:2023/09/07 15:53:56 wazuh-modulesd:vulnerability-detector: WARNING: (5500): The 'SUSE Linux Enterprise Server 11' database could not be fetched.
/var/ossec/logs/ossec.log:2023/09/07 16:00:07 wazuh-modulesd:vulnerability-detector: ERROR: (5513): CVE database could not be updated.

Wazuh server - worker - env1

[root@wazuh-manager-worker-0 wazuh-user]# /var/ossec/bin/wazuh-control info
WAZUH_VERSION="v4.6.0"
WAZUH_REVISION="40600"
WAZUH_TYPE="server"

ossec.log

Connection issues related to network, found only one message of each one, worker reconnect successfully

/var/ossec/logs/cluster.log:2023/09/07 13:29:21 ERROR: [Worker worker_01] [Main] Error sending sendsync response to local client: Error 3020 - Timeout sending request: ok
/var/ossec/logs/cluster.log:2023/09/07 13:29:24 INFO: [Worker worker_01] [Agent-info sync] Starting.
/var/ossec/logs/cluster.log:2023/09/07 13:29:24 INFO: [Worker worker_01] [Agent-info sync] Finished in 0.003s. Updated 0 chunks.
/var/ossec/logs/cluster.log:2023/09/07 13:44:49 ERROR: [Local Server] [Main] Could not connect to master. Trying again in 10 seconds.
/var/ossec/logs/cluster.log:2023/09/07 13:44:59 INFO: [Worker worker_01] [Main] Successfully connected to master.

Related (same principle) https://github.com/wazuh/wazuh-jenkins/issues/4867

/var/ossec/logs/ossec.log:2023/09/07 13:34:19 wazuh-remoted: WARNING: (1408): Invalid ID 002 for the source ip: 'xx.xx.xx.xx' (name 'unknown').
/var/ossec/logs/ossec.log:2023/09/07 13:35:11 wazuh-remoted: WARNING: (1408): Invalid ID 003 for the source ip: 'xx.xx.xx.xx' (name 'unknown').
/var/ossec/logs/ossec.log:2023/09/07 13:40:32 wazuh-remoted: WARNING: Agent key already in use: agent ID '004'

Related to SUSE OVAL feeds download are too slow #18593 and Tests fail during feed downloads and others finished with setup errors in the VD suite wazuh-qa#4370 (comment)

/var/ossec/logs/ossec.log:2023/09/07 14:37:40 wazuh-modulesd:vulnerability-detector: WARNING: (5500): The 'SUSE Linux Enterprise Server 15' database could not be fetched.
/var/ossec/logs/ossec.log:2023/09/07 15:07:50 wazuh-modulesd:vulnerability-detector: WARNING: (5500): The 'SUSE Linux Enterprise Desktop 15' database could not be fetched.
/var/ossec/logs/ossec.log:2023/09/07 15:38:00 wazuh-modulesd:vulnerability-detector: WARNING: (5500): The 'SUSE Linux Enterprise Server 12' database could not be fetched.
/var/ossec/logs/ossec.log:2023/09/07 16:12:29 wazuh-modulesd:vulnerability-detector: WARNING: (5500): The 'SUSE Linux Enterprise Server 11' database could not be fetched.
/var/ossec/logs/ossec.log:2023/09/07 16:19:48 wazuh-modulesd:vulnerability-detector: ERROR: (5513): CVE database could not be updated.

rauldpm · 2023-09-08T14:23:48Z

Analysis report

Check that wazuh daemons are running with the expected user.

Wazuh agent - Ubuntu 22.04.2 LTS

[root@ip-10-0-1-46 wazuh-user]# ps -ef | grep wazuh | grep -v grep | grep -v 'wazuh-u'
root       59484       1  0 Sep07 ?        00:00:01 /var/ossec/bin/wazuh-execd
wazuh      59496       1  0 Sep07 ?        00:00:27 /var/ossec/bin/wazuh-agentd
root       59511       1  0 Sep07 ?        00:00:48 /var/ossec/bin/wazuh-syscheckd
root       59525       1  0 Sep07 ?        00:00:14 /var/ossec/bin/wazuh-logcollector
root       59548       1  0 Sep07 ?        00:00:21 /var/ossec/bin/wazuh-modulesd

Wazuh agent - Red Hat 9

[root@ip-10-0-1-46 wazuh-user]# ps -ef | grep wazuh | grep -v grep | grep -v 'wazuh-u'
root       59484       1  0 Sep07 ?        00:00:01 /var/ossec/bin/wazuh-execd
wazuh      59496       1  0 Sep07 ?        00:00:27 /var/ossec/bin/wazuh-agentd
root       59511       1  0 Sep07 ?        00:00:48 /var/ossec/bin/wazuh-syscheckd
root       59525       1  0 Sep07 ?        00:00:14 /var/ossec/bin/wazuh-logcollector
root       59548       1  0 Sep07 ?        00:00:21 /var/ossec/bin/wazuh-modulesd

Wazuh agent - CentOS 8

[root@ip-10-0-1-191 wazuh-user]# ps -ef | grep wazuh | grep -v grep | grep -v 'wazuh-u'
root        9283       1  0 Sep07 ?        00:00:02 /var/ossec/bin/wazuh-execd
wazuh       9295       1  0 Sep07 ?        00:00:14 /var/ossec/bin/wazuh-agentd
root        9310       1  0 Sep07 ?        00:00:43 /var/ossec/bin/wazuh-syscheckd
root        9325       1  0 Sep07 ?        00:00:08 /var/ossec/bin/wazuh-logcollector
root        9344       1  0 Sep07 ?        00:00:06 /var/ossec/bin/wazuh-modulesd

Wazuh agent - Windows Server 2019 Datacenter

PS C:\Users\Administrator> tasklist /v | Select-String "wazuh"

wazuh-agent.exe               1604 Services                   0     35,400 K Unknown         NT AUTHORITY\SYSTEM                                     0:04:39 N/A
win32ui.exe                   2484 RDP-Tcp#51                 2     12,104 K Running         EC2AMAZ-SQTHV5K\Administrator                           0:00:00 Wazuh Agent Manager

Wazuh agent - Debian 11

root@ip-10-0-1-185:/home/wazuh-user# ps -ef | grep wazuh | grep -v grep | grep -v 'wazuh-u'
root        8206       1  0 Sep07 ?        00:00:03 /var/ossec/bin/wazuh-execd
wazuh       8217       1  0 Sep07 ?        00:00:19 /var/ossec/bin/wazuh-agentd
root        8231       1  0 Sep07 ?        00:00:36 /var/ossec/bin/wazuh-syscheckd
root        8246       1  0 Sep07 ?        00:00:13 /var/ossec/bin/wazuh-logcollector
root        8265       1  0 Sep07 ?        00:00:07 /var/ossec/bin/wazuh-modulesd

Wazuh agent - Amazon Linux 2

[root@ip-10-0-1-231 wazuh-user]# ps -ef | grep wazuh | grep -v grep | grep -v 'wazuh-u'
root     15800     1  0 Sep07 ?        00:00:02 /var/ossec/bin/wazuh-execd
wazuh    15812     1  0 Sep07 ?        00:00:12 /var/ossec/bin/wazuh-agentd
root     15827     1  0 Sep07 ?        00:00:36 /var/ossec/bin/wazuh-syscheckd
root     15842     1  0 Sep07 ?        00:00:07 /var/ossec/bin/wazuh-logcollector
root     15860     1  0 Sep07 ?        00:00:05 /var/ossec/bin/wazuh-modulesd

Wazuh dashboard - Amazon Linux 2

[root@ip-10-0-0-232 wazuh-user]# ps -ef | grep -v grep | grep -v wazuh-u | grep wazuh
wazuh-i+ 17334     1  3 Sep07 ?        00:53:13 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms2560m -Xmx2560m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-9871495143653250211 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=1342177280 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet
wazuh-d+ 20957     1  0 Sep07 ?        00:05:45 /usr/share/wazuh-dashboard/node/bin/node --no-warnings --max-http-header-size=65536 --unhandled-rejections=warn /usr/share/wazuh-dashboard/src/cli/dist -c /etc/wazuh-dashboard/opensearch_dashboards.yml

Amazon Linux 2 - Wazuh indexer - node3

[root@ip-10-0-2-13 wazuh-user]# ps -ef | grep -v grep | grep wazuh | grep -v wazuh-u
wazuh-i+ 15794     1  5 Sep07 ?        01:21:20 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms3948m -Xmx3948m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-14948807468172416516 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=2069889024 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

Amazon Linuz 2 - Wazuh indexer - node2

[root@ip-10-0-2-117 wazuh-user]# ps -ef | grep -v grep | grep wazuh | grep -v wazuh-u
wazuh-i+ 15764     1  5 Sep07 ?        01:16:12 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms3948m -Xmx3948m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-1755572708142381348 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=2069889024 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

Amazon Linux 2 - Wazuh indexer - node1

[root@ip-10-0-2-239 wazuh-user]# ps -ef | grep -v grep | grep wazuh | grep -v wazuh-u
wazuh-i+ 15829     1  5 Sep07 ?        01:15:26 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms3948m -Xmx3948m -XX:+UseG1GC -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -Djava.io.tmpdir=/tmp/opensearch-999654823370953880 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Dclk.tck=100 -Djdk.attach.allowAttachSelf=true -Djava.security.policy=file:///usr/share/wazuh-indexer/plugins/opensearch-performance-analyzer/pa_config/opensearch_security.policy -XX:MaxDirectMemorySize=2069889024 -Dopensearch.path.home=/usr/share/wazuh-indexer -Dopensearch.path.conf=/etc/wazuh-indexer -Dopensearch.distribution.type=rpm -Dopensearch.bundled_jdk=true -cp /usr/share/wazuh-indexer/lib/* org.opensearch.bootstrap.OpenSearch -p /run/wazuh-indexer/wazuh-indexer.pid --quiet

Wazuh server - master - env1

[root@wazuh-manager-master-0 wazuh-user]# ps -ef | grep -v grep | grep -v wazuh-u | grep  wazuh
root      3581 25730  0 10:12 ?        00:00:00 /bin/sh wodles/aws/aws-s3 --bucket wazuh-aws-wodle --aws_profile prod --trail_prefix vpc --only_logs_after 2020-MAY-01 --type vpcflow
root      3588  3581 13 10:12 ?        00:31:11 /var/ossec/framework/python/bin/python3 /var/ossec/wodles/aws/aws-s3.py --bucket wazuh-aws-wodle --aws_profile prod --trail_prefix vpc --only_logs_after 2020-MAY-01 --type vpcflow
wazuh    25484     1  0 Sep07 ?        00:01:02 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    25485 25484  0 Sep07 ?        00:00:02 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    25488 25484  0 Sep07 ?        00:01:22 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    25491 25484  0 Sep07 ?        00:00:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    25518     1  0 Sep07 ?        00:01:32 /var/ossec/bin/wazuh-integratord
root     25538     1  0 Sep07 ?        00:03:50 /var/ossec/bin/wazuh-authd
wazuh    25555     1  0 Sep07 ?        00:01:43 /var/ossec/bin/wazuh-db
root     25580     1  0 Sep07 ?        00:00:02 /var/ossec/bin/wazuh-execd
wazuh    25595     1 14 Sep07 ?        03:27:51 /var/ossec/bin/wazuh-analysisd
root     25607     1  0 Sep07 ?        00:00:43 /var/ossec/bin/wazuh-syscheckd
wazuh    25628     1  0 Sep07 ?        00:03:44 /var/ossec/bin/wazuh-remoted
root     25662     1  0 Sep07 ?        00:00:08 /var/ossec/bin/wazuh-logcollector
wazuh    25681     1  0 Sep07 ?        00:01:13 /var/ossec/bin/wazuh-monitord
root     25730     1  5 Sep07 ?        01:21:20 /var/ossec/bin/wazuh-modulesd
wazuh    25849     1  0 Sep07 ?        00:02:19 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    25860 25849  0 Sep07 ?        00:00:22 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    25861 25849  0 Sep07 ?        00:00:21 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py

Wazuh server - master - env2

[root@wazuh-manager-master-0 wazuh-user]# ps -ef | grep -v grep | grep -v wazuh-u | grep  wazuh
wazuh    25111     1  0 Sep07 ?        00:00:45 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    25112 25111  0 Sep07 ?        00:00:01 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    25115 25111  0 Sep07 ?        00:01:18 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    25118 25111  0 Sep07 ?        00:00:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    25146     1  0 Sep07 ?        00:01:32 /var/ossec/bin/wazuh-integratord
root     25165     1  0 Sep07 ?        00:03:42 /var/ossec/bin/wazuh-authd
wazuh    25182     1  0 Sep07 ?        00:01:09 /var/ossec/bin/wazuh-db
root     25207     1  0 Sep07 ?        00:00:02 /var/ossec/bin/wazuh-execd
wazuh    25222     1 13 Sep07 ?        03:24:49 /var/ossec/bin/wazuh-analysisd
root     25234     1  0 Sep07 ?        00:00:43 /var/ossec/bin/wazuh-syscheckd
wazuh    25255     1  0 Sep07 ?        00:02:19 /var/ossec/bin/wazuh-remoted
root     25289     1  0 Sep07 ?        00:00:08 /var/ossec/bin/wazuh-logcollector
wazuh    25309     1  0 Sep07 ?        00:01:11 /var/ossec/bin/wazuh-monitord
root     25358     1  3 Sep07 ?        00:44:54 /var/ossec/bin/wazuh-modulesd
wazuh    25485     1  0 Sep07 ?        00:00:30 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    25491 25485  0 Sep07 ?        00:00:21 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    25492 25485  0 Sep07 ?        00:00:21 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
root     31766 25358  0 10:12 ?        00:00:00 /bin/sh wodles/aws/aws-s3 --bucket wazuh-aws-wodle --aws_profile prod --trail_prefix vpc --only_logs_after 2020-MAY-01 --type vpcflow
root     31773 31766 13 10:12 ?        00:32:20 /var/ossec/framework/python/bin/python3 /var/ossec/wodles/aws/aws-s3.py --bucket wazuh-aws-wodle --aws_profile prod --trail_prefix vpc --only_logs_after 2020-MAY-01 --type vpcflow

Wazuh server - worker - env1

[root@wazuh-manager-worker-0 wazuh-user]# ps -ef | grep -v grep | grep -v wazuh-u | grep  wazuh
wazuh    21341     1  0 Sep07 ?        00:00:11 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    21342 21341  0 Sep07 ?        00:00:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    21345 21341  0 Sep07 ?        00:00:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    21348 21341  0 Sep07 ?        00:00:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    21376     1  0 Sep07 ?        00:00:03 /var/ossec/bin/wazuh-integratord
wazuh    21393     1  0 Sep07 ?        00:01:12 /var/ossec/bin/wazuh-db
root     21418     1  0 Sep07 ?        00:00:02 /var/ossec/bin/wazuh-execd
wazuh    21432     1  0 Sep07 ?        00:00:14 /var/ossec/bin/wazuh-analysisd
root     21445     1  0 Sep07 ?        00:00:43 /var/ossec/bin/wazuh-syscheckd
wazuh    21467     1  0 Sep07 ?        00:02:18 /var/ossec/bin/wazuh-remoted
root     21500     1  0 Sep07 ?        00:00:09 /var/ossec/bin/wazuh-logcollector
wazuh    21519     1  0 Sep07 ?        00:00:04 /var/ossec/bin/wazuh-monitord
root     21568     1  2 Sep07 ?        00:39:28 /var/ossec/bin/wazuh-modulesd
wazuh    21713     1  0 Sep07 ?        00:02:39 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    21840 21713  0 Sep07 ?        00:00:57 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    22678 21713  0 Sep07 ?        00:00:00 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py

Note

The Wazuh dashboard instance has also a Wazuh indexer installed, this must be indicated

rauldpm · 2023-09-08T15:18:51Z

Analysis report

Check that the status of the indexer cluster is the expected.

Amazon Linux 2 - Wazuh indexer - node3

[root@ip-10-0-2-13 wazuh-user]# curl -k -u xxxx:xxxx https://10.0.2.13:9200/_cat/nodes?v
ip         heap.percent ram.percent cpu load_1m load_5m load_15m node.role node.roles                                        cluster_manager name
10.0.2.13            35          77   2    0.00    0.00     0.00 dimr      cluster_manager,data,ingest,remote_cluster_client *               node-3
10.0.0.232           56          78   2    0.00    0.00     0.00 dimr      cluster_manager,data,ingest,remote_cluster_client -               node-7
10.0.2.239           40          81   2    0.00    0.00     0.00 dimr      cluster_manager,data,ingest,remote_cluster_client -               node-1
10.0.2.117           39          79   2    0.00    0.00     0.00 dimr      cluster_manager,data,ingest,remote_cluster_client -               node-2

[root@ip-10-0-2-13 wazuh-user]# curl -k -u xxxx:xxxx https://10.0.2.13:9200/_cluster/health?pretty
{
  "cluster_name" : "wazuh",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 4,
  "number_of_data_nodes" : 4,
  "discovered_master" : true,
  "discovered_cluster_manager" : true,
  "active_primary_shards" : 20,
  "active_shards" : 41,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

[root@ip-10-0-2-13 wazuh-user]# curl -k -u xxxx:xxxx https://10.0.2.13:9200/
{
  "name" : "node-3",
  "cluster_name" : "wazuh",
  "cluster_uuid" : "FRyhPsRvSbeDIvllMsUrdQ",
  "version" : {
    "number" : "7.10.2",
    "build_type" : "rpm",
    "build_hash" : "db90a415ff2fd428b4f7b3f800a51dc229287cb4",
    "build_date" : "2023-06-03T06:24:25.112415503Z",
    "build_snapshot" : false,
    "lucene_version" : "9.6.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

rauldpm · 2023-09-08T16:25:20Z

Analysis report

Check that there are no errors in the browser's developer console when browsing the App.

Wazuh dashboard WUI

Main

Reported at Web console errors accesing Wazuh Dashboard wazuh-dashboard-plugins#4121

Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-75XtnrpDA0UHDMcl7S8lvswryIOd0RqgacRh0AMOgdk='), or a nonce ('nonce-...') is required to enable inline execution.

Wazuh:352

Reported at Web console errors accesing Wazuh Dashboard wazuh-dashboard-plugins#4121

bootstrap.js:43 ^ A single error about an inline script not firing due to content security policy is expected!

Reported at Multiple unexpected Errors during Wazuh app session wazuh-dashboard-plugins#5821

core.entry.js:15 Detected an unhandled Promise rejection.
TypeError: Cannot read properties of undefined (reading 'split')

Reported at: reportsDashboards error in console - compatibility OpenSearch 2.6 & 1.3 wazuh-dashboard-plugins#5332

reportsDashboards.plugin.js:24 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'split')
    at _callee4$ (reportsDashboards.plugin.js:24:201073)
    at tryCatch (reportsDashboards.plugin.js:24:180012)
    at Generator.invoke [as _invoke] (reportsDashboards.plugin.js:24:183963)
    at Generator.next (reportsDashboards.plugin.js:24:181151)
    at asyncGeneratorStep (reportsDashboards.plugin.js:1:73707)
    at _next (reportsDashboards.plugin.js:1:74018)
    at reportsDashboards.plugin.js:1:74168
    at new Promise (<anonymous>)
    at reportsDashboards.plugin.js:1:73930
    at checkURLParams (reportsDashboards.plugin.js:24:202215)

Side menu - Alerting

Reported at Configured indices are not found: [.opendistro-alerting-config] in Demo environment wazuh-dashboard-plugins#5869

error getting monitors: 
{ok: false, resp: '[alerting_exception] Configured indices are not found: [.opendistro-alerting-config]'}
  ok: false
  resp: "[alerting_exception] Configured indices are not found: [.opendistro-alerting-config]"
  [[Prototype]]: Object

Back to Main from Alerting (env1)

Reported at Errors with code 401 have been found in the demo environment wazuh-dashboard-plugins#5871
Also seen in Wazuh -> Modules -> Ubuntu -> Amazon AWS and Wazuh -> Management -> Rules (change from env2 to env1)

wazuh.plugin.js:8 
 POST https://demo.wazuh.info/api/request 401 (Unauthorized)
dispatchXhrRequest	@	wazuh.plugin.js:8
xhr	@	wazuh.plugin.js:8
dispatchRequest	@	wazuh.plugin.js:8
request	@	wazuh.plugin.js:8
wrap	@	wazuh.plugin.js:1
_callee$	@	wazuh.plugin.js:1
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
asyncGeneratorStep	@	wazuh.plugin.js:1
_next	@	wazuh.plugin.js:1
(anonymous)	@	wazuh.plugin.js:1
(anonymous)	@	wazuh.plugin.js:1
request	@	wazuh.plugin.js:1
_callee$	@	wazuh.plugin.js:1
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
asyncGeneratorStep	@	wazuh.plugin.js:1
_next	@	wazuh.plugin.js:1
(anonymous)	@	wazuh.plugin.js:1
(anonymous)	@	wazuh.plugin.js:1
genericReq	@	wazuh.plugin.js:1
_callee2$	@	wazuh.plugin.js:1
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
asyncGeneratorStep	@	wazuh.plugin.js:1
_next	@	wazuh.plugin.js:1
(anonymous)	@	wazuh.plugin.js:1
(anonymous)	@	wazuh.plugin.js:1
apiReq	@	wazuh.plugin.js:1
_callee7$	@	wazuh.chunk.10.js:1
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
agents_preview_asyncGeneratorStep	@	wazuh.chunk.10.js:1
_next	@	wazuh.chunk.10.js:1
(anonymous)	@	wazuh.chunk.10.js:1
(anonymous)	@	wazuh.chunk.10.js:1
getWazuhVersion	@	wazuh.chunk.10.js:1
getWazuhVersion	@	wazuh.chunk.10.js:1
wrapped	@	wazuh.chunk.6.js:17
_callee3$	@	wazuh.chunk.10.js:1
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
register_agent_asyncGeneratorStep	@	wazuh.chunk.10.js:1
_next	@	wazuh.chunk.10.js:1
(anonymous)	@	wazuh.chunk.10.js:1
(anonymous)	@	wazuh.chunk.10.js:1
fetchData	@	wazuh.chunk.10.js:1
(anonymous)	@	wazuh.chunk.10.js:1
is	@	osd-ui-shared-deps.js:436
gc	@	osd-ui-shared-deps.js:436
t.unstable_runWithPriority	@	osd-ui-shared-deps.js:444
ji	@	osd-ui-shared-deps.js:436
bc	@	osd-ui-shared-deps.js:436
Qs	@	osd-ui-shared-deps.js:436
(anonymous)	@	osd-ui-shared-deps.js:436
t.unstable_runWithPriority	@	osd-ui-shared-deps.js:444
ji	@	osd-ui-shared-deps.js:436
Ui	@	osd-ui-shared-deps.js:436
Hi	@	osd-ui-shared-deps.js:436
tc	@	osd-ui-shared-deps.js:436
jc	@	osd-ui-shared-deps.js:436
t.render	@	osd-ui-shared-deps.js:436
(anonymous)	@	wazuh.chunk.6.js:17
$digest	@	osd-ui-shared-deps.js:428
(anonymous)	@	osd-ui-shared-deps.js:428
Jr.completeTask	@	osd-ui-shared-deps.js:428
(anonymous)	@	osd-ui-shared-deps.js:428
setTimeout (async)		
o.defer	@	osd-ui-shared-deps.js:428
$evalAsync	@	osd-ui-shared-deps.js:428
(anonymous)	@	osd-ui-shared-deps.js:428
p	@	osd-ui-shared-deps.js:428
e	@	osd-ui-shared-deps.js:428
o	@	osd-ui-shared-deps.js:428
Promise.then (async)		
e	@	osd-ui-shared-deps.js:428
d	@	osd-ui-shared-deps.js:428
M	@	osd-ui-shared-deps.js:428
(anonymous)	@	osd-ui-shared-deps.js:428
y	@	osd-ui-shared-deps.js:428
Nr.O.all	@	osd-ui-shared-deps.js:428
resolveLocals	@	wazuh.chunk.11.js:48
(anonymous)	@	osd-ui-shared-deps.js:428
(anonymous)	@	osd-ui-shared-deps.js:428
$digest	@	osd-ui-shared-deps.js:428
$apply	@	osd-ui-shared-deps.js:428
(anonymous)	@	osd-ui-shared-deps.js:428
invoke	@	osd-ui-shared-deps.js:428
i	@	osd-ui-shared-deps.js:428
Le	@	osd-ui-shared-deps.js:428
mountWazuhApp	@	wazuh.chunk.29.js:1
_callee$	@	wazuh.chunk.29.js:1
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
asyncGeneratorStep	@	wazuh.chunk.29.js:1
_next	@	wazuh.chunk.29.js:1
Promise.then (async)		
asyncGeneratorStep	@	wazuh.chunk.29.js:1
_next	@	wazuh.chunk.29.js:1
(anonymous)	@	wazuh.chunk.29.js:1
(anonymous)	@	wazuh.chunk.29.js:1
_renderApp	@	wazuh.chunk.29.js:1
renderApp	@	wazuh.chunk.29.js:1
_callee$	@	wazuh.plugin.js:25
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
asyncGeneratorStep	@	wazuh.plugin.js:25
_next	@	wazuh.plugin.js:25
Promise.then (async)		
asyncGeneratorStep	@	wazuh.plugin.js:25
_next	@	wazuh.plugin.js:25
Promise.then (async)		
asyncGeneratorStep	@	wazuh.plugin.js:25
_next	@	wazuh.plugin.js:25
Promise.then (async)		
asyncGeneratorStep	@	wazuh.plugin.js:25
_next	@	wazuh.plugin.js:25
(anonymous)	@	wazuh.plugin.js:25
(anonymous)	@	wazuh.plugin.js:25
mount	@	wazuh.plugin.js:25
_callee$	@	core.entry.js:15
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
application_service_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
(anonymous)	@	core.entry.js:15
(anonymous)	@	core.entry.js:15
(anonymous)	@	core.entry.js:15
_callee$	@	core.entry.js:15
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
(anonymous)	@	core.entry.js:15
(anonymous)	@	core.entry.js:15
mount	@	core.entry.js:15
(anonymous)	@	core.entry.js:15
is	@	osd-ui-shared-deps.js:436
os	@	osd-ui-shared-deps.js:436
fc	@	osd-ui-shared-deps.js:436
t.unstable_runWithPriority	@	osd-ui-shared-deps.js:444
ji	@	osd-ui-shared-deps.js:436
dc	@	osd-ui-shared-deps.js:436
Qs	@	osd-ui-shared-deps.js:436
(anonymous)	@	osd-ui-shared-deps.js:436
t.unstable_runWithPriority	@	osd-ui-shared-deps.js:444
ji	@	osd-ui-shared-deps.js:436
Ui	@	osd-ui-shared-deps.js:436
Hi	@	osd-ui-shared-deps.js:436
tc	@	osd-ui-shared-deps.js:436
jc	@	osd-ui-shared-deps.js:436
t.render	@	osd-ui-shared-deps.js:436
start	@	core.entry.js:15
_callee2$	@	core.entry.js:15
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
core_system_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
Promise.then (async)		
core_system_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
Promise.then (async)		
core_system_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
Promise.then (async)		
core_system_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
Promise.then (async)		
core_system_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
Promise.then (async)		
core_system_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
Promise.then (async)		
core_system_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
Promise.then (async)		
core_system_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
Promise.then (async)		
core_system_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
Promise.then (async)		
core_system_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
Promise.then (async)		
core_system_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
Promise.then (async)		
core_system_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
(anonymous)	@	core.entry.js:15
(anonymous)	@	core.entry.js:15
start	@	core.entry.js:15
_callee$	@	core.entry.js:15
u	@	osd-ui-shared-deps.js:382
(anonymous)	@	osd-ui-shared-deps.js:382
(anonymous)	@	osd-ui-shared-deps.js:382
osd_bootstrap_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
Promise.then (async)		
osd_bootstrap_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
Promise.then (async)		
osd_bootstrap_asyncGeneratorStep	@	core.entry.js:15
_next	@	core.entry.js:15
(anonymous)	@	core.entry.js:15
(anonymous)	@	core.entry.js:15
_osdBootstrap__	@	core.entry.js:15
__osdBootstrap__	@	core.entry.js:15
(anonymous)	@	bootstrap.js:179
innerCb	@	bootstrap.js:108
load (async)		
loadScript	@	bootstrap.js:98
(anonymous)	@	bootstrap.js:117
load	@	bootstrap.js:104
window.onload	@	bootstrap.js:122
load (async)		
(anonymous)	@	bootstrap.js:48

This may be related to the 401 error observed in the Wazuh dashboard service

Sep 07 19:47:48 ip-10-0-0-232.us-west-1.compute.internal opensearch-dashboards[20957]: {"type":"response","@timestamp":"2023-09-07T19:47:48Z","tags":[],"pid":20957,"method":"get","statusCode":401,"req":{"url":"/favicon.ico","method":"get","headers":{"host":"10.0.0.232:5601","connection":"close","user-agent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)","accept-encoding":"gzip"},"remoteAddress":"10.0.0.232","userAgent":"Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)"},"res":{"statusCode":401,"responseTime":3,"contentLength":9},"message":"GET /favicon.ico 401 3ms - 9.0B"}

Main - reset password

Reported at Password field is not contained in a form wazuh-dashboard-plugins#5873

[DOM] Password field is not contained in a form: (More info: https://goo.gl/9p2vKq) <input type="password" id="i470de791-4e5e-11ee-a2e5-45558a0d0d97" class="euiFieldPassword" data-test-subj="current-password" aria-describedby="i470de791-4e5e-11ee-a2e5-45558a0d0d97-help-0" spellcheck="false" value>
/app/wazuh#/overview/?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-24h,to:now))&_a=(columns:!(_source),filters:!(),index:'wazuh-alerts-*',interval:auto,query:(language:kuery,query:''),sort:!())&tabView=panels&tab=welcome:1 
[DOM] Password field is not contained in a form: (More info: https://goo.gl/9p2vKq) <input type="password" id="i470e0ea1-4e5e-11ee-a2e5-45558a0d0d97" class="euiFieldPassword" data-test-subj="new-password" aria-describedby="i470e0ea1-4e5e-11ee-a2e5-45558a0d0d97-help-0" spellcheck="false" value>
/app/wazuh#/overview/?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-24h,to:now))&_a=(columns:!(_source),filters:!(),index:'wazuh-alerts-*',interval:auto,query:(language:kuery,query:''),sort:!())&tabView=panels&tab=welcome:1 
[DOM] Password field is not contained in a form: (More info: https://goo.gl/9p2vKq) <input type="password" id="i470e0ea3-4e5e-11ee-a2e5-45558a0d0d97" class="euiFieldPassword" data-test-subj="reenter-new-password" aria-describedby="i470e0ea3-4e5e-11ee-a2e5-45558a0d0d97-help-0" spellcheck="false" value>

Wazuh -> Security -> Create new user

Reported at Corrupted RBAC database when trying to create a user using the WUI #18900

User name: myuser
Password: --
Allow run as: true
User roles: agents_admin

Note: The same happens while trying to create a role

wazuh.plugin.js:8 
 POST https://demo.wazuh.info/api/request 500 (Internal Server Error)
dispatchXhrRequest	@	wazuh.plugin.js:8
xhr	@	wazuh.plugin.js:8
dispatchRequest	@	wazuh.plugin.js:8
request	@	wazuh.plugin.js:8
wrap	@	wazuh.plugin.js:1
_callee$	@	wazuh.plugin.js:1
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
asyncGeneratorStep	@	wazuh.plugin.js:1
_next	@	wazuh.plugin.js:1
(anonymous)	@	wazuh.plugin.js:1
(anonymous)	@	wazuh.plugin.js:1
request	@	wazuh.plugin.js:1
_callee$	@	wazuh.plugin.js:1
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
asyncGeneratorStep	@	wazuh.plugin.js:1
_next	@	wazuh.plugin.js:1
(anonymous)	@	wazuh.plugin.js:1
(anonymous)	@	wazuh.plugin.js:1
genericReq	@	wazuh.plugin.js:1
_callee2$	@	wazuh.plugin.js:1
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
asyncGeneratorStep	@	wazuh.plugin.js:1
_next	@	wazuh.plugin.js:1
(anonymous)	@	wazuh.plugin.js:1
(anonymous)	@	wazuh.plugin.js:1
apiReq	@	wazuh.plugin.js:1
_callee$	@	wazuh.chunk.10.js:1
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
create_user_service_asyncGeneratorStep	@	wazuh.chunk.10.js:1
_next	@	wazuh.chunk.10.js:1
(anonymous)	@	wazuh.chunk.10.js:1
(anonymous)	@	wazuh.chunk.10.js:1
CreateUserService	@	wazuh.chunk.10.js:1
_callee$	@	wazuh.chunk.10.js:1
tryCatch	@	customImportMapDashboards.plugin.js:13
invoke	@	customImportMapDashboards.plugin.js:13
(anonymous)	@	customImportMapDashboards.plugin.js:13
create_user_asyncGeneratorStep	@	wazuh.chunk.10.js:1
_next	@	wazuh.chunk.10.js:1
(anonymous)	@	wazuh.chunk.10.js:1
(anonymous)	@	wazuh.chunk.10.js:1
editUser	@	wazuh.chunk.10.js:1
s	@	osd-ui-shared-deps.js:436
f	@	osd-ui-shared-deps.js:436
(anonymous)	@	osd-ui-shared-deps.js:436
m	@	osd-ui-shared-deps.js:436
at	@	osd-ui-shared-deps.js:436
it	@	osd-ui-shared-deps.js:436
st	@	osd-ui-shared-deps.js:436
ft	@	osd-ui-shared-deps.js:436
D	@	osd-ui-shared-deps.js:436
F	@	osd-ui-shared-deps.js:436
Jt	@	osd-ui-shared-deps.js:436
Zt	@	osd-ui-shared-deps.js:436
t.unstable_runWithPriority	@	osd-ui-shared-deps.js:444
ji	@	osd-ui-shared-deps.js:436
W	@	osd-ui-shared-deps.js:436
Gt	@	osd-ui-shared-deps.js:436

AxiosError: Wazuh API error: ERR_BAD_RESPONSE - Corrupted RBAC database
    at settle (https://demo.wazuh.info/46000/bundles/plugin/wazuh/wazuh.plugin.js:8:20234)
    at XMLHttpRequest.onloadend (https://demo.wazuh.info/46000/bundles/plugin/wazuh/wazuh.plugin.js:8:25708)

The context of this point is too broad, it is necessary to study if a dedicated issue should be created

rauldpm · 2023-09-08T17:31:55Z

Blocked by https://github.com/wazuh/wazuh-automation/issues/1283

There are 13,371,724 alerts from wazuh-aws-wodle of a total of 13,402,731

[root@wazuh-manager-master-0 wazuh-user]# du -sh /var/ossec/logs/alerts
16G	/var/ossec/logs/alerts
[root@wazuh-manager-master-0 wazuh-user]# df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        1.9G     0  1.9G   0% /dev
tmpfs           2.0G  180K  2.0G   1% /dev/shm
tmpfs           2.0G  476K  2.0G   1% /run
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
/dev/nvme0n1p1   20G   20G   20K 100% /
tmpfs           390M     0  390M   0% /run/user/1000
tmpfs           390M     0  390M   0% /run/user/1001

Reported at AWS wodle shows abnormal behavior with the number of generated alerts #18901
The issue will continue to be worked on as much as possible, but it is possible that what was analyzed will be affected by this behavior

rauldpm · 2023-09-11T13:01:21Z

Analysis report

Check that there are alerts for each of the modules configured.

The environment has been modified by @wazuh/cicd due to AWS wodle shows abnormal behavior with the number of generated alerts #18901, when accessing the WUI the following error is observed

INFO: Current API id [env-2]
INFO: Checking current API id [env-2]...
INFO: Current API id [env-2] has some problem: 3002 - Request failed with status code 400
INFO: Getting API hosts...
INFO: API hosts found: 2
INFO: Checking API host id [env-1]...
INFO: Could not connect to API id [env-1]: 3099 - ERROR3099 - Some Wazuh daemons are not ready yet in node "master" (wazuh-analysisd->stopped)
INFO: Checking API host id [env-2]...
INFO: Could not connect to API id [env-2]: 3099 - ERROR3099 - Some Wazuh daemons are not ready yet in node "master" (wazuh-analysisd->stopped)
INFO: Removed [navigate] cookie
ERROR: No API available to connect

[API connection] No API available to connect

When observing the Wazuh manager service, it can be seen that both are active, but the env-2 Wazuh manager service is active (exited) with live processes, while the env-1 Wazuh manager service is active (running) with live processes

env-1 node

[root@wazuh-manager-master-0 wazuh-user]# systemctl status wazuh-manager.service 
● wazuh-manager.service - Wazuh manager
   Loaded: loaded (/usr/lib/systemd/system/wazuh-manager.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2023-09-08 19:27:13 UTC; 2 days ago
  Process: 16250 ExecStop=/usr/bin/env /var/ossec/bin/wazuh-control stop (code=exited, status=0/SUCCESS)
  Process: 16413 ExecStart=/usr/bin/env /var/ossec/bin/wazuh-control start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/wazuh-manager.service
           ├─16478 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
           ├─16479 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
           ├─16482 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
           ├─16485 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
           ├─16513 /var/ossec/bin/wazuh-integratord
           ├─16532 /var/ossec/bin/wazuh-authd
           ├─16550 /var/ossec/bin/wazuh-db
           ├─16576 /var/ossec/bin/wazuh-execd
           ├─16606 /var/ossec/bin/wazuh-syscheckd
           ├─16628 /var/ossec/bin/wazuh-remoted
           ├─16661 /var/ossec/bin/wazuh-logcollector
           ├─16682 /var/ossec/bin/wazuh-monitord
           ├─16732 /var/ossec/bin/wazuh-modulesd
           ├─16869 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
           ├─16898 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
           └─16899 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
[root@wazuh-manager-master-0 wazuh-user]# /var/ossec/bin/wazuh-control status
wazuh-clusterd is running...
wazuh-modulesd is running...
wazuh-monitord is running...
wazuh-logcollector is running...
wazuh-remoted is running...
wazuh-syscheckd is running...
wazuh-analysisd not running...
wazuh-maild not running...
wazuh-execd is running...
wazuh-db is running...
wazuh-authd is running...
wazuh-agentlessd not running...
wazuh-integratord is running...
wazuh-dbd not running...
wazuh-csyslogd not running...
wazuh-apid is running...
[root@wazuh-manager-master-0 wazuh-user]# ps -ef | grep wazuh 
root     14656 12357  0 12:01 ?        00:00:00 sshd: wazuh-user [priv]
wazuh-u+ 14679 14656  0 12:01 ?        00:00:00 sshd: wazuh-user@pts/0
wazuh-u+ 14680 14679  0 12:01 pts/0    00:00:00 -bash
wazuh    16478     1  0 Sep08 ?        00:01:16 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    16479 16478  0 Sep08 ?        00:00:02 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    16482 16478  0 Sep08 ?        00:01:30 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    16485 16478  0 Sep08 ?        00:00:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    16513     1  0 Sep08 ?        00:01:30 /var/ossec/bin/wazuh-integratord
root     16532     1  0 Sep08 ?        00:09:52 /var/ossec/bin/wazuh-authd
wazuh    16550     1  0 Sep08 ?        00:05:50 /var/ossec/bin/wazuh-db
root     16576     1  0 Sep08 ?        00:00:05 /var/ossec/bin/wazuh-execd
root     16606     1  0 Sep08 ?        00:00:46 /var/ossec/bin/wazuh-syscheckd
wazuh    16628     1  0 Sep08 ?        00:06:06 /var/ossec/bin/wazuh-remoted
root     16661     1  0 Sep08 ?        00:00:28 /var/ossec/bin/wazuh-logcollector
wazuh    16682     1  0 Sep08 ?        00:06:22 /var/ossec/bin/wazuh-monitord
root     16732     1  1 Sep08 ?        00:50:13 /var/ossec/bin/wazuh-modulesd
wazuh    16869     1  0 Sep08 ?        00:08:33 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    16898 16869  0 Sep08 ?        00:01:04 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    16899 16869  0 Sep08 ?        00:01:05 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
root     17870 12357  0 12:49 ?        00:00:00 sshd: wazuh-user [priv]
wazuh-u+ 17876 17870  0 12:49 ?        00:00:00 sshd: wazuh-user@pts/1
wazuh-u+ 17877 17876  0 12:49 pts/1    00:00:00 -bash
root     18122 18019  0 12:51 pts/1    00:00:00 grep --color=auto wazuh

env-2 node

[root@wazuh-manager-master-0 wazuh-user]# systemctl status wazuh-manager.service 
● wazuh-manager.service - Wazuh manager
   Loaded: loaded (/usr/lib/systemd/system/wazuh-manager.service; enabled; vendor preset: disabled)
   Active: active (exited) since Thu 2023-09-07 13:26:14 UTC; 3 days ago
  Process: 17922 ExecStop=/usr/bin/env /var/ossec/bin/wazuh-control stop (code=exited, status=0/SUCCESS)
  Process: 18081 ExecStart=/usr/bin/env /var/ossec/bin/wazuh-control start (code=exited, status=0/SUCCESS)

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
[root@wazuh-manager-master-0 wazuh-user]# /var/ossec/bin/wazuh-control status
wazuh-clusterd is running...
wazuh-modulesd is running...
wazuh-monitord is running...
wazuh-logcollector is running...
wazuh-remoted is running...
wazuh-syscheckd is running...
wazuh-analysisd not running...
wazuh-maild not running...
wazuh-execd is running...
wazuh-db is running...
wazuh-authd is running...
wazuh-agentlessd not running...
wazuh-integratord is running...
wazuh-dbd not running...
wazuh-csyslogd not running...
wazuh-apid is running...
[root@wazuh-manager-master-0 wazuh-user]# ps -ef | grep wazuh
root      8381 12350  0 12:49 ?        00:00:00 sshd: wazuh-user [priv]
wazuh-u+  8383  8381  0 12:49 ?        00:00:00 sshd: wazuh-user@pts/0
wazuh-u+  8384  8383  0 12:49 pts/0    00:00:00 -bash
root      8520  8429  0 12:51 pts/0    00:00:00 grep --color=auto wazuh
wazuh    25111     1  0 Sep07 ?        00:01:57 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    25112 25111  0 Sep07 ?        00:00:18 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    25115 25111  0 Sep07 ?        00:03:20 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    25118 25111  0 Sep07 ?        00:00:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
wazuh    25146     1  0 Sep07 ?        00:03:04 /var/ossec/bin/wazuh-integratord
root     25165     1  0 Sep07 ?        00:14:24 /var/ossec/bin/wazuh-authd
wazuh    25182     1  0 Sep07 ?        00:06:00 /var/ossec/bin/wazuh-db
root     25207     1  0 Sep07 ?        00:00:08 /var/ossec/bin/wazuh-execd
root     25234     1  0 Sep07 ?        00:01:21 /var/ossec/bin/wazuh-syscheckd
wazuh    25255     1  0 Sep07 ?        00:07:56 /var/ossec/bin/wazuh-remoted
root     25289     1  0 Sep07 ?        00:00:42 /var/ossec/bin/wazuh-logcollector
wazuh    25309     1  0 Sep07 ?        00:10:53 /var/ossec/bin/wazuh-monitord
root     25358     1  2 Sep07 ?        02:27:17 /var/ossec/bin/wazuh-modulesd
wazuh    25485     1  0 Sep07 ?        00:02:13 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    25491 25485  0 Sep07 ?        00:01:19 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
wazuh    25492 25485  0 Sep07 ?        00:01:20 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py

Reported at: #18946

Due to this it is not possible to continue analyzing the environment and the analysis will be completed in the next testing iteration, the environment is available for investigation.

rauldpm · 2023-09-11T18:13:26Z

Analysis report

Check that no warning symbols appear in the browser's developer console when browsing the App.

Done in #18843 (comment)

teddytpc1 · 2023-09-12T14:30:29Z

The environment seems to be working fine. It was requested to @rauldpm to provide evidence for the remaining items:

Check that there are alerts for each of the modules configured.
Generate an alert and check that this alert appears in the dashboard (end to end)
Check that the search engine works without specifying a field and using *

rauldpm · 2023-09-12T14:41:02Z

Analysis report

The environment is still in a defective state

Check that there are alerts for each of the modules configured.

Images provided by @teddytpc1

Generate an alert and check that this alert appears in the dashboard (end to end)

This task will not done since it implies modifying the Wazuh manager configuration, in the next step it is observed that there are alerts generated

Check that the search engine works without specifying a field and using *

davidjiglesias · 2023-09-13T09:01:59Z

LGTM!

davidjiglesias added level/subtask type/test labels Sep 7, 2023

wazuhci added this to Release 4.6.0 Sep 7, 2023

wazuhci moved this to Triage in Release 4.6.0 Sep 7, 2023

rauldpm mentioned this issue Sep 7, 2023

Release 4.6.0 - Alpha 1 - E2E UX tests #18857

Closed

rauldpm self-assigned this Sep 7, 2023

wazuhci moved this from Triage to Blocked in Release 4.6.0 Sep 7, 2023

wazuhci moved this from Blocked to In progress in Release 4.6.0 Sep 7, 2023

rauldpm mentioned this issue Sep 8, 2023

Corrupted RBAC database when trying to create a user using the WUI #18900

Closed

wazuhci moved this from In progress to Blocked in Release 4.6.0 Sep 8, 2023

wazuhci moved this from Blocked to In progress in Release 4.6.0 Sep 8, 2023

rauldpm mentioned this issue Sep 11, 2023

wazuh-analysisd is stopped for no apparent reason in the demo environment #18946

Closed

3 tasks

This was referenced Sep 11, 2023

Message classified as INFO informs of an error when not finding a DLL file of the agent #18957

Open

Error message when performing Auditpol backup #18959

Open

ResponseError messages found in the Wazuh dashboard logs wazuh/wazuh-packages#2442

Closed

wazuhci moved this from In progress to Pending review in Release 4.6.0 Sep 11, 2023

wazuhci moved this from Pending review to Pending final review in Release 4.6.0 Sep 12, 2023

davidjiglesias closed this as completed Sep 13, 2023

wazuhci moved this from Pending final review to Done in Release 4.6.0 Sep 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release 4.6.0 - Alpha 1 - E2E UX tests - Demo environment #18843

Release 4.6.0 - Alpha 1 - E2E UX tests - Demo environment #18843

davidjiglesias commented Sep 7, 2023 •

edited

Loading

rauldpm commented Sep 7, 2023 •

edited

Loading

rauldpm commented Sep 7, 2023 •

edited

Loading

rauldpm commented Sep 8, 2023 •

edited

Loading

rauldpm commented Sep 8, 2023

rauldpm commented Sep 8, 2023 •

edited

Loading

rauldpm commented Sep 8, 2023 •

edited

Loading

rauldpm commented Sep 11, 2023 •

edited

Loading

rauldpm commented Sep 11, 2023

teddytpc1 commented Sep 12, 2023

rauldpm commented Sep 12, 2023

davidjiglesias commented Sep 13, 2023

Release 4.6.0 - Alpha 1 - E2E UX tests - Demo environment #18843

Release 4.6.0 - Alpha 1 - E2E UX tests - Demo environment #18843

Comments

davidjiglesias commented Sep 7, 2023 • edited Loading

End-to-End (E2E) Testing Guideline

Deployment requirements

Test description

Known issues

Conclusions

Feedback

Reviewers validation

rauldpm commented Sep 7, 2023 • edited Loading

Update report

rauldpm commented Sep 7, 2023 • edited Loading

Analysis report

Check that there are no errors in manager, agent, cluster, indexer and dashboard logs

rauldpm commented Sep 8, 2023 • edited Loading

Analysis report

Check that wazuh daemons are running with the expected user.

Note

rauldpm commented Sep 8, 2023

Analysis report

Check that the status of the indexer cluster is the expected.

rauldpm commented Sep 8, 2023 • edited Loading

Analysis report

Check that there are no errors in the browser's developer console when browsing the App.

rauldpm commented Sep 8, 2023 • edited Loading

rauldpm commented Sep 11, 2023 • edited Loading

Analysis report

Check that there are alerts for each of the modules configured.

rauldpm commented Sep 11, 2023

Analysis report

Check that no warning symbols appear in the browser's developer console when browsing the App.

teddytpc1 commented Sep 12, 2023

rauldpm commented Sep 12, 2023

Analysis report

Check that there are alerts for each of the modules configured.

Generate an alert and check that this alert appears in the dashboard (end to end)

Check that the search engine works without specifying a field and using *

davidjiglesias commented Sep 13, 2023

davidjiglesias commented Sep 7, 2023 •

edited

Loading

rauldpm commented Sep 7, 2023 •

edited

Loading

rauldpm commented Sep 7, 2023 •

edited

Loading

rauldpm commented Sep 8, 2023 •

edited

Loading

rauldpm commented Sep 8, 2023 •

edited

Loading

rauldpm commented Sep 8, 2023 •

edited

Loading

rauldpm commented Sep 11, 2023 •

edited

Loading