Release 4.5.0 - Alpha 1 - Integration tests #18114

roronoasins · 2023-07-27T09:16:49Z

Description

Wazuh QA: Branch	Wazuh QA: Commit	Wazuh: Tag	Wazuh: Commit
`v4.5.0-alpha1`	wazuh/wazuh-qa@`ede149c`	`v4.5.0-alpha1`	`39df1e7`

We are going to check that the integration tests of the 4.5 branch of wazuh-qa work correctly using the v4.5.0-alpha1 version of wazuh.

The tests will be performed in Jenkins using centOS as the manager OS. As for the agents, Linux, Windows , Solaris, macOS will be used as required.

Tests Integration - Status

Main RC issue

Release 4.5.0 - Alpha 1 #18058

References

Color	Status
🟢	All tests passed successfully
🟡	All tests passed but there are some warnings
🔴	Some tests have failures or errors
🔵	Test execution in progress
⚫	To Do
🟠	Jenkins provision fails
🟣	All skipped

Test Integration - Results

Name	Jenkins
OS	Linux		Windows	Solaris	macOS
Target	Manager	Agent	Agent	Agent	Agent
active_response	🟢	🟢	🟢	NA	NA
agentd	NA	🟢	🟢	NA	NA
analysisd	🟢	NA	NA	NA	NA
api	🟢	NA	NA	NA	NA
authd	🟢	NA	NA	NA	NA
enrollment	NA	🟢	🟢	NA	NA
fim	🟢	🟢	🟢	🟢	🟢
gcloud	🟢	NA	NA	NA	NA
github	🟢	🟢	NA	NA	NA
logcollector	🟢	🟢	🟢	🟢	🔴
logtest	🟢	NA	NA	NA	NA
office365	🟢	🟢	NA	NA	NA
remoted	🟢	NA	NA	NA	NA
rids	🟢	NA	NA	NA	NA
rootcheck	🟢	NA	NA	NA	NA
vulnerability_detector	🔴	NA	NA	NA	NA
wazuh_db	🟢	NA	NA	NA	NA
syscollector	🟢	🟢	🔴	NA	🟢

Evidence

IT Launcher build: https://ci.wazuh.info/job/Test_integration_launcher/180/

Conclusion 🔴

Some tests failed, and a known issue was reported

Syscollector scanning IT fails for Windows agents wazuh-qa#4230

leading to the creation of some new issues.

Auditors validation

The definition of done for this one is the validation of the conclusions and the test results from all auditors.

All checks from below must be accepted in order to close this issue.

The text was updated successfully, but these errors were encountered:

roronoasins · 2023-07-27T15:49:04Z

Some of them failed. After a quick review a few are expected(vd, logcollector, syscollector). Tomorrow all the reports will be added

roronoasins · 2023-07-28T08:58:17Z

FIM 🔴 (not valid tag used - it does not apply to this testing)

non-valid jenkins tag used

Agent

Details

Build: https://ci.wazuh.info/job/Test_integration/41715/
Report: agent_solaris_html_report_Test_integration_B41715_20230727102907.zip

Failed tests:

test_fim/test_files/test_basic_usage/test_basic_usage_realtime_unsupported.py::test_realtime_unsupported[get_configuration0]

Issue:

Solaris FIM IT : test_realtime_unsupported fail wazuh-qa#2881

Description

The test_execd_firewall_drop test aims to verify if the current OS platform falls to the 'scheduled' mode when 'realtime' is not available. To achieve this, the test performs a CUD set of operations to a file with 'realtime' mode set as, but the test fails due to a TimeoutError.

        regular_file_cud(directory_str, log_monitor, file_list=[test_file], time_travel=False, triggers_event=True,
>                        event_mode="scheduled", min_timeout=15)

test_fim/test_files/test_basic_usage/test_basic_usage_realtime_unsupported.py:144:

                        logger.error(error_message)
                        logger.error(f"Results accumulated: "
                                     f"{len(result) if isinstance(result, list) else 0}")
                        logger.error(f"Results expected: {accum_results}")
>                   raise TimeoutError(error_message)
E                   TimeoutError: TimeoutError was raised because a single 'added' event was expected for ['/dir/testfile.txt'] but was not detected.

roronoasins · 2023-07-28T09:13:42Z

Logcollector 🔴

Agent

Details

Build: https://ci.wazuh.info/job/Test_integration/41761/
Report:

MacOS error

15:28:59  TASK [Launch macos pytests] ****************************************************
15:28:59  task path: /home/ec2-user/workspace/Test_integration/quality/tests/integration/test_integration.yaml:25
15:28:59  fatal: [vagrant_host]: UNREACHABLE! => {
15:28:59      "changed": false,
15:28:59      "unreachable": true
15:28:59  }
15:28:59  
15:28:59  MSG:
15:28:59  
15:28:59  Data could not be sent to remote host "10.10.0.251". Make sure this host can be reached over ssh: ssh: connect to host 10.10.0.251 port 63805: Connection timed out

15:28:59  
15:28:59  
15:28:59  PLAY RECAP *********************************************************************
15:28:59  vagrant_host               : ok=0    changed=0    unreachable=1    failed=0    skipped=1    rescued=0    ignored=0

Issue:

https://github.com/wazuh/wazuh-jenkins/issues/5468

Description

The test for MacOS failed due to the machine connectivity

roronoasins · 2023-07-28T09:32:32Z

Vulnerability Detector 🔴

Manager

Details

Build: https://ci.wazuh.info/job/Test_integration/41727/
Report: manager_html_report_Test_integration_B41727_20230727102433.zip

Failed tests:

test_vulnerability_detector/test_feeds/test_download_feeds.py
- SUSE Linux Enterprise Server 11
- SUSE Linux Enterprise Server 12
- SUSE Linux Enterprise Server 15
- SUSE Linux Enterprise Desktop 15

Issue:

Tests fail during feed downloads and others finished with setup errors in the VD suite wazuh-qa#4370

Description

The test_execd_firewall_drop test aims to verify if modulesd downloads successfully the feeds from different providers and os, and check that. the feed update date does not exceed a set limit. A timeout is raised like

>                   raise TimeoutError(error_message)
E                   TimeoutError: Could not find SUSE Linux Enterprise Server 12 feed finished log                  TimeoutError: TimeoutError was raised because a single 'added' event was expected for ['/dir/testfile.txt'] but was not detected.

It could be related to a download that exceeds the timeout

roronoasins · 2023-07-28T09:33:12Z

Syscollector 🔴

Details

Build https://ci.wazuh.info/job/Test_integration/41729/ https://ci.wazuh.info/job/Test_integration/41705/
Issue:

Syscollector scanning IT fails for Windows agents wazuh-qa#4230

Description

Syscollector IT module fails. It is because the monitoring file is not being initialized properly, leading to the issue above

pro-akim · 2023-07-28T11:43:53Z

Update

Moved to on hold,
Tests will be relaunched

pro-akim · 2023-07-31T16:34:21Z

LGTM

davidjiglesias · 2023-08-01T07:55:08Z

The conclusion is not updated in the main page. This is mandatory so the reviews can be performed in a fast and efficient manner.

davidjiglesias · 2023-08-01T10:01:22Z

LGTM!

roronoasins added type/test level/task labels Jul 27, 2023

roronoasins mentioned this issue Jul 27, 2023

Release 4.5.0 - Alpha 1 #18058

Closed

wazuhci added this to Release 4.5.0 Jul 27, 2023

wazuhci moved this to In progress in Release 4.5.0 Jul 27, 2023

roronoasins self-assigned this Jul 27, 2023

roronoasins mentioned this issue Jul 28, 2023

Tests fail during feed downloads and others finished with setup errors in the VD suite wazuh/wazuh-qa#4370

Closed

wazuhci moved this from In progress to Pending review in Release 4.5.0 Jul 28, 2023

wazuhci moved this from Pending review to In review in Release 4.5.0 Jul 28, 2023

wazuhci moved this from In review to In progress in Release 4.5.0 Jul 28, 2023

wazuhci moved this from In progress to Pending review in Release 4.5.0 Jul 31, 2023

wazuhci moved this from Pending review to Pending final review in Release 4.5.0 Jul 31, 2023

wazuhci moved this from Pending final review to On hold in Release 4.5.0 Aug 1, 2023

wazuhci moved this from On hold to Pending final review in Release 4.5.0 Aug 1, 2023

davidjiglesias closed this as completed Aug 1, 2023

wazuhci moved this from Pending final review to Done in Release 4.5.0 Aug 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release 4.5.0 - Alpha 1 - Integration tests #18114

Release 4.5.0 - Alpha 1 - Integration tests #18114

roronoasins commented Jul 27, 2023 •

edited

Loading

roronoasins commented Jul 27, 2023

roronoasins commented Jul 28, 2023 •

edited

Loading

roronoasins commented Jul 28, 2023 •

edited

Loading

roronoasins commented Jul 28, 2023

roronoasins commented Jul 28, 2023 •

edited

Loading

pro-akim commented Jul 28, 2023 •

edited

Loading

pro-akim commented Jul 31, 2023

davidjiglesias commented Aug 1, 2023

davidjiglesias commented Aug 1, 2023

Release 4.5.0 - Alpha 1 - Integration tests #18114

Release 4.5.0 - Alpha 1 - Integration tests #18114

Comments

roronoasins commented Jul 27, 2023 • edited Loading

Description

Tests Integration - Status

Main RC issue

References

Test Integration - Results

Evidence

Conclusion 🔴

Auditors validation

roronoasins commented Jul 27, 2023

roronoasins commented Jul 28, 2023 • edited Loading

FIM 🔴 (not valid tag used - it does not apply to this testing)

Agent

Details

Description

roronoasins commented Jul 28, 2023 • edited Loading

Logcollector 🔴

Agent

Details

Description

roronoasins commented Jul 28, 2023

Vulnerability Detector 🔴

Manager

Details

Description

roronoasins commented Jul 28, 2023 • edited Loading

Syscollector 🔴

Details

Description

pro-akim commented Jul 28, 2023 • edited Loading

Update

pro-akim commented Jul 31, 2023

davidjiglesias commented Aug 1, 2023

davidjiglesias commented Aug 1, 2023

roronoasins commented Jul 27, 2023 •

edited

Loading

roronoasins commented Jul 28, 2023 •

edited

Loading

roronoasins commented Jul 28, 2023 •

edited

Loading

roronoasins commented Jul 28, 2023 •

edited

Loading

pro-akim commented Jul 28, 2023 •

edited

Loading