DTT1 - Execution time analysis #5191
Assignees
Labels
Comments
UpdateWorking first on #5125 |
Update1. Analyze results of issue #5125 and provide times in a single commentAllocation + All the agent tests (install,registration,restart,stop,uninstall) (no clean-up) VAGRANT (1 Manager + 12 agents)
AWS (1 Manager + 13 agents)
2. Launch new battery of test after fixing the bugs #5125 (comment)VAGRANT (Antonio localhost)
(Federico localhost)
(Antonio localhost)
AWS
3. Central components testsVAGRANT (Antonio localhost)
AWS
Note:Antonio LocalhostLenovo ThinkPad T14 Gen 1 Federico LocalhostLenovo ThinkPad L15 Gen 1 |
UpdateWaiting for some definitions to move to the next analysis. |
UpdateYAML
1 Manager 1 Agents1 Manager 6 Agents1 Manager 10 Agents1 Manager 13 Agents |
UpdateYAML
1 Managers 0 Agents6 Managers 0 Agents10 Managers 0 Agents13 Managers 0 Agents@pro-akim I modified the YAML input to fix some errors, I updated it, and here are the results:
|
UpdateSome error found in the provision: NoteThe EC2 is being provisioned but it is still not approachable by the provision, this error is related to that unavailability. The Allocation module will have to check the EC2 before moving to the next step, then this issue will be blocked until that update. Related issue: #5198 |
|
After the daily meeting today, it has been discussed the time results, these comment result times include the stack (Wazuh manager, Filebeat, Wazuh dashboard, and Wazuh indexer) but the Testing module is only testing the Wazuh manager The original idea of this issue was to check the flow using the Wazuh manager and Wazuh agent, we need to review this behavior as the flow should allow provisioning of the stack components individually, for example:
Those cases should contemplate installing the Filebeat, Wazuh dashboard, and Wazuh indexer (single or cluster), so the provision should generate certificates and copy them between nodes, also, the AIO deployment should be considered If any of those cases are not contemplated in the current development, we should create an issue and discuss if it should be added for DTT1 or DTT2 |
|
Moved this issue to DTT1 as we need to test the threading fix |
rauldpm
changed the title
|
Updated #5191 (comment) with AWS results Due to the thread bug issue, this issue will be edited so it reports the following data:
The parallel tests will have to report the same test cases (1-N, M-1) but for multiple threads {2, 4, 6, 8, 10), with this data, we will be able to determine which AWS instance type we need for the Jenkins CI Blocked as |
1 task
|
Moved ETA to 30/04/2024 as we need all DTT1 changes to test it correctly (main ETA 29/04/2024) |
|
Moved ETA to 02/05/2024 as we need time to test the main branch: #4495 (comment) |
TEST PLAN - ETA May 15Testing Plan Guideline
Test mix table and time metrics
Workflow filesAGENTSTest Agents workflow fileTBDPLATFORMS:
Agent platform not included in the tests
MANAGERSTest Managers workflow fileTBDPLATFORMS: (TBD)
AIOTest AIO workflow fileTBDPLATFORMS: (TBD)
|
This was referenced May 8, 2024
|
Issue unblocked, as the 17 June is a holiday in Argentina, the issue's ETA is moved to 18 June, we will do general testing (each test will launch all available systems), for each deployment type (Agents, clusters, etc) @fcaffieri please provide a log file for each test and an execution duration time for each test |
UpdateTestsAgent tests AWS --threads=4 Agent 4.7.4 live = False YAMLERROR details 🔴
|
UpdateTestsManagers tests AWS --threads=4 Manager 4.7.4 live = False YAMLERROR details 🔴Managers tests AWS no --threads Manager 4.7.4 live = False ERROR details 🔴 |
UpdateTestsCentral component tests AWS no --threads Manager 4.7.4 live = False YAMLERROR details 🔴
|
UpdateAfter considering some fixes in the yaml:
After changes 🔴 Wazuh Manager has been correctly installed Running test only with Windows EC2s and Ubuntu Server 🔴 |
Update statusThe reported errors are due to the following:
These were the errors reported, which are due to incorrect execution of the tests, since what is going to be tested with these tests are unreleased packages, in this case if the parameter live = False is used and for the released packages live = True and the latest released version must be used. |
TestAgent testingInput yaml:Full Log: Summary Result:
Status Windows testThe Windows tests failed because the test module could not connect to the VMs and the allocator could not raise them correctly. The DevOps team was contacted to validate why it could not raise the VMS, but the tests worked right. Manager testing:Input yaml:Full Log: Summary Result:
Central components testing:Test summary:
Input yaml:Full Log: |
UpdateAsking to DevOps team about Windows EC2's issueAfter a brief conversation with David Correa, it was decided to open an issue for analysis By changing pywinrm from |
TestAgent testingCommand: Input yaml:Full Log: Summary Result:
|
Execution timeAgents Test
Manager Test
Central component Test
|
|
LGTM |
|
LGTM |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
We need to perform an analysis of DTT1 functionality and execution times so that we have reference times. The idea of this issue is to collect information about the branch's status.
Related:
Tasks
The text was updated successfully, but these errors were encountered: