-
Notifications
You must be signed in to change notification settings - Fork 13
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #152 from michalvavrik/feature/update-QUARKUS-874
Update test plan for QUARKUS-874
- Loading branch information
Showing
1 changed file
with
25 additions
and
19 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,17 +2,24 @@ | |
|
|
||
| JIRA link: https://issues.redhat.com/browse/QUARKUS-874 | ||
|
|
||
| The goal is to ensure some level of testing and identify existing limitations on Quarkus side for Windows. | ||
| The goal is to provide production support for JVM mode on Windows and verify that deployment from Windows to the OpenShift works. | ||
|
|
||
| ## Scope of the testing | ||
| The selected Test Suites will be executed on the selected Windows Platforms to ensure functionality is fully accomplished. Moreover, these Selected Test Suites might need adjustments for the correct execution on Windows. | ||
| ## Existing coverage | ||
| We already have Windows as part of the Quarkus QE Test Suite GitHub CI, where we run tests that do not require Linux containers. | ||
| As part of the extended platform trigger, we already run StartStop Special Chars tests, as well as the Quarkus QE Test Suite | ||
| in OpenShift (http-minimum module) using the OpenShift extension. The Quarkus Quickstarts modules that doesn't require | ||
| Linux containers are run as well. | ||
|
|
||
| In order to accommodate this in our current test suites, we need to rename existing jobs in Jenkins in order to differentiate them for those running on RHEL and on Windows platforms. | ||
| ## Scope of the testing | ||
| In addition to what we already test, our primary focus needs to be on bare-metal JVM tests and ensure we run | ||
| same tests as we run for the Red Hat Enterprise Linux operating system, except for extensions that require native | ||
| libraries (like the Apache Kafka Streams). | ||
|
|
||
| ### Selected Windows Platforms | ||
| - Windows Server 2019 for GitHub CI and Jenkins environment | ||
|
|
||
| NOTE: We will prepare steps to get Windows 10 instance to be able to handle situations when we need manual verification of specific fix for a customer case reproducible just on Windows 10. | ||
| - Windows Server 2019 for GitHub CI and Jenkins Start-Stop Test Suite and Test Suite JVM in OpenShift | ||
| - Windows Server 2022 for Jenkins jobs for weekly Test Suite runs and jobs that are part of RHBQ extended trigger: | ||
| - Quarkus Quickstarts in JVM mode on bare metal | ||
| - Quarkus QE Test Suite in JVM on bare metal | ||
|
|
||
| ### Selected Test Suites | ||
| - Start-Stop Test Suite | ||
|
|
@@ -23,26 +30,25 @@ NOTE: We will prepare steps to get Windows 10 instance to be able to handle situ | |
| - New Jenkins Jobs | ||
| - Combinations of Quarkus extensions Test Suite | ||
| - In Github CI - for Pull Requests and Daily runs | ||
| - OpenShift Test Suite | ||
| - New Jenkins Jobs on selected modules only (the ones using deployment strategies) | ||
|
|
||
| ### Platform testing | ||
| JVM mode on is the focus for Windows platform. This RFE does not include native executable (Mandrel) or any extension that requires native libraries (like Kafka streams client). | ||
| - Quarkus QE Test Suite | ||
| - In Github CI - for Pull Requests and Daily runs | ||
| - New weekly Jenkins Job running against Quarkus main | ||
|
|
||
| ## Impact on Jenkins | ||
| - New Jobs definitions | ||
| - Rename existing jobs to differentiate them for those running on RHEL and on Windows platforms | ||
| - New Jenkins AWS runners as our tests rely mostly on the TestContainers, we need Linux containers and hardware virtualization respectively, which is not possible in the RHOS-D. | ||
|
|
||
| ## Impact on resources | ||
| The major impact is on the PSI and Jenkins capacity: | ||
| - Windows instances will be executed next to the RHEL instances | ||
| - Overall test execution for RHEL and Windows jobs will be affected (doubled in the worse case) | ||
| - Windows virtual machines usually require more resources than RHEL ones, expecting large or xlarge flavor to be used | ||
| - More CPU / memory in the lab would be needed to ensure similar execution time as now. | ||
| - Remote Docker machines instances based on RHEL to run the containers with services like DBs / Keycloak / Kafka / Infinispan for Windows jobs | ||
| - Windows on PSI does not support Linux containers yet, so we want to try to use RHEL instances to run the containers as a workaround | ||
| - Windows instances will be created from RHEL instances, therefore we need at least one RHEL instance of medium flavor for each job. | ||
| - Windows instances that requires Docker will be run in AWS, therefore only impact to the current capacity is above-mentioned RHEL instance. | ||
| - Windows instances that do not require Docker will be executed next to the RHEL instances, but they are mostly lightweight short-running jobs | ||
| - Overall test execution for RHEL and Windows jobs will be affected (doubled in the worse case) | ||
| - More jobs pulling artifacts from Artifactory instance | ||
| - We may need to use more resources for Artifactory | ||
| - We may need to use more resources for Artifactory | ||
| - Weekly Test Suite run on AWS will require around 9.88 h per week. The execution time will be longer as our Test Suite will cover more topics over time. | ||
|
|
||
| ## Contacts | ||
| * Tester: Jose Carvajal <[email protected]> | ||
| * Tester: Michal Vavřík <[email protected]> | ||