Onboard to bundle bwc tests with feature flag -PcustomDistributionDow… (

#575) * Onboard to bundle bwc tests with feature flag -PcustomDistributionDownloadType=bundle. Signed-off-by: Finn Carroll <[email protected]> * Move multi-node-test-workflow to bundled bwc test suite. Signed-off-by: Finn Carroll <[email protected]> * Remove lingering local prev version plugins. Signed-off-by: Finn Carroll <[email protected]> * Strip SNAPSHOT tag from bundle versions. Signed-off-by: Finn Carroll <[email protected]> * Add add bundle bwc flag to readme. Signed-off-by: Finn Carroll <[email protected]> * Remove bundled run from repo workflow. Signed-off-by: Finn Carroll <[email protected]> * Update readme, exclude but note bundle dist level tests. Signed-off-by: Finn Carroll <[email protected]> * Non functional formatting/code cleanliness. Signed-off-by: Finn Carroll <[email protected]> --------- Signed-off-by: Finn Carroll <[email protected]>
opensearch-project · May 28, 2024 · e7056f6 · e7056f6
1 parent f7a40bb
commit e7056f6
Show file tree

Hide file tree

Showing 4 changed files with 78 additions and 33 deletions.
diff --git a/README.md b/README.md
@@ -66,20 +66,28 @@ The project in this package uses the [Gradle](https://docs.gradle.org/current/us
 
 ### Building from the command line
 
-1. `./gradlew build` builds and tests project.
-2. `./gradlew run` launches a single node cluster with the asynchronous search plugin installed.
-3. `./gradlew run -PnumNodes=3` launches a multi-node cluster with the asynchronous search plugin installed.
-4. `./gradlew integTest` launches a single node cluster with the asynchronous search plugin installed and runs all integ tests.
-5. `./gradlew integTest -PnumNodes=3` launches a multi-node cluster with the asynchronous search plugin installed and runs all integ tests.
-6. `./gradlew integTest -Dtests.class=*AsynchronousSearchRestIT` runs a single integ class
-7. `./gradlew integTest -Dtests.class=*AsynchronousSearchRestIT -Dtests.method="testSubmitWithRetainedResponse"` runs a single integ test method (remember to quote the test method name if it contains spaces)
-8. `./gradlew asynSearchCluster#mixedClusterTask -Dtests.security.manager=false` launches a cluster of three nodes of bwc version of OpenSearch with async search plugin and tests backwards compatibility by performing rolling upgrade of one node with the current version of OpenSearch with async search plugin.
-9. `./gradlew asynSearchCluster#rollingUpgradeClusterTask -Dtests.security.manager=false` launches a cluster with three nodes of bwc version of OpenSearch with async search plugin and tests backwards compatibility by performing rolling upgrade of all nodes with the current version of OpenSearch with async search plugin.
-10. `./gradlew asynSearchCluster#fullRestartClusterTask -Dtests.security.manager=false` launches a cluster with three nodes of bwc version of OpenSearch with async search plugin and tests backwards compatibility by performing a full restart on the cluster upgrading all the nodes with the current version of OpenSearch with async search plugin.
-11. `./gradlew bwcTestSuite -Dtests.security.manager=false` runs all the above bwc tests combined.
-12. `./gradlew integTestRemote -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="docker-cluster" -Dhttps=true -Duser=admin -Dpassword=admin` launches integration tests against a local cluster and run tests with security`./gradlew integTestRemote -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="docker-cluster" -Dhttps=true -Duser=admin -Dpassword=<admin-password>` launches integration tests against a local cluster and run tests with security
-
-When launching a cluster using one of the above commands, logs are placed in `build/testclusters/integTest-0/logs`. Though the logs are teed to the console, in practices it's best to check the actual log file.
+- `./gradlew build` builds and tests project.
+- `./gradlew run` launches a single node cluster with the asynchronous search plugin installed.
+- `./gradlew run -PnumNodes=3` launches a multi-node cluster with the asynchronous search plugin installed.
+
+### Integration tests
+
+- `./gradlew integTest` launches a single node cluster with the asynchronous search plugin installed and runs all integ tests.
+- `./gradlew integTest -PnumNodes=3` launches a multi-node cluster with the asynchronous search plugin installed and runs all integ tests.
+- `./gradlew integTest -Dtests.class=*AsynchronousSearchRestIT` runs a single integ class
+- `./gradlew integTest -Dtests.class=*AsynchronousSearchRestIT -Dtests.method="testSubmitWithRetainedResponse"` runs a single integ test method (remember to quote the test method name if it contains spaces)
+- `./gradlew integTestRemote -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="docker-cluster" -Dhttps=true -Duser=admin -Dpassword=admin` launches integration tests against a local cluster and run tests with security`./gradlew integTestRemote -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="docker-cluster" -Dhttps=true -Duser=admin -Dpassword=<admin-password>` launches integration tests against a local cluster and run tests with security
+
+### Backwards compatibility tests
+
+Note that while all backwards compatibility tests support the `-PcustomDistributionDownloadType=bundle` flag for testing with on the distribution bundle level, **only** changes previously published to the bundled distribution will be captured by these tests.
+
+- `./gradlew asynSearchCluster#mixedClusterTask -Dtests.security.manager=false` launches a cluster of three nodes of bwc version of OpenSearch with async search plugin and tests backwards compatibility by performing rolling upgrade of one node with the current version of OpenSearch with async search plugin.
+- `./gradlew asynSearchCluster#rollingUpgradeClusterTask -Dtests.security.manager=false` launches a cluster with three nodes of bwc version of OpenSearch with async search plugin and tests backwards compatibility by performing rolling upgrade of all nodes with the current version of OpenSearch with async search plugin.
+- `./gradlew asynSearchCluster#fullRestartClusterTask -Dtests.security.manager=false` launches a cluster with three nodes of bwc version of OpenSearch with async search plugin and tests backwards compatibility by performing a full restart on the cluster upgrading all the nodes with the current version of OpenSearch with async search plugin.
+- `./gradlew bwcTestSuite -Dtests.security.manager=false` runs all the above bwc tests combined.
+
+When launching a cluster using one of the above commands, logs are placed in `build/testclusters/<node-identifier>/logs`. Though the logs are tied to the console, in practices it's best to check the actual log file.
 
 ### Debugging
 

diff --git a/build.gradle b/build.gradle
@@ -5,6 +5,7 @@
 import java.util.concurrent.Callable
 import org.opensearch.gradle.testclusters.StandaloneRestIntegTestTask
 import org.opensearch.gradle.test.RestIntegTestTask
+
 buildscript {
     ext {
         distribution = 'oss-zip'
@@ -35,6 +36,7 @@ buildscript {
         classpath "org.jacoco:org.jacoco.agent:0.8.11"
     }
 }
+
 //****************************************************************************/
 // Build configurations
 //****************************************************************************/
@@ -258,33 +260,51 @@ ext.getPluginResource = { download_to_folder, download_from_src ->
     return fileTree(download_to_folder).getSingleFile()
 }
 
+Boolean bwcBundleTest = (project.findProperty('customDistributionDownloadType') != null &&
+        project.properties['customDistributionDownloadType'] == "bundle");
+
+/*
+When testing between major versions note that ONLY the latest minor release of the previous version is api compatible. 
+*/
+
+String bwcVersion = "2.15.0.0"
+String bwcVersionShort = bwcVersion.replaceAll(/\.0+$/, "")
+String nxtVersionShort = opensearch_version.replaceAll(/\.0+$/, "").replaceAll("-SNAPSHOT", "")
+
 String baseName = "asynSearchCluster"
-String bwcVersionShort = "2.15.0"
-String bwcVersion = bwcVersionShort + ".0"
 String bwcFilePath = "src/test/resources/org/opensearch/search/asynchronous/bwc/"
-String bwcRemoteFile = "https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/"+ bwcVersionShort + "/latest/linux/x64/tar/builds/opensearch/plugins/opensearch-asynchronous-search-"+ bwcVersion +".zip"
+String bwcRemoteFile = "https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/" + bwcVersionShort + "/latest/linux/x64/tar/builds/opensearch/plugins/opensearch-asynchronous-search-" + bwcVersion + ".zip"
+
 // Creates two test clusters of previous version and loads opensearch plugin of bwcVersion
 2.times { i ->
     testClusters {
         "${baseName}$i" {
             testDistribution = "ARCHIVE"
-            versions = [bwcVersionShort,opensearch_version]
+            versions = [bwcVersionShort, nxtVersionShort]
             numberOfNodes = 3
-            plugin(provider(new Callable<RegularFile>() {
-                @Override
-                RegularFile call() throws Exception {
-                    return new RegularFile() {
-                        @Override
-                        File getAsFile() {
-                            if (new File(bwcFilePath + bwcVersion).exists()) {
-                                project.delete(files(bwcFilePath + bwcVersion))
+
+            if (!bwcBundleTest) {
+                plugin(provider(new Callable<RegularFile>() {
+                    @Override
+                    RegularFile call() throws Exception {
+                        return new RegularFile() {
+                            @Override
+                            File getAsFile() {
+                                if (new File(bwcFilePath + bwcVersion).exists()) {
+                                    project.delete(files(bwcFilePath + bwcVersion))
+                                }
+                                getPluginResource(bwcFilePath + bwcVersion, bwcRemoteFile)
+                                return fileTree(bwcFilePath + bwcVersion).getSingleFile()
                             }
-                            getPluginResource(bwcFilePath + bwcVersion, bwcRemoteFile)
-                            return fileTree(bwcFilePath + bwcVersion).getSingleFile()
                         }
                     }
+                }))
+            } else {
+                nodes.each { node ->
+                    node.setting("plugins.security.disabled", "true")
                 }
-            }))
+            }
+
             setting 'path.repo', "${buildDir}/cluster/shared/repo/${baseName}"
             setting 'http.content_type.required', 'true'
             systemProperty "java.library.path", "$rootDir/src/test/resources/org/opensearch/search/asynchronous/lib:$rootDir/jni/release"
@@ -340,7 +360,11 @@ task "${baseName}#mixedClusterTask"(type: StandaloneRestIntegTestTask) {
     useCluster testClusters."${baseName}0"
     dependsOn "${baseName}#oldVersionClusterTask0"
     doFirst {
-        testClusters."${baseName}0".upgradeNodeAndPluginToNextVersion(plugins)
+        if (bwcBundleTest){
+            testClusters."${baseName}0".nextNodeToNextVersion()
+        } else {
+            testClusters."${baseName}0".upgradeNodeAndPluginToNextVersion(plugins)
+        }
     }
     filter {
         includeTestsMatching "org.opensearch.search.asynchronous.bwc.*IT"
@@ -350,14 +374,19 @@ task "${baseName}#mixedClusterTask"(type: StandaloneRestIntegTestTask) {
     nonInputProperties.systemProperty('tests.rest.cluster', "${-> testClusters."${baseName}0".allHttpSocketURI.join(",")}")
     nonInputProperties.systemProperty('tests.clustername', "${-> testClusters."${baseName}0".getName()}")
 }
+
 // Upgrades the second node to new OpenSearch version with upgraded plugin version after the first node is upgraded.
 // This results in a mixed cluster with 1 node on the old version and 2 upgraded nodes.
 // This is used for rolling upgrade.
 task "${baseName}#twoThirdsUpgradedClusterTask"(type: StandaloneRestIntegTestTask) {
     dependsOn "${baseName}#mixedClusterTask"
     useCluster testClusters."${baseName}0"
     doFirst {
-        testClusters."${baseName}0".upgradeNodeAndPluginToNextVersion(plugins)
+        if (bwcBundleTest){
+            testClusters."${baseName}0".nextNodeToNextVersion()
+        } else {
+            testClusters."${baseName}0".upgradeNodeAndPluginToNextVersion(plugins)
+        }
     }
     filter {
         includeTestsMatching "org.opensearch.search.asynchronous.bwc.*IT"
@@ -375,7 +404,11 @@ task "${baseName}#rollingUpgradeClusterTask"(type: StandaloneRestIntegTestTask)
     dependsOn "${baseName}#twoThirdsUpgradedClusterTask"
     useCluster testClusters."${baseName}0"
     doFirst {
-        testClusters."${baseName}0".upgradeNodeAndPluginToNextVersion(plugins)
+        if (bwcBundleTest){
+            testClusters."${baseName}0".nextNodeToNextVersion()
+        } else {
+            testClusters."${baseName}0".upgradeNodeAndPluginToNextVersion(plugins)
+        }
     }
     filter {
         includeTestsMatching "org.opensearch.search.asynchronous.bwc.*IT"
@@ -393,7 +426,11 @@ task "${baseName}#fullRestartClusterTask"(type: StandaloneRestIntegTestTask) {
     dependsOn "${baseName}#oldVersionClusterTask1"
     useCluster testClusters."${baseName}1"
     doFirst {
-        testClusters."${baseName}1".upgradeAllNodesAndPluginsToNextVersion(plugins)
+        if (bwcBundleTest){
+            testClusters."${baseName}1".nextNodeToNextVersion()
+        } else {
+            testClusters."${baseName}1".upgradeNodeAndPluginToNextVersion(plugins)
+        }
     }
     filter {
         includeTestsMatching "org.opensearch.search.asynchronous.bwc.*IT"

diff --git a/...g/opensearch/search/asynchronous/bwc/2.11.0.0/opensearch-asynchronous-search-2.11.0.0.zip b/...g/opensearch/search/asynchronous/bwc/2.11.0.0/opensearch-asynchronous-search-2.11.0.0.zip
diff --git a/...g/opensearch/search/asynchronous/bwc/2.12.0.0/opensearch-asynchronous-search-2.12.0.0.zip b/...g/opensearch/search/asynchronous/bwc/2.12.0.0/opensearch-asynchronous-search-2.12.0.0.zip