Run Integ test in a secure opensearch cluster (#535)

* Run Integ test in secure opensearch cluster Signed-off-by: Varun Jain <[email protected]> Co-authored-by: Owais Kazi <[email protected]>
opensearch-project · Jan 16, 2024 · a6841cc · a6841cc
1 parent bd1dfbf
commit a6841cc
Show file tree

Hide file tree

Showing 5 changed files with 269 additions and 99 deletions.
diff --git a/.github/workflows/backwards_compatibility_tests_workflow.yml b/.github/workflows/backwards_compatibility_tests_workflow.yml
@@ -24,7 +24,7 @@ jobs:
       BWC_VERSION_RESTART_UPGRADE: ${{ matrix.bwc_version }}
 
     steps:
-      - name: Checkout NeuralSearch
+      - name: Checkout neural-search
         uses: actions/checkout@v1
 
       - name: Setup Java ${{ matrix.java }}
@@ -51,7 +51,7 @@ jobs:
       BWC_VERSION_ROLLING_UPGRADE: ${{ matrix.bwc_version }}
 
     steps:
-      - name: Checkout NeuralSearch
+      - name: Checkout neural-search
         uses: actions/checkout@v1
 
       - name: Setup Java ${{ matrix.java }}

diff --git a/.github/workflows/test_security.yml b/.github/workflows/test_security.yml
@@ -0,0 +1,50 @@
+name: Test neural-search on Secure Cluster
+on:
+  schedule:
+    - cron: '0 0 * * *'  # every night
+  push:
+    branches:
+      - "*"
+      - "feature/**"
+  pull_request:
+    branches:
+      - "*"
+      - "feature/**"
+
+jobs:
+  Get-CI-Image-Tag:
+    uses: opensearch-project/opensearch-build/.github/workflows/get-ci-image-tag.yml@main
+    with:
+      product: opensearch
+
+  integ-test-with-security-linux:
+    strategy:
+      matrix:
+        java: [11, 17, 21]
+
+    name: Run Integration Tests on Linux
+    runs-on: ubuntu-latest
+    needs: Get-CI-Image-Tag
+    container:
+      # using the same image which is used by opensearch-build team to build the OpenSearch Distribution
+      # this image tag is subject to change as more dependencies and updates will arrive over time
+      image: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-version-linux }}
+      # need to switch to root so that github actions can install runner binary on container without permission issues.
+      options: --user root
+
+    steps:
+      - name: Checkout neural-search
+        uses: actions/checkout@v1
+        with:
+          submodules: true
+
+      - name: Setup Java ${{ matrix.java }}
+        uses: actions/setup-java@v1
+        with:
+          java-version: ${{ matrix.java }}
+
+      - name: Run tests
+        # switching the user, as OpenSearch cluster can only be started as root/Administrator on linux-deb/linux-rpm/windows-zip.
+        run: |
+          chown -R 1000:1000 `pwd`
+          su `id -un 1000` -c "whoami && java -version && ./gradlew integTest -Dsecurity.enabled=true"
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -24,6 +24,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Fix Flaky test reported in #433 ([#533](https://github.com/opensearch-project/neural-search/pull/533))
 ### Infrastructure
 - BWC tests for Neural Search ([#515](https://github.com/opensearch-project/neural-search/pull/515))
+- Github action to run integ tests in secure opensearch cluster ([#535](https://github.com/opensearch-project/neural-search/pull/535))
 ### Documentation
 ### Maintenance
 ### Refactoring

diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md
@@ -131,6 +131,38 @@ curl localhost:9200
   "tagline" : "The OpenSearch Project: https://opensearch.org/"
 }
 ```
+
+Additionally, it is also possible to run a cluster with security enabled:
+```shell script
+./gradlew run -Dsecurity.enabled=true
+```
+
+By default, if `-Dsecurity.enabled=true` is passed the following defaults will be used: `https=true`, `user=admin` and
+`password=admin`.
+
+Then, to access the cluster, we can run
+```bash
+curl https://localhost:9200 --insecure -u admin:admin
+
+{
+  "name" : "integTest-0",
+  "cluster_name" : "integTest",
+  "cluster_uuid" : "kLsNk4JDTMyp1yQRqog-3g",
+  "version" : {
+    "distribution" : "opensearch",
+    "number" : "3.0.0-SNAPSHOT",
+    "build_type" : "tar",
+    "build_hash" : "9d85e566894ef53e5f2093618b3d455e4d0a04ce",
+    "build_date" : "2023-10-30T18:34:06.996519Z",
+    "build_snapshot" : true,
+    "lucene_version" : "9.8.0",
+    "minimum_wire_compatibility_version" : "2.12.0",
+    "minimum_index_compatibility_version" : "2.0.0"
+  },
+  "tagline" : "The OpenSearch Project: https://opensearch.org/"
+}
+```
+
 ### Run Multi-node Cluster Locally
 
 It can be useful to test and debug on a multi-node cluster. In order to launch a 3 node cluster with the neural-search plugin installed, run the following command:
@@ -145,6 +177,11 @@ In order to run the integration tests with a 3 node cluster, run this command:
 ./gradlew :integTest -PnumNodes=3
 ```
 
+Additionally, to run integration tests on multi nodes with security enabled, run
+```
+./gradlew :integTest -Dsecurity.enabled=true -PnumNodes=3
+```
+
 Integration tests can be run with remote cluster. For that run the following command and replace host/port/cluster name values with ones for the target cluster:
 
 ```