Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Onboard prod jenkins docker images to github actions #591

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 0 additions & 68 deletions .github/workflows/cd.yml

This file was deleted.

118 changes: 79 additions & 39 deletions .github/workflows/gradle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,28 @@ on:
- "*"

jobs:
build_pa_pkg:
Get-CI-Image-Tag:
uses: opensearch-project/opensearch-build/.github/workflows/get-ci-image-tag.yml@main
with:
product: opensearch

build_pa_pkg_on_prod_docker:
needs: Get-CI-Image-Tag
strategy:
matrix:
java:
- 11
- 17
fail-fast: false
runs-on: [ubuntu-latest]
name: Building PA package
container:
# using the same image which is used by opensearch-build team to build the OpenSearch Distribution
# this image tag is subject to change as more dependencies and updates will arrive over time
image: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-version-linux }}
# need to switch to root so that github actions can install runner binary on container without permission issues.
options: --user root

name: Building PA package on prod docker
steps:
- name: Set up JDK
uses: actions/setup-java@v1
Expand All @@ -26,42 +39,69 @@ jobs:
# Performance Analyzer in ./tmp/performance-analyzer
- name: Checkout Performance Analyzer package
uses: actions/checkout@v2
with:
path: ./tmp/performance-analyzer
# Explicitly set the docker-compose program path so that our build scripts in RCA can run the program
# This is necessary because of the Github Actions environment and the workingDir of the Gradle environment
- name: Set docker-compose path
run: DOCKER_COMPOSE_LOCATION=$(which docker-compose)
# Set the vm.max_map_count system property to the minimum required to run OpenSearch
- name: Set vm.max_map_count
run: sudo sysctl -w vm.max_map_count=262144
- name: Build Performance Analyzer and run Unit Tests
working-directory: ./tmp/performance-analyzer
#with:
# path: ./tmp/performance-analyzer
- name: Build PA with same command on prod docker
#working-directory: ./tmp/performance-analyzer
run: |
#chown -R 1000:1000 `pwd`
./gradlew build -Dperformance-analyzer-rca.build=true \
-Dperformance-analyzer-rca.repo="https://github.com/opensearch-project/performance-analyzer-rca.git" \
-Dperformance-analyzer-rca.branch=main
- name: Assemble PA jar for BWC tests
working-directory: ./tmp/performance-analyzer
run: |
./gradlew assemble
version=`./gradlew properties -q | grep "opensearch_version:" | awk '{print $2}'`
IFS='-' read -r -a version_array <<< "$version"
plugin_version="${version_array[0]}.0"; for entry in ${version_array[@]:1}; do plugin_version+="-$entry"; done
echo $plugin_version
mkdir -p ./src/test/resources/org/opensearch/performanceanalyzer/bwc/$plugin_version
cp ./build/distributions/*.zip ./src/test/resources/org/opensearch/performanceanalyzer/bwc/$plugin_version
- name: Generate Jacoco coverage report
working-directory: ./tmp/performance-analyzer
run: ./gradlew jacocoTestReport
- name: Upload coverage report
working-directory: ./tmp/performance-analyzer
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
run: bash <(curl -s https://codecov.io/bash) -f ./build/reports/jacoco/test/jacocoTestReport.xml
- name: Run Integration Tests
working-directory: ./tmp/performance-analyzer
run: ./gradlew integTest -Dtests.enableIT -Dtests.useDockerCluster
- name: Run PerformanceAnalzyer Backwards Compatibility Tests
working-directory: ./tmp/performance-analyzer
run: ./gradlew bwcTestSuite -Dtests.security.manager=false
-Dperformance-analyzer-rca.repo="https://github.com/opensearch-project/performance-analyzer-rca.git" \
-Dperformance-analyzer-rca.branch=main

# build_pa_pkg:
# strategy:
# matrix:
# java:
# - 11
# - 17
# fail-fast: false
# runs-on: [ubuntu-latest]
# name: Building PA package
# steps:
# - name: Set up JDK
# uses: actions/setup-java@v1
# with:
# java-version: ${{matrix.java}}
# # Performance Analyzer in ./tmp/performance-analyzer
# - name: Checkout Performance Analyzer package
# uses: actions/checkout@v2
# with:
# path: ./tmp/performance-analyzer
# # Explicitly set the docker-compose program path so that our build scripts in RCA can run the program
# # This is necessary because of the Github Actions environment and the workingDir of the Gradle environment
# - name: Set docker-compose path
# run: DOCKER_COMPOSE_LOCATION=$(which docker-compose)
# # Set the vm.max_map_count system property to the minimum required to run OpenSearch
# - name: Set vm.max_map_count
# run: sudo sysctl -w vm.max_map_count=262144
# - name: Build Performance Analyzer and run Unit Tests
# working-directory: ./tmp/performance-analyzer
# run: |
# ./gradlew build -Dperformance-analyzer-rca.build=true \
# -Dperformance-analyzer-rca.repo="https://github.com/opensearch-project/performance-analyzer-rca.git" \
# -Dperformance-analyzer-rca.branch=main
# - name: Assemble PA jar for BWC tests
# working-directory: ./tmp/performance-analyzer
# run: |
# ./gradlew assemble
# version=`./gradlew properties -q | grep "opensearch_version:" | awk '{print $2}'`
# IFS='-' read -r -a version_array <<< "$version"
# plugin_version="${version_array[0]}.0"; for entry in ${version_array[@]:1}; do plugin_version+="-$entry"; done
# echo $plugin_version
# mkdir -p ./src/test/resources/org/opensearch/performanceanalyzer/bwc/$plugin_version
# cp ./build/distributions/*.zip ./src/test/resources/org/opensearch/performanceanalyzer/bwc/$plugin_version
# - name: Generate Jacoco coverage report
# working-directory: ./tmp/performance-analyzer
# run: ./gradlew jacocoTestReport
# - name: Upload coverage report
# working-directory: ./tmp/performance-analyzer
# env:
# CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
# run: bash <(curl -s https://codecov.io/bash) -f ./build/reports/jacoco/test/jacocoTestReport.xml
# - name: Run Integration Tests
# working-directory: ./tmp/performance-analyzer
# run: ./gradlew integTest -Dtests.enableIT -Dtests.useDockerCluster
# - name: Run PerformanceAnalzyer Backwards Compatibility Tests
# working-directory: ./tmp/performance-analyzer
# run: ./gradlew bwcTestSuite -Dtests.security.manager=false
Loading