Skip to content

Commit

Permalink
Chronos: PR validation job to Github action (#5349)
Browse files Browse the repository at this point in the history
Migrate current jenkins CI/CD jobs to github action, this is chronos PR validation-Spark2.4-ray job
  • Loading branch information
plusbang authored Aug 23, 2022
1 parent 8e170be commit 6b9d090
Showing 1 changed file with 99 additions and 0 deletions.
99 changes: 99 additions & 0 deletions .github/workflows/chronos-prvn-python-spark-2.4-py37-ray.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
name: Chronos PRVN Python Spark2.4 py37 ray

# Controls when the action will run.
on:
# Triggers the workflow on pull request events but only for the main branch
pull_request:
branches: [ main ]
paths:
- 'python/chronos/**'
- '.github/workflows/chronos-prvn-python-spark-2.4-py37-ray.yml'
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
chronos-PRVN-ray:
runs-on: [ self-hosted, Gondolin, ubuntu-20.04-lts ]
strategy:
fail-fast: false
matrix:
python-version: ["3.7"]
steps:
- uses: actions/checkout@v3
- name: Set up JDK 8
uses: actions/setup-java@v3
with:
java-version: '8'
distribution: 'temurin'
#server-id: github # Value of the distributionManagement/repository/id field of the pom.xml
settings-path: ${{ github.workspace }} # location for the settings.xml file

- name: Set up Maven
uses: stCarolas/[email protected]
with:
maven-version: 3.8.2

- name: Set up Maven Settings
uses: s4u/[email protected]
with:
sonatypeSnapshots: true
apacheSnapshots: true
servers: |
[{
"id": "central",
"configuration": {
"httpConfiguration": {
"all": {
"connectionTimeout": "3600000",
"readTimeout": "3600000"
}
}
}
}]
mirrors: '[{"id": "ardaNexus", "name": "ardaNexus", "mirrorOf": "*", "url": "${NEXUS_URL}" }]'

- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install --upgrade setuptools==58.0.4
python -m pip install --upgrade wheel
pip install pytest==5.4.1
pip install prophet==1.1
pip install pmdarima==1.8.4
pip install neural_compressor==1.11
pip install tsfresh==0.17.0
pip install ray==1.9.2 ray[tune]==1.9.2 ray[default]==1.9.2
pip install pyarrow==6.0.1
pip install ConfigSpace==0.5.0
pip install optuna==2.10.1
pip install onnxruntime==1.11.1
pip install onnx==1.11.0
apt-get update
apt-get install patchelf
- name: Run Chronos python test
shell: bash
run: |
pip uninstall -y bigdl-friesian bigdl-friesian-spark3 bigdl-dllib bigdl-dllib-spark3 bigdl-orca pyspark bigdl-orca-spark3 bigdl-chronos bigdl-chronos-spark3 bigdl-friesian bigdl-friesian-spark3
sed -i "s/'bigdl-core=='+VERSION/'bigdl-core==2.1.0b20220811'/g" python/dllib/src/setup.py
bash python/dev/release_default_linux_spark246.sh default false false
bash python/nano/dev/build_and_install.sh linux default false pytorch --force-reinstall --no-deps -U
whl_name=`ls python/nano/dist/`
pip install python/nano/dist/${whl_name}[tensorflow,pytorch]
pip install python/dllib/src/dist/bigdl_dllib-*-py3-none-manylinux1_x86_64.whl
pip install python/orca/src/dist/bigdl_orca-*-py3-none-manylinux1_x86_64.whl
pip install python/chronos/src/dist/bigdl_chronos-*-py3-none-manylinux1_x86_64.whl
export SPARK_LOCAL_HOSTNAME=localhost
export KERAS_BACKEND=tensorflow
python --version
pip install scipy==1.5.4
bash python/chronos/dev/test/run-pytests.sh
env:
ANALYTICS_ZOO_ROOT: ${{ github.workspace }}

0 comments on commit 6b9d090

Please sign in to comment.