Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add orca py37 UT to github action #5578

Merged
merged 5 commits into from
Aug 31, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .github/actions/orca-python-py37-spark3-action/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: 'Run Orca Python Py37 Spark3'
description: 'Run Orca Python Py37 Spark3'
runs:
using: "composite"
steps:
- name: Setup Env
shell: bash
run: |
apt-get update
apt-get install wget

- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.7'

- name: Install dependencies
shell: bash
run: |
python -m pip install --upgrade pip
python -m pip install --upgrade setuptools==58.0.4
pip uninstall -y bigdl-friesian bigdl-friesian-spark3 bigdl-dllib bigdl-dllib-spark3 bigdl-orca pyspark bigdl-orca-spark3 bigdl-chronos bigdl-chronos-spark3 bigdl-friesian bigdl-friesian-spark3
pip uninstall -r python/orca/dev/example/requirements-ray.txt -y
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r python/orca/dev/example/requirements-ray.txt
pip install pytest
pip uninstall -y opencv-python-headless
pip install opencv-python-headless
sed -i "s/'bigdl-core=='+VERSION/'bigdl-core==2.1.0b20220811'/g" python/dllib/src/setup.py
bash python/dev/release_default_linux_spark312.sh default false false false -Ddata-store-url=$HTTP_URI -U

# install dllib
pip install python/dllib/src/dist/bigdl_dllib*-py3-none-manylinux1_x86_64.whl

# install orca
pip install python/orca/src/dist/bigdl_orca*-py3-none-manylinux1_x86_64.whl

- name: Run Test
shell: bash
run: |
export SPARK_LOCAL_HOSTNAME=localhost

chmod a+x python/orca/dev/test/run-pytests-spark.sh
python/orca/dev/test/run-pytests-spark.sh
env:
BIGDL_ROOT: ${{ github.workspace }}
ANALYTICS_ZOO_ROOT: ${{ github.workspace }}
18 changes: 17 additions & 1 deletion .github/workflows/nightly_test.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
name: Nightly Test

on:
#pull_request:
schedule:
- cron: '0 13 * * *' # GMT time, 13:00 GMT == 21:00 China
workflow_dispatch:
Expand All @@ -16,11 +17,13 @@ on:
- Orca-Python-ExampleTest-Ray-Py37-Spark3
- Orca-Jep-ExampleTest-Py37-Spark2
- Orca-Python-Ray-Py37-Spark3
- Orca-Python-Py37-Spark3
- Orca-Ray-Ctx-Example
- Scala-App
- Dllib-Scala-UT
- Friesian-Scala-UT
- PPML-Scala-UT
- Orca-Ray-Ctx-Example


permissions:
contents: read
Expand Down Expand Up @@ -80,6 +83,19 @@ jobs:
- name: Run test
uses: ./.github/actions/orca-python-ray-py37-spark3-action

Orca-Python-Py37-Spark3:
if: ${{ github.event.schedule || github.event.inputs.artifact == 'Orca-Python-Py37-Spark3' || github.event.inputs.artifact == 'all' }}
runs-on: [self-hosted, Gondolin, ubuntu-20.04-lts]

steps:
- uses: actions/checkout@v3
- name: Set up JDK8
uses: ./.github/actions/jdk-setup-action
- name: Set up maven
uses: ./.github/actions/maven-setup-action
- name: Run test
uses: ./.github/actions/orca-python-py37-spark3-action

Scala-App:
if: ${{ github.event.schedule || github.event.inputs.artifact == 'Scala-App' || github.event.inputs.artifact == 'all' }}
runs-on: [self-hosted, Gondolin, ubuntu-20.04-lts]
Expand Down