Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add orca ray python test to github action #5550

Merged
merged 7 commits into from
Aug 26, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/actions/orca-python-ray-py37-spark3-action/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: 'Run Orca Python Ray Py37 Spark3'
description: 'Run Orca Python Ray Py37 Spark3'
runs:
using: "composite"
steps:
- name: Setup Env
shell: bash
run: |
apt-get update
apt-get install wget

- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.7'

- name: Install dependencies
shell: bash
run: |
python -m pip install --upgrade pip
python -m pip install --upgrade setuptools==58.0.4
pip uninstall -y bigdl-friesian bigdl-friesian-spark3 bigdl-dllib bigdl-dllib-spark3 bigdl-orca pyspark bigdl-orca-spark3 bigdl-chronos bigdl-chronos-spark3 bigdl-friesian bigdl-friesian-spark3
pip uninstall -r python/orca/dev/example/requirements-ray.txt -y
pip install -i https://pypi.org/simple --pre --upgrade bigdl-orca-spark3
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r python/orca/dev/example/requirements-ray.txt
pip install pytest
pip uninstall -y opencv-python-headless
pip install opencv-python-headless
- name: Run Test
shell: bash
run: |
export SPARK_LOCAL_HOSTNAME=localhost
chmod a+x python/orca/dev/test/run-pytests-ray
python/orca/dev/test/run-pytests-ray
env:
BIGDL_ROOT: ${{ github.workspace }}
ANALYTICS_ZOO_ROOT: ${{ github.workspace }}
15 changes: 15 additions & 0 deletions .github/workflows/nightly_test.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
name: Nightly Test

on:
#pull_request:
schedule:
- cron: '0 13 * * *' # GMT time, 13:00 GMT == 21:00 China
workflow_dispatch:
Expand All @@ -15,6 +16,7 @@ on:
- Orca-Python-ExampleTest-Py37-Spark3
- Orca-Python-ExampleTest-Ray-Py37-Spark3
- Orca-Jep-ExampleTest-Py37-Spark2
- Orca-Python-Ray-Py37-Spark3
- Scala-App
- Dllib-Scala-UT
- Friesian-Scala-UT
Expand Down Expand Up @@ -65,6 +67,19 @@ jobs:
- name: Run test
uses: ./.github/actions/orca-jep-exampletest-action

Orca-Python-Ray-Py37-Spark3:
if: ${{ github.event.schedule || github.event.inputs.artifact == 'Orca-Python-Ray-Py37-Spark3' || github.event.inputs.artifact == 'all' }}
runs-on: [self-hosted, Gondolin, ubuntu-20.04-lts]

steps:
- uses: actions/checkout@v3
- name: Set up JDK8
uses: ./.github/actions/jdk-setup-action
- name: Set up maven
uses: ./.github/actions/maven-setup-action
- name: Run test
uses: ./.github/actions/orca-python-ray-py37-spark3-action

Scala-App:
if: ${{ github.event.schedule || github.event.inputs.artifact == 'Scala-App' || github.event.inputs.artifact == 'all' }}
runs-on: [self-hosted, Gondolin, ubuntu-20.04-lts]
Expand Down
3 changes: 3 additions & 0 deletions python/orca/dev/example/requirements-ray.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,6 @@ argparse
Pillow
pyarrow==4.0.1
h5py==2.10.0
fsspec==2021.9.0
scikit-optimize==0.8.1
pandas==1.1.5