Skip to content

Commit

Permalink
[Test] Move Azure Guidance in CI (#1000)
Browse files Browse the repository at this point in the history
Move the Azure Guidance endpoint test to its own portion of the CI build. This spawns off another variant of the test workflow YAML (since it need variable and secret access). On the flip side, it does at least run all of the `model_integration` tests.
  • Loading branch information
riedgar-ms authored Aug 23, 2024
1 parent c8ab79d commit 89c1482
Showing 1 changed file with 37 additions and 3 deletions.
40 changes: 37 additions & 3 deletions .github/workflows/ci_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,6 @@ jobs:
- transformers_gemma2_9b_cpu
- transformers_gemma2_9b_gpu
- transformers_llama3_8b_cpu
- azure_guidance
# - transformers_llama3_8b_gpu Does not reliably work on GPU build machines

steps:
Expand Down Expand Up @@ -162,8 +161,6 @@ jobs:
- name: Run minimal tests for ${{ matrix.model }}
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
AZUREAI_GUIDANCE_ENABLED_URL: ${{ vars.AZUREAI_GUIDANCE_ENABLED_URL }}
AZUREAI_GUIDANCE_ENABLED_URL_KEY: ${{ secrets.AZUREAI_GUIDANCE_ENABLED_URL_KEY }}
run: |
pytest -vv --cov=guidance --cov-report=xml --cov-report=term-missing \
--selected_model ${{ matrix.model }} \
Expand All @@ -173,3 +170,40 @@ jobs:
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}



remote-endpoint-tests-linux-python:
strategy:
fail-fast: false # Don't cancel all on first failure
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]
model:
- "azure_guidance"
runs-on: Large_Linux
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
shell: bash
run: |
python -m pip install --upgrade pip
pip install pytest
pip install -e .[test]
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Run tests (except server)
shell: bash
env:
AZUREAI_GUIDANCE_ENABLED_URL: ${{ vars.AZUREAI_GUIDANCE_ENABLED_URL }}
AZUREAI_GUIDANCE_ENABLED_URL_KEY: ${{ secrets.AZUREAI_GUIDANCE_ENABLED_URL_KEY }}
run: |
pytest --cov=guidance --cov-report=xml --cov-report=term-missing \
--selected_model ${{ matrix.model }} \
./tests/unit ./tests/model_integration ./tests/model_specific
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}

0 comments on commit 89c1482

Please sign in to comment.