Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEV - Update dev workflows and CI #801

Closed
wants to merge 18 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
- run: echo "Running on ${{ matrix.directory }}"

- name: "Check package build - ${{ matrix.directory }} 📦"
uses: hynek/build-and-inspect-python-package@main
uses: hynek/build-and-inspect-python-package@v2
with:
path: ${{ matrix.directory }}
upload-name-suffix: "-${{ matrix.directory }}"
Expand Down
61 changes: 27 additions & 34 deletions .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
@@ -1,35 +1,28 @@
name: Tests

env:
DEFAULT_PYTHON_VERSION: "3.10"
FORCE_COLOR: "1" # Make tools pretty.
trallard marked this conversation as resolved.
Show resolved Hide resolved
PYTHONUNBUFFERED: "1" # Make stdout and stderr behave well

on:
pull_request:
push:
branches:
- main


# ensuring only one instance is running at a given time
concurrency:
group: ${{ github.ref }}
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true

jobs:
test-conda-store-server:
name: "unit-test conda-store-server"
strategy:
trallard marked this conversation as resolved.
Show resolved Hide resolved
fail-fast: false
matrix:
os: ["ubuntu", "macos", "windows"]
include:
- os: ubuntu
environment-file: conda-store-server/environment-dev.yaml
- os: macos
environment-file: conda-store-server/environment-macos-dev.yaml
- os: windows
environment-file: conda-store-server/environment-windows-dev.yaml
runs-on: ${{ matrix.os }}-latest
os: ["ubuntu-latest", "macos-latest", "macos-14", "windows-latest"]
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash -el {0}
Expand All @@ -39,9 +32,9 @@ jobs:
uses: actions/checkout@v4

- name: "Set up env ${{ matrix.os }} 🐍"
trallard marked this conversation as resolved.
Show resolved Hide resolved
uses: conda-incubator/setup-miniconda@v2
uses: conda-incubator/setup-miniconda@v3
with:
environment-file: ${{ matrix.environment-file }}
environment-file: conda-store-server/environment-dev.yaml
miniforge-version: latest

# This fixes a "DLL not found" issue importing ctypes from the hatch env
Expand All @@ -51,20 +44,20 @@ jobs:
timeout_minutes: 9999
max_attempts: 6
command:
conda install --channel=conda-forge --quiet --yes python=${{ matrix.python }}
conda install --channel=conda-forge --quiet --yes python="3.10"
if: matrix.os == 'windows'

- name: "Linting Checks 🧹"
run: |
hatch env run -e lint lint

- name: "Build package 📦"
- name: "Install conda-store-server for tests 📦"
run: |
hatch build
python -m pip install .

- name: "Unit tests ✅"
run: |
pytest -m "not extended_prefix and not user_journey" tests
python -m pytest -m "not extended_prefix and not user_journey" tests

# https://github.com/actions/runner-images/issues/1052
- name: "Windows extended prefix unit tests ✅"
Expand All @@ -90,18 +83,18 @@ jobs:
- name: "Checkout Repository 🛎"
uses: actions/checkout@v4

- name: "Set up env 🐍"
- name: "Set up env ubuntu-latest 🐍"
uses: conda-incubator/setup-miniconda@v3
with:
environment-file: conda-store-server/environment-dev.yaml
miniforge-version: latest

- name: "Install build dependencies 📦"
run: |
pip install hatch
sudo apt update
sudo apt install wait-for-it -y

- name: "Deploy docker-compose"
- name: "Deploy docker-compose 🏗"
run: |
docker-compose up -d
docker ps
Expand All @@ -110,26 +103,29 @@ jobs:
wait-for-it localhost:9000 # minio
wait-for-it localhost:8080 # conda-store-server

- name: "Install conda-store-server for tests 📦"
run: |
python -m pip install .

- name: "Run Playwright tests 🎭"
run: |
playwright install
pytest --video on ../tests/test_playwright.py
playwright install --with-deps chromium
python -m pytest --video on ../tests/test_playwright.py

- name: "Upload test results 📤"
uses: actions/upload-artifact@v4
if: ${{ always() }}
with:
name: playwright-tests
path: conda-store-server/test-results
path: conda-store-server/test-results/

- name: "Run integration tests ✅"
run: |
export PYTHONPATH=$PYTHONPATH:$PWD
pytest ../tests/test_api.py ../tests/test_metrics.py
python -m pytest ../tests/test_api.py ../tests/test_metrics.py

- name: "Run user journey tests ✅"
run: |
pytest -m "user_journey"
python -m pytest -m "user_journey"

- name: "Get Docker logs 🔍"
if: ${{ failure() }}
Expand Down Expand Up @@ -157,16 +153,13 @@ jobs:
- name: "Install Dependencies 📦"
run: |
pip install hatch
sudo apt update
sudo apt install wait-for-it -y

- name: "Linting Checks 🧹"
run: |
hatch env run -e lint lint

- name: "Build package 📦"
run: |
hatch build

- name: "Deploy docker-compose"
run: |
docker-compose up -d
Expand All @@ -178,9 +171,9 @@ jobs:

- name: "Install conda-store for tests 📦"
run: |
pip install .
python -m pip install .

- name: "Run basic tests - not authenticated"
- name: "Run basic tests - not authenticated"
run: |
sleep 20
./tests/unauthenticated-tests.sh
Expand All @@ -203,7 +196,7 @@ jobs:
docker-compose logs

build-docker-image:
name: "Build docker images"
name: "Build Docker images 🐳"
runs-on: ubuntu-latest
strategy:
matrix:
Expand Down
63 changes: 16 additions & 47 deletions conda-store-server/environment-dev.yaml
Original file line number Diff line number Diff line change
@@ -1,63 +1,32 @@
# only development dependencies should be added to this file, if you add a new dev dependency
# you must ensure to add it to the [tool.hatch.envs.dev] section of the pyproject.toml file too
# required conda-store-server or conda-store (core) dependencies MUST ONLY be added/updated in the [dependencies]
# section of the corresponding pyproject.toml file
# the only exceptions to this rule are conda and constructor which are only conda installable
name: conda-store-server-dev
channels:
- conda-forge
- microsoft
- nodefaults
dependencies:
- python ==3.10
- python =3.10
# conda builds
- conda ==23.5.2
- conda-docker >= 0.1.2
- python-docker
- conda-pack
- conda-lock >=1.0.5
- conda-package-handling
- conda-package-streaming
# web server
- celery
- flower
- redis-py
- sqlalchemy<=1.4.47
- psycopg2
- pymysql
- requests
- uvicorn
- fastapi
- pydantic < 2.0
- pyyaml
- traitlets
- yarl
- pyjwt
- filelock
- itsdangerous
- jinja2
- python-multipart
- alembic
# artifact storage
- minio
# installer
- constructor
# CLI
- typer

- conda
# dev dependencies
- aiohttp>=3.8.1
- hatch
- pytest
- pytest-celery
- pytest-mock
- black ==22.3.0
- flake8
- ruff
- sphinx
- myst-parser
- sphinx-panels
- sphinx-copybutton
- pydata-sphinx-theme
- playwright
- docker-py<7 # for docker-compose
- docker-compose
# build dependencies
- hatch
- build
- twine>=5.0.0
- pkginfo>=1.10 # needed to support metadata 2.3
# installer
- constructor # runtime dep but must be conda installed
# linting
- pre_commit
- pip

- pip:
- pytest-playwright
63 changes: 0 additions & 63 deletions conda-store-server/environment-macos-dev.yaml

This file was deleted.

63 changes: 0 additions & 63 deletions conda-store-server/environment-windows-dev.yaml

This file was deleted.

Loading
Loading