Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CI #772

Merged
merged 14 commits into from
Jan 30, 2024
Merged

Update CI #772

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
d4c9c72
Update checkout action
IsabelParedes Jan 23, 2024
d3b1de3
Update micromamba setup
IsabelParedes Jan 23, 2024
4995682
Clean up
IsabelParedes Jan 23, 2024
e01f102
Change env name
IsabelParedes Jan 23, 2024
155ce00
Remove windows files for testing ci
IsabelParedes Jan 23, 2024
5be513d
Set env to ci
IsabelParedes Jan 23, 2024
bb16778
Update boa command
IsabelParedes Jan 24, 2024
c9b24e6
Test build all
IsabelParedes Jan 24, 2024
d8825d1
Pin ruamel.yaml to v0.17
IsabelParedes Jan 24, 2024
56ca7ef
Relax pin
IsabelParedes Jan 24, 2024
96da331
Set env name to ci
IsabelParedes Jan 24, 2024
7781764
Fix attribute error with CondaBuildSpec
IsabelParedes Jan 24, 2024
ab24087
Remove duplicates
IsabelParedes Jan 24, 2024
2ba4760
Set build all to run on schedule only
IsabelParedes Jan 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 24 additions & 30 deletions .github/workflows/build_all.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,76 +16,70 @@ jobs:
emsdk_ver: ["3.1.45"]

steps:

################################################################
# SETUP
################################################################
- name: Checkout repo
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: delete fortran...
run: |
sudo apt-get remove gfortran -y
- name: Delete fortran
run: sudo apt-get remove gfortran -y

################################################################
# CONFIG
################################################################
- name: global config
- name: Global config
shell: bash -el {0}
run: |
git config --global advice.detachedHead false
cp $GITHUB_WORKSPACE/.github/workflows/.condarc $HOME/.condarc
cp $GITHUB_WORKSPACE/conda_build_config.yaml $HOME
run: git config --global advice.detachedHead false

################################################################
# MAMBA
################################################################
- uses: mamba-org/setup-micromamba@v1
- name: Install micromamba
uses: mamba-org/setup-micromamba@v1
with:
environment-file: ci_env.yml
environment-name: ci
init-shell: >-
bash
init-shell: bash
cache-environment: true
post-cleanup: 'all'

################################################################
# POST ENV INSTALL CONFIG
################################################################
- name: post env install config
- name: Post env install config
shell: bash -el {0}
run: |
cp $GITHUB_WORKSPACE/.github/workflows/.condarc $HOME/.condarc
cp $GITHUB_WORKSPACE/.github/workflows/.condarc $HOME
cp $GITHUB_WORKSPACE/conda_build_config.yaml $HOME
playwright install

################################################################
# setup emsdk
# EMSDK
################################################################
- name: "setup emsdk"
- name: Setup emsdk
shell: bash -el {0}
run: |
./emsdk/setup_emsdk.sh ${{ matrix.emsdk_ver }} $(pwd)/emsdk_install


./emsdk/setup_emsdk.sh ${{ matrix.emsdk_ver }} $(pwd)/emsdk_install

################################################################
# install / clone custom non-master things
# Install / clone custom non-master things
# for pip, we use --no-deps --ignore-installed
# and make sure that all dependencies are already contained
# in the conda env named "ci-env"
################################################################
- name: "install custom non-master dependencies"
- name: Install custom boa
shell: bash -el {0}
run: |
micromamba activate ci-env
python -m pip install git+https://github.com/DerThorsten/boa.git@python_api_v2 --no-deps --ignore-installed

python -m pip install git+https://github.com/DerThorsten/boa.git@python_api_v2 --no-deps --ignore-installed

################################################################
# build ALL recipes
# Build ALL recipes
################################################################
- name: "build packages workflow_dispatch"
- name: Build ALL packages
shell: bash -el {0}
run: |
cd ${GITHUB_WORKSPACE}
python builder.py build directory $GITHUB_WORKSPACE/recipes/recipes --skip-tests
python builder.py build directory $GITHUB_WORKSPACE/recipes/recipes_emscripten --emscripten-wasm32
python builder.py build directory $GITHUB_WORKSPACE/recipes/recipes --skip-tests
python builder.py build directory $GITHUB_WORKSPACE/recipes/recipes_emscripten --emscripten-wasm32
88 changes: 37 additions & 51 deletions .github/workflows/build_recipes.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,132 +18,118 @@ jobs:
emsdk_ver: ["3.1.45"]

steps:
################################################################
# SETUP
################################################################
- name: Checkout repo
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
fetch-depth: 0



- name: Maximize build space
run: |
sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf "/usr/local/share/boost"
sudo rm -rf "$AGENT_TOOLSDIRECTORY"


- name: delete fortran...
run: |
sudo apt-get remove gfortran -y
- name: Delete fortran
run: sudo apt-get remove gfortran -y

################################################################
# CONFIG
################################################################
- name: global config
- name: Global config
shell: bash -el {0}
run: |
git config --global advice.detachedHead false
cp $GITHUB_WORKSPACE/.github/workflows/.condarc $HOME
cp $GITHUB_WORKSPACE/conda_build_config.yaml $HOME
run: git config --global advice.detachedHead false

################################################################
# MAMBA
################################################################
- uses: mamba-org/setup-micromamba@v1
- name: Install micromamba
uses: mamba-org/setup-micromamba@v1
with:
environment-file: ci_env.yml
environment-name: ci
init-shell: >-
bash
init-shell: bash
cache-environment: true
post-cleanup: 'all'

################################################################
# setup emsdk
# EMSDK
################################################################
- name: "setup emsdk"
- name: Setup emsdk
shell: bash -el {0}
run: |
./emsdk/setup_emsdk.sh ${{ matrix.emsdk_ver }} $(pwd)/emsdk_install
./emsdk/setup_emsdk.sh ${{ matrix.emsdk_ver }} $(pwd)/emsdk_install

################################################################
# POST ENV INSTALL CONFIG
################################################################
- name: post env install config
- name: Post env install config
shell: bash -el {0}
run: |
cp $GITHUB_WORKSPACE/.github/workflows/.condarc $HOME/.condarc
cp $GITHUB_WORKSPACE/.github/workflows/.condarc $HOME
cp $GITHUB_WORKSPACE/conda_build_config.yaml $HOME
playwright install



################################################################
# install / clone custom non-master things
# Install / clone custom non-master things
# for pip, we use --no-deps --ignore-installed
# and make sure that all dependencies are already contained
# in the conda env named "ci-env"
################################################################
- name: "install custom non-master dependencies"
- name: Install custom boa
shell: bash -el {0}
run: |
python -m pip install git+https://github.com/DerThorsten/boa.git@python_api_v2 --no-deps --ignore-installed
python -m pip install git+https://github.com/DerThorsten/boa.git@python_api_v2 --no-deps --ignore-installed

################################################################
# run pytests
# PYTESTS
################################################################
- name: "test config files"
- name: Test config files
shell: bash -el {0}
run: |
cd testing
pytest test_empack_config.py
run: pytest test_empack_config.py
working-directory: testing

# ################################################################
# # build pyjs
# # PYJS
# ################################################################
# - name: "build package"
# - name: Build pyjs
# shell: bash -el {0}
# run: |
# cd ${GITHUB_WORKSPACE}
# python builder.py build explicit $GITHUB_WORKSPACE/recipes/recipes/emscripten_emscripten-wasm32
# python builder.py build explicit $GITHUB_WORKSPACE/recipes/recipes_emscripten/pyjs --emscripten-wasm32


# python builder.py build explicit $GITHUB_WORKSPACE/recipes/recipes_emscripten/pyjs --emscripten-wasm32

################################################################
# build recipes with changes PUSH
# PUSH
################################################################
- name: "build packages push"
- name: Build recipes with changes PUSH
shell: bash -el {0}
if: github.event_name == 'push'
run: |
cd ${GITHUB_WORKSPACE}
python builder.py build changed $GITHUB_WORKSPACE/ origin/main~1 origin/main


################################################################
# build recipes with changes PULL_REQUEST
# PULL_REQUEST
################################################################
- name: "build packages pull_request"
- name: Build recipes with changes PULL_REQUEST
shell: bash -el {0}
if: github.event_name == 'pull_request'
run: |
cd ${GITHUB_WORKSPACE}
python builder.py build changed $GITHUB_WORKSPACE/ origin/main HEAD

python builder.py build changed $GITHUB_WORKSPACE/ origin/main HEAD

################################################################
# quetz upload packages
# UPLOAD
################################################################
- name: "quetz upload packages"
- name: Upload packages to Quetz
if: github.event_name == 'push'
shell: bash -el {0}
run: |

mkdir -p ${CONDA_PREFIX}/conda-bld/emscripten-wasm32
mkdir -p ${CONDA_PREFIX}/conda-bld/linux-64
mkdir -p ${CONDA_PREFIX}/conda-bld/noarch



if [ $(ls ${CONDA_PREFIX}/conda-bld/emscripten-wasm32/*.tar.bz2 | wc -l) -ne 0 ]; then
QUETZ_API_KEY=${{ secrets.QUETZ_API_KEY}} quetz-client https://beta.mamba.pm/channels/emscripten-forge ${CONDA_PREFIX}/conda-bld/emscripten-wasm32/*.tar.bz2
fi
Expand Down
51 changes: 24 additions & 27 deletions .github/workflows/build_simple.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,77 +19,74 @@ jobs:
emsdk_ver: ["3.1.45"]

steps:

################################################################
# SETUP
################################################################
- name: Checkout repo
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
fetch-depth: 0

################################################################
# CONFIG
################################################################
- name: global config
- name: Global config
shell: bash -el {0}
run: |
git config --global advice.detachedHead false
cp $GITHUB_WORKSPACE/.github/workflows/.condarc $HOME/.condarc
cp $GITHUB_WORKSPACE/conda_build_config.yaml $HOME
run: git config --global advice.detachedHead false

################################################################
# MAMBA
################################################################
- uses: mamba-org/setup-micromamba@v1
- name: Install micromamba
uses: mamba-org/setup-micromamba@v1
with:
environment-file: ci_env.yml
environment-name: ci
init-shell: >-
bash
init-shell: bash
cache-environment: true
post-cleanup: 'all'

################################################################
# POST ENV INSTALL CONFIG
################################################################
- name: post env install config
- name: Post env install config
shell: bash -el {0}
run: |
cp $GITHUB_WORKSPACE/.github/workflows/.condarc $HOME/.condarc
cp $GITHUB_WORKSPACE/.github/workflows/.condarc $HOME
cp $GITHUB_WORKSPACE/conda_build_config.yaml $HOME
playwright install

################################################################
# setup emsdk
# EMSDK
################################################################
- name: "setup emsdk"
- name: Setup emsdk
shell: bash -el {0}
run: |
./emsdk/setup_emsdk.sh ${{ matrix.emsdk_ver }} $(pwd)/emsdk_install
./emsdk/setup_emsdk.sh ${{ matrix.emsdk_ver }} $(pwd)/emsdk_install

################################################################
# install / clone custom non-master things
# Install / clone custom non-master things
# for pip, we use --no-deps --ignore-installed
# and make sure that all dependencies are already contained
# in the conda env named "ci-env"
################################################################
- name: "install custom non-master dependencies"
- name: Install custom boa
shell: bash -el {0}
run: |
python -m pip install git+https://github.com/DerThorsten/boa.git@python_api_v2 --no-deps --ignore-installed

python -m pip install git+https://github.com/DerThorsten/boa.git@python_api_v2 --no-deps --ignore-installed

################################################################
# build a package which is **not** on emscripten forge
# Build a package which is **not** on emscripten forge
################################################################
- name: "build package"
- name: Build package not on emscripten-forge
shell: bash -el {0}
run: |
cd ${GITHUB_WORKSPACE}
python builder.py build explicit $GITHUB_WORKSPACE/testing/test_recipes/only_py_tests --emscripten-wasm32
python builder.py build explicit $GITHUB_WORKSPACE/testing/test_recipes/only_py_tests --emscripten-wasm32

################################################################
# build a simple package
# Build a simple package
################################################################
- name: "build package"
- name: Build regex
shell: bash -el {0}
run: |
cd ${GITHUB_WORKSPACE}
python builder.py build explicit $GITHUB_WORKSPACE/recipes/recipes_emscripten/regex --emscripten-wasm32
python builder.py build explicit $GITHUB_WORKSPACE/recipes/recipes_emscripten/regex --emscripten-wasm32
6 changes: 3 additions & 3 deletions .github/workflows/new_versions.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ jobs:
update_versions:
runs-on: [ubuntu-latest]
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

- name: install mamba
uses: mamba-org/provision-with-micromamba@main
- name: Install micromamba
uses: mamba-org/setup-micromamba@v1
with:
environment-file: tickbot.yml

Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ channels:
**5a** Build a package (simple version):
To build a package, run this from the root of the repository (replace `regex` with the package you want to build)
```bash
boa build --target=emscripten-wasm32 recipes/recipes_emscripten/regex
boa build --target-platform=emscripten-wasm32 recipes/recipes_emscripten/regex -m conda_build_config.yaml
```
This should work in principle, but will not run the python tests of the package.

Expand Down
Loading