Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEV - Set a canonical default Python version (3.12) #844

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
6c5724d
Update dockerignore
trallard Jul 3, 2024
38492d1
Remove unnecessary files and simplify Dockerfile
trallard Jul 3, 2024
4d13734
Update conda-store Dockerfile
trallard Jul 3, 2024
4affcde
Add channel to install
trallard Jul 3, 2024
fafb023
Activate environments
trallard Jul 3, 2024
293504f
Ensure Docker images use the right python
trallard Jul 3, 2024
0ad1dad
Ensure we do not create cache dirs
trallard Jul 3, 2024
5d8d75c
Bump base image
trallard Jul 3, 2024
06af11e
Add missing conda install
trallard Jul 3, 2024
b3fd9b9
Move env to the top in Dockerfile
trallard Jul 3, 2024
1e3b74c
:wrench: Update ignore files
trallard Jul 4, 2024
7a4a7fb
Ensure __pycache__ is always ignored
trallard Jul 4, 2024
225ca45
Ensure GH workflows add the dockerignore file
trallard Jul 4, 2024
186c793
:bug: Ensure files are findable in GH actions
trallard Jul 4, 2024
6932b1a
Ensure we use bash to copy
trallard Jul 4, 2024
334ebe1
Merge branch 'trallard/update-dev-environments' into trallard/clean-d…
trallard Jul 4, 2024
7cf362a
🤦 Remove workspace env since we are in the root dir
trallard Jul 4, 2024
52d267e
:wrench: Add hadolint config
trallard Jul 4, 2024
9a1a443
Last touches before merge - comments and cleaning
trallard Jul 4, 2024
a3b7598
:fire: Remove yet another environment.yaml
trallard Jul 4, 2024
1849d00
Merge branch 'trallard/clean-docker' into trallard/default-python-ver…
trallard Jul 4, 2024
a9d2ba2
:wrench: Add default python version file
trallard Jul 4, 2024
f9e538e
:pushpin: Use default python version in GH actions workflows
trallard Jul 4, 2024
0ad03ec
:wrench: Set python version as Docker build arg in CI
trallard Jul 4, 2024
cf823f8
:heavy_minus_sign: Remove hardcoded Python version from Dockerfiles
trallard Jul 4, 2024
1cdf2eb
:construction_worker: Update more python versions in GH actions workf…
trallard Jul 4, 2024
e2cdeb8
:pencil2: Fix typo in miniconda-setup
trallard Jul 4, 2024
360a96c
Add comment about version sync
trallard Jul 4, 2024
b38548e
Merge remote-tracking branch 'origin/trallard/update-dev-environments…
trallard Jul 5, 2024
b6995ed
Update pre-commit
trallard Jul 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/build_docker_image.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ jobs:
- name: "Checkout Repository 🛎"
uses: actions/checkout@v4

- name: "Get project's default Python version 🏷️"
run: |
echo "PYTHON_VERSION_DEFAULT=$(cat .python-version-default)" >> $GITHUB_ENV

- name: "Retrieve secret from Vault 🗝"
uses: hashicorp/vault-action@v3
with:
Expand Down Expand Up @@ -83,3 +87,5 @@ jobs:
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
python_version=${{ env.PYTHON_VERSION_DEFAULT }}
18 changes: 14 additions & 4 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ permissions:
jobs:
# Always build & verify package.
build-package:
name: Build & verify package
name: "Build & verify package"
runs-on: ubuntu-latest
strategy:
matrix:
Expand Down Expand Up @@ -55,7 +55,7 @@ jobs:

# Upload to Test PyPI on every commit on main
release-test-pypi:
name: Publish in-dev to test.pypi.org
name: "Publish in-dev to test.pypi.org"
environment: release-test-pypi
if: github.repository_owner == 'conda-incubator' && github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
Expand All @@ -82,9 +82,9 @@ jobs:
print-hash: true
verbose: true


# Upload to PyPI on every tag/release
release-pypi:
name: Publish released package to pypi.org
name: "Publish released package to pypi.org"
environment: release-pypi
if: github.repository_owner == 'conda-incubator' && github.event_name == 'release' && startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
Expand Down Expand Up @@ -128,6 +128,15 @@ jobs:
- name: "Checkout Repository 🛎"
uses: actions/checkout@v4

- name: "Get project's default Python version 🏷️"
run: |
echo "PYTHON_VERSION_DEFAULT=$(cat .python-version-default)" >> $GITHUB_ENV

- name: "Copy .dockerignore"
run: |
cp .dockerignore ${{ matrix.docker-image }}/.dockerignore
shell: bash

- name: "Retrieve secret from Vault 🗝"
uses: hashicorp/vault-action@v2
with:
Expand Down Expand Up @@ -177,6 +186,7 @@ jobs:
file: "${{ matrix.docker-image }}/Dockerfile"
build-args: |
RELEASE_VERSION=${{github.ref_name}}
python_version=${{ env.PYTHON_VERSION_DEFAULT }}
tags: |
${{ steps.meta.outputs.tags }}
push: true
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/test_build_docker_image.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
name: "Test build Docker image"

env:
DEFAULT_PYTHON_VERSION: "3.10"
FORCE_COLOR: "1" # Make tools pretty.
PYTHONUNBUFFERED: "1" # Make stdout and stderr behave well

Expand Down Expand Up @@ -37,6 +36,10 @@ jobs:
- name: "Checkout Repository 🛎"
uses: actions/checkout@v4

- name: "Get project's default Python version 🏷️"
run: |
echo "PYTHON_VERSION_DEFAULT=$(cat .python-version-default)" >> $GITHUB_ENV

- name: "Set up Docker Buildx 🏗"
uses: docker/setup-buildx-action@v3

Expand Down Expand Up @@ -73,3 +76,5 @@ jobs:
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
python_version=${{ env.PYTHON_VERSION_DEFAULT }}
4 changes: 2 additions & 2 deletions .github/workflows/test_conda_store.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
name: "Test conda-store"

env:
DEFAULT_PYTHON_VERSION: "3.10"
FORCE_COLOR: "1" # Make tools pretty.
PYTHONUNBUFFERED: "1" # Make stdout and stderr behave well

Expand Down Expand Up @@ -41,7 +40,8 @@ jobs:
- name: "Set up Python 🐍"
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
python-version-file: .python-version-default
cache: "pip"

- name: "Install Dependencies 📦"
run: |
Expand Down
8 changes: 6 additions & 2 deletions .github/workflows/test_conda_store_server_integration.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
name: "Test conda-store-server (integration)"

env:
DEFAULT_PYTHON_VERSION: "3.10"
FORCE_COLOR: "1" # Make tools pretty.
PYTHONUNBUFFERED: "1" # Make stdout and stderr behave well

Expand Down Expand Up @@ -39,13 +38,18 @@ jobs:
- name: "Checkout Repository 🛎"
uses: actions/checkout@v4

- name: "Set up env 🐍"
- name: "Get project's default Python version 🏷️"
run: |
echo "PYTHON_VERSION_DEFAULT=$(cat .python-version-default)" >> $GITHUB_ENV

- name: "Set up conda env 🐍"
uses: conda-incubator/setup-miniconda@v3
with:
environment-file: conda-store-server/environment-dev.yaml
miniforge-version: latest
auto-activate-base: false
activate-environment: conda-store-server-dev
python-version: ${{ env.PYTHON_VERSION_DEFAULT }}

- name: "Install dependencies 📦"
run: |
Expand Down
14 changes: 9 additions & 5 deletions .github/workflows/test_conda_store_server_unit.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
name: "Test conda-store-server (unit)"

env:
# needs to keep the same as the conda-store-server/environment-dev.yaml
DEFAULT_PYTHON_VERSION: "3.10"
FORCE_COLOR: "1" # Make tools pretty.

on:
Expand Down Expand Up @@ -36,7 +34,8 @@ jobs:
- name: "Set up Python 🐍"
uses: actions/setup-python@v5
with:
python-version: ${{ env.DEFAULT_PYTHON_VERSION }}
python-version-file: .python-version-default
cache: "pip"

- name: "Install Dependencies 📦"
run: |
Expand All @@ -62,22 +61,27 @@ jobs:
- name: "Checkout Repository 🛎"
uses: actions/checkout@v4

- name: "Get project's default Python version 🏷️"
run: |
echo "PYTHON_VERSION_DEFAULT=$(cat .python-version-default)" >> $GITHUB_ENV

- name: "Set up env ${{ matrix.os }} 🐍"
uses: conda-incubator/setup-miniconda@v3
with:
environment-file: conda-store-server/environment-dev.yaml
miniforge-version: latest
auto-activate-base: false
activate-environment: conda-store-server-dev
python-version: ${{ env.PYTHON_VERSION_DEFAULT }}

# This fixes a "DLL not found" issue importing ctypes from the hatch env
- name: "Reinstall Python 3.10 on Windows runner"
- name: "Reinstall Python on Windows runner"
uses: nick-fields/[email protected]
with:
timeout_minutes: 9999
max_attempts: 6
command:
conda install --channel=conda-forge --quiet --yes python=${{ env.DEFAULT_PYTHON_VERSION }}
conda install --channel=conda-forge --quiet --yes python=${{ env.PYTHON_VERSION_DEFAULT }}
if: matrix.os == 'windows-latest'

- name: "Install conda-store server 📦"
Expand Down
3 changes: 1 addition & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,10 @@ repos:
rev: v4.6.0
hooks:
- id: trailing-whitespace
files: ".*\\.py"
- id: check-added-large-files
- id: check-toml
- id: end-of-file-fixer
- id: trailing-whitespace
exclude: ".python-version-default"

- repo: https://github.com/pre-commit/mirrors-prettier
rev: v4.0.0-alpha.8
Expand Down
1 change: 1 addition & 0 deletions .python-version-default
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.12
5 changes: 3 additions & 2 deletions conda-store-server/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,16 @@
# the generated Docker image is used with docker-compose to run the conda-store
# server and public conda-store Docker images

FROM condaforge/mambaforge:24.3.0-0 as base
FROM condaforge/mambaforge:24.3.0-0 AS base

LABEL org.opencontainers.image.authors="conda-store development team"

# ensure we are using the conda environment
ENV PATH=/opt/conda/condabin:/opt/conda/envs/conda-store-server/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:${PATH}
ENV TZ=Etc/UTC

ARG python_version=3.10
# must be passed at build environment (should use .python-version-default)
ARG python_version
ARG conda_env_name="conda-store-server"
ARG user_no=1000

Expand Down
3 changes: 2 additions & 1 deletion conda-store-server/environment-dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ channels:
- conda-forge
- nodefaults
dependencies:
- python=3.10
# must be kept in sync with .python-version-default
- python=3.12
# conda builds
- conda
# dev dependencies
Expand Down
3 changes: 2 additions & 1 deletion conda-store/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ FROM condaforge/mambaforge:24.3.0-0 AS base

LABEL org.opencontainers.image.authors="conda-store development team"

ARG python_version=3.10
# must be passed at build environment (should use .python-version-default)
ARG python_version
ARG conda_env_name="conda-store"
ARG user_no=1000

Expand Down
1 change: 1 addition & 0 deletions recipe/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{% set name = "conda-store" %}
# version MUST be updated when a new release is made
{% set version = "2024.6.1" %}

package:
Expand Down
Loading