Change workflow to mamba (#271)

* Change workflow to mamba * Address comments * Revert sphinx-multiversion handling * Get sphinx-multiversion from conda-forge, not pypi * Use conda environment for docs in CI --------- Co-authored-by: Xylar Asay-Davis <[email protected]>
E3SM-Project · Jun 19, 2023 · dfca67d · dfca67d
1 parent bc928e0
commit dfca67d
Show file tree

Hide file tree

Showing 5 changed files with 91 additions and 65 deletions.
diff --git a/.github/workflows/build_workflow.yml b/.github/workflows/build_workflow.yml
@@ -35,10 +35,26 @@ jobs:
     steps:
       - uses: actions/checkout@v3
 
-      - name: Set up Python
-        uses: actions/setup-python@v4
+      - name: Cache Conda
+        uses: actions/cache@v3
+        env:
+          CACHE_NUMBER: 0
         with:
-          python-version: 3.9
+          path: ~/conda_pkgs_dir
+          key: ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{
+            hashFiles('conda/dev.yml') }}
+
+      - name: Build Conda Environment
+        uses: conda-incubator/setup-miniconda@v2
+        with:
+          activate-environment: zstash_dev
+          miniforge-variant: Mambaforge
+          miniforge-version: latest
+          use-mamba: true
+          mamba-version: "*"
+          environment-file: conda/dev.yml
+          channel-priority: strict
+          auto-update-conda: true
 
       - name: Install `zstash` Package
         run: |
@@ -63,25 +79,26 @@ jobs:
           persist-credentials: false
           fetch-depth: 0
 
-      - name: Set up Python
-        uses: actions/setup-python@v4
+      - name: Cache Conda
+        uses: actions/cache@v3
+        env:
+          CACHE_NUMBER: 0
         with:
-          python-version: 3.9
+          path: ~/conda_pkgs_dir
+          key: ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{
+            hashFiles('conda/dev.yml') }}
 
-      - name: Cache pip
-        uses: actions/cache@v3
+      - name: Build Conda Environment
+        uses: conda-incubator/setup-miniconda@v2
         with:
-          # This path is specific to Ubuntu
-          path: ~/.cache/pip
-          # Look to see if there is a cache hit for the corresponding requirements file
-          key: ${{ runner.os }}-pip-publish-docs
-          restore-keys: |
-            ${{ runner.os }}-pip-
-            ${{ runner.os }}-
-
-      - name: Install Dependencies
-        run: |
-          pip install sphinx==5.2.3 sphinx_rtd_theme==1.0.0 sphinx-multiversion==0.2.4 docutils==0.16 "jinja2<3.1"
+          activate-environment: zstash_dev
+          miniforge-variant: Mambaforge
+          miniforge-version: latest
+          use-mamba: true
+          mamba-version: "*"
+          environment-file: conda/dev.yml
+          channel-priority: strict
+          auto-update-conda: true
 
       - name: Build Sphinx Docs
         run: |

diff --git a/.github/workflows/release_workflow.yml b/.github/workflows/release_workflow.yml
@@ -17,26 +17,26 @@ jobs:
           persist-credentials: false
           fetch-depth: 0
 
-      - name: Set up Python
-        uses: actions/setup-python@v4
-        with:
-          python-version: 3.9
-
-      - name: Cache pip
+      - name: Cache Conda
         uses: actions/cache@v3
+        env:
+          CACHE_NUMBER: 0
         with:
-          # This path is specific to Ubuntu
-          path: ~/.cache/pip
-          # Look to see if there is a cache hit for the corresponding requirements file
-          key: ${{ runner.os }}-pip-publish-docs
-          restore-keys: |
-            ${{ runner.os }}-pip-
-            ${{ runner.os }}-
+          path: ~/conda_pkgs_dir
+          key: ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{
+            hashFiles('conda/dev.yml') }}
 
-      # Using pip for Sphinx dependencies because it takes too long to reproduce a conda environment (~10 secs vs. 3-4 mins)
-      - name: Install Dependencies
-        run: |
-          pip install sphinx==5.2.3 sphinx_rtd_theme==1.0.0 sphinx-multiversion==0.2.4 docutils==0.16 "jinja2<3.1"
+      - name: Build Conda Environment
+        uses: conda-incubator/setup-miniconda@v2
+        with:
+          activate-environment: zstash_dev
+          miniforge-variant: Mambaforge
+          miniforge-version: latest
+          use-mamba: true
+          mamba-version: "*"
+          environment-file: conda/dev.yml
+          channel-priority: strict
+          auto-update-conda: true
 
       - name: Build Sphinx Docs
         run: |

diff --git a/conda/dev.yml b/conda/dev.yml
@@ -24,10 +24,9 @@ dependencies:
   # If versions are updated, also update in `.github/workflows/workflow.yml`
   - jinja2<3.1
   - sphinx=5.2.3
+  - sphinx-multiversion=0.2.4
   - sphinx_rtd_theme=1.0.0
   # Need to pin docutils because 0.17 has a bug with unordered lists
   # https://github.com/readthedocs/sphinx_rtd_theme/issues/1115
   - docutils=0.16
-  - pip:
-      - sphinx-multiversion==0.2.4
 prefix: /opt/miniconda3/envs/zstash_dev
diff --git a/docs/source/getting_started.rst b/docs/source/getting_started.rst
@@ -81,32 +81,38 @@ Others/Local
 
 If the system doesn't come with conda pre-installed, follow these instructions:
 
-1. Download Conda
+1. Download Mambaforge
 
     Linux
         ::
 
-            wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
+            wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh
 
-    MacOS (note that ``zstash`` is not supported on MacOS, but it may be useful to contribute to the documentation on MacOS)
+    MacOS x86_64 (note that ``zstash`` is not supported on MacOS, but it may be useful to contribute to the documentation on MacOS)
         ::
 
-            wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
+            wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-MacOSX-x86_64.sh
 
-2. Install Conda
+2. Install Mambaforge
 
     Linux
         ::
 
-            bash ./Miniconda3-latest-Linux-x86_64.sh
+            bash ./Mambaforge-Linux-x86_64.sh
 
 
-    MacOS
+    MacOS x86_64
         ::
 
-            bash ./Miniconda3-latest-MacOSX-x86_64.sh
+            bash ./Mambaforge-MacOSX-x86_64.sh
 
-    - ``Do you wish the installer to initialize Miniconda3 by running conda init? [yes|no] yes``
+    When you see: ::
+
+        by running conda init? [yes|no]
+        [no] >>> yes
+
+    respond with ``yes`` so ``conda`` and ``mamba`` commands are available on
+    initializing a new bash terminal.
 
 3. If you are working on a machine/network that intercepts SSL communications (such as
 acme1), you will get an SSL error unless you disable the SSL verification:
@@ -116,31 +122,40 @@ acme1), you will get an SSL error unless you disable the SSL verification:
         conda config --set ssl_verify false
         binstar config --set ssl_verify False
 
-4. Once conda is properly working, you can install the **(a) Latest Stable Release** or
+4. Once conda and mamba are properly working, you can install the **(a) Latest Stable Release** or
 create a **(b) Development Environment**.
 
 (a) Latest Stable Release
 =========================
 
-Installation using conda
+Installation using mamba
 ------------------------
 
 First, make sure that you're using ``bash``. ::
 
-   $ bash
+   bash
 
-You must have Anaconda installed as well.
+You must have a conda base enviornment installed as well.
 See :ref:`"Installation in a Conda Environment" <conda_environment>` section above for
 installing conda.
 Create a new Anaconda environment with zstash installed and activate it: ::
 
-   $ conda create -n zstash_env -c e3sm -c conda-forge zstash
-   $ source activate zstash_env
+These steps should not be necessary if you installed Mambaforge as suggested
+above but may be needed if you have previously installed Miniconda3 instead: ::
+
+   conda install -y -n base mamba
+   conda config --add channels conda-forge
+   conda config --set channel_priority strict
+
+Create a new conda environment with ``zstash`` installed and activate it: ::
 
-Or you can install zstash in an existing environment. ::
+   mamba create -n zstash_env zstash
+   conda activate zstash_env
 
-   $ conda install zstash -c e3sm -c conda-forge
+Or (less recommended because of potential conflicts) you can install ``zstash``
+in an existing environment. ::
 
+   mamba install zppy
 
 Installation on NERSC
 ---------------------
@@ -154,10 +169,10 @@ not directly available there, so you will need to manually activate Anaconda bef
 Updating
 --------
 
-If you **installed via Anaconda** (e.g., not through the unified environment),
-you can update ``zstash`` by doing the following:  ::
+If you **installed into your own conda environment** (e.g., not through the
+unified environment), you can update ``zstash`` by doing the following:  ::
 
-    conda update zstash -c e3sm -c conda-forge
+    mamba update zstash
 
 .. _dev-env:
 
@@ -224,7 +239,7 @@ Furthermore, the dev environment includes quality assurance (QA) tools such as c
 
     ::
 
-        conda clean --all
+        mamba clean --all
 
 4. Enter the fork's clone.
 
@@ -239,7 +254,7 @@ Furthermore, the dev environment includes quality assurance (QA) tools such as c
 
     ::
 
-        conda env create -f conda/dev.yml
+        mamba env create -f conda/dev.yml
         conda activate zstash_dev
 
 6. Install ``pre-commit``.

diff --git a/setup.py b/setup.py
@@ -8,10 +8,5 @@
     description="Long term HPSS archiving software for E3SM",
     packages=find_packages(include=["zstash", "zstash.*"]),
     python_requires=">=3.6",
-    install_requires=[
-        "fair-research-login>=0.2.6,<0.3.0",
-        "globus-sdk>=3.0.0,<4.0.0",
-        "six",
-    ],
     entry_points={"console_scripts": ["zstash=zstash.main:main"]},
 )