Skip to content

Commit

Permalink
try creating a new conda environment
Browse files Browse the repository at this point in the history
  • Loading branch information
jameslamb committed Oct 1, 2024
1 parent 4a9eae6 commit 3879447
Show file tree
Hide file tree
Showing 7 changed files with 18 additions and 18 deletions.
8 changes: 5 additions & 3 deletions .github/workflows/test-notebooks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ jobs:
steps:
- name: Install required tools
run: |
mamba install -n base --freeze-installed \
mamba install -n rapids --freeze-installed \
curl \
git
- name: Install gha-tools
Expand All @@ -88,12 +88,14 @@ jobs:
rapids-logger "nvidia-smi"
nvidia-smi
- name: Test notebooks
run: /home/rapids/test_notebooks.py -i /home/rapids/notebooks -o /home/rapids/notebooks_output
run: |
. /opt/conda/etc/profile.d/conda.sh; conda activate rapids
/home/rapids/test_notebooks.py -i /home/rapids/notebooks -o /home/rapids/notebooks_output
- name: Install awscli
if: '!cancelled()'
run: |
PATH="$PATH:$GHA_TOOLS_DIR"
rapids-mamba-retry install -n base awscli
rapids-mamba-retry install -n rapids awscli
- uses: aws-actions/configure-aws-credentials@v4
if: '!cancelled()'
with:
Expand Down
6 changes: 3 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ conda config --show-sources
conda list --show-channel-urls

# Install RAPIDS
mamba install -y -n base \
mamba env create -y -n rapids \
"rapids=${RAPIDS_VER}.*" \
"python=${PYTHON_VER}.*" \
"cuda-version=${CUDA_VER%.*}.*" \
Expand Down Expand Up @@ -90,12 +90,12 @@ COPY --from=dependencies --chown=rapids /test_notebooks_dependencies.yaml test_n
COPY --from=dependencies --chown=rapids /notebooks /home/rapids/notebooks

RUN <<EOF
mamba env update -n base -f test_notebooks_dependencies.yaml
mamba env update -n rapids -f test_notebooks_dependencies.yaml
conda clean -afy
EOF

RUN <<EOF
mamba install -y -n base \
mamba install -y -n rapids \
"jupyterlab=4" \
dask-labextension \
jupyterlab-nvdashboard
Expand Down
6 changes: 4 additions & 2 deletions context/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,21 @@ EOF

if [ -e "/home/rapids/environment.yml" ]; then
echo "environment.yml found. Installing packages."
timeout ${CONDA_TIMEOUT:-600} mamba env update -n base -f /home/rapids/environment.yml || exit $?
timeout ${CONDA_TIMEOUT:-600} mamba env update -n rapids -f /home/rapids/environment.yml || exit $?
fi

if [ "$EXTRA_CONDA_PACKAGES" ]; then
echo "EXTRA_CONDA_PACKAGES environment variable found. Installing packages."
timeout ${CONDA_TIMEOUT:-600} mamba install -n base -y $EXTRA_CONDA_PACKAGES || exit $?
timeout ${CONDA_TIMEOUT:-600} mamba install -n rapids -y $EXTRA_CONDA_PACKAGES || exit $?
fi

if [ "$EXTRA_PIP_PACKAGES" ]; then
echo "EXTRA_PIP_PACKAGES environment variable found. Installing packages.".
timeout ${PIP_TIMEOUT:-600} pip install $EXTRA_PIP_PACKAGES || exit $?
fi

. /opt/conda/etc/profile.d/conda.sh; conda activate rapids

# Run whatever the user wants.
if [ "${UNQUOTE}" = "true" ]; then
exec $@
Expand Down
2 changes: 1 addition & 1 deletion dockerhub-readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ The `rapidsai/notebooks` container has notebooks for the RAPIDS libraries in `/h

### Extending RAPIDS Images

All RAPIDS images use `conda` as their package manager, and all RAPIDS packages are available in the `base` conda environment. These image run as the `rapids` user.
All RAPIDS images use `conda` as their package manager, and all RAPIDS packages are available in the `rapids` conda environment. These image run as the `rapids` user.

### Access Documentation within Notebooks

Expand Down
2 changes: 1 addition & 1 deletion raft-ann-bench/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ docker run --gpus all --rm -it \
This will drop you into a command line in the container, with RAFT and the `raft_ann_benchmarks` python package ready to use:

```
(base) root@00b068fbb862:/home/rapids#
(rapids) root@00b068fbb862:/home/rapids#
```

Additionally, the containers could be run in dettached form without any issue.
7 changes: 2 additions & 5 deletions raft-ann-bench/cpu/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ SHELL ["/bin/bash", "-euo", "pipefail", "-c"]
RUN <<EOF
mkdir /data
chmod 777 /data
echo ". /opt/conda/etc/profile.d/conda.sh; conda activate base" >> /etc/bash.bashrc
echo ". /opt/conda/etc/profile.d/conda.sh; conda activate rapids" >> /etc/bash.bashrc
EOF

# we need perl temporarily for the remaining benchmark perl scripts
Expand All @@ -27,10 +27,7 @@ RUN apt-get install perl -y
# an older conda with newer packages still works well
# ref: https://github.com/rapidsai/ci-imgs/issues/185
RUN <<EOF
mamba update --all -y -n base
mamba install -y -n base "python=${PYTHON_VER}"
mamba update --all -y -n base
mamba install -y -n base \
mamba env create -y -n rapids \
"raft-ann-bench-cpu=${RAPIDS_VER}.*" \
"python=${PYTHON_VER}"
conda clean -afy
Expand Down
5 changes: 2 additions & 3 deletions raft-ann-bench/gpu/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,14 @@ SHELL ["/bin/bash", "-euo", "pipefail", "-c"]
RUN <<EOF
mkdir /data
chmod 777 /data
echo ". /opt/conda/etc/profile.d/conda.sh; conda activate base" >> /etc/bash.bashrc
echo ". /opt/conda/etc/profile.d/conda.sh; conda activate rapids" >> /etc/bash.bashrc
EOF

# we need perl temporarily for the remaining benchmark perl scripts
RUN apt-get install perl -y

RUN <<EOF
mamba update --all -y -n base
mamba install -y -n base \
mamba env create -y -n rapids \
"raft-ann-bench=${RAPIDS_VER}.*" \
"cuda-version=${CUDA_VER%.*}.*"
conda clean -afy
Expand Down

0 comments on commit 3879447

Please sign in to comment.