-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add new conda environments #120
base: branch-0.35
Are you sure you want to change the base?
Add new conda environments #120
Conversation
e454494
to
af5b587
Compare
$ mamba env create -n ucxx -f conda/environments/ucxx-cuda118_arch-x86_64.yml | ||
$ conda activate ucxx | ||
$ mamba env create -n ucxx -f conda/environments/all_cuda-118_arch-x86_64.yaml | ||
$ conda activate all_cuda-118_arch-x86_64 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should match the name from -n ucxx
above.
$ conda activate all_cuda-118_arch-x86_64 | |
$ conda activate ucxx |
@@ -0,0 +1,34 @@ | |||
# This file is generated by `rapids-dependency-file-generator`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All environment files need to be generated by dependencies.yaml
.
The failure in pre-commit is because rapids-dependency-file-generator
looks for files that it generated and removes them before re-generating the new files. You have the header here saying it's a generated file, but there's no declaration of this environment file in dependencies.yaml
.
This is how you tell it to generate a given environment file:
Lines 3 to 17 in 4464ca9
all: | |
output: conda | |
matrix: | |
cuda: ["11.8", "12.0"] | |
arch: [x86_64] | |
includes: | |
- build_cpp | |
- build_python | |
- checks | |
- cudatoolkit | |
- dev | |
- py_version | |
- run_python | |
- test_cpp | |
- test_python |
For example, you might want to add a section like this for the C++ dependency list.
files:
cpp: # This is used as part of the environment's filename
output: conda
matrix:
arch: [x86_64]
includes:
- build_cpp
- checks
- dev
- test_cpp
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Bradley, this is helpful. I'd still like to trim that more though, for example the above will result in:
# This file is generated by `rapids-dependency-file-generator`.
# To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.
channels:
- rapidsai
- rapidsai-nightly
- dask/label/dev
- conda-forge
- nvidia
dependencies:
- autoconf
- automake
- c-compiler
- cmake>=3.26.4
- cxx-compiler
- dask-cuda==23.12.*
- dask-cudf==23.12.*
- fmt>=9.1.0,<10
- gmock>=1.13.0
- gtest>=1.13.0
- librmm==23.12.*
- libtool
- ninja
- pip
- pkg-config
- pre-commit
- spdlog>=1.11.0,<1.12
name: cpp_arch-x86_64
That include a bunch of things we don't need: rapidsai
/rapidsai-nightly
/dask/label/dev
/nvidia
channels and dask-cuda
/dask-cudf
/librmm
/pip
/pkg-config
. Is there a way to minimize that?
Additionally, this is supposed to be valid for aarch64
too, can we make it architecture agnostic?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's currently no way to drop extra channels from the generated files. You can file an issue to rapids-dependency-file-generator if you have a proposal for how the dependencies.yaml
schema should accommodate that idea, but I'm not sure if it's worth changing. https://github.com/rapidsai/dependency-file-generator
If you depend only on architecture-agnostic packages, you can just drop the arch from the files:
spec.
files:
cpp: # This is used as part of the environment's filename
output: conda
includes:
- build_cpp
- checks
- dev
- test_cpp
UCXX is not necessarily CUDA dependent. This change adds two new environments:
cpp.yaml
for C++-only builds andall.yaml
for C++ and Python builds.