Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CUDA - Waiting for a meaningful proposal #11448

Open
2 tasks
jgsogo opened this issue Jun 30, 2022 · 2 comments
Open
2 tasks

CUDA - Waiting for a meaningful proposal #11448

jgsogo opened this issue Jun 30, 2022 · 2 comments
Assignees
Labels
question Further information is requested

Comments

@jgsogo
Copy link
Contributor

jgsogo commented Jun 30, 2022

There have been some issues around CUDA, and some suggestions about how to support it inside CCI. So far it looks like there are different things to take into account:

  • CUDA license doesn't allow us to redistribute the binaries
  • Binaries for CUDA are huge (several Gb)

We want to provide proper support for CUDA ecosystem and we will contact them to see how we can deal with these issues and be able to use CUDA as a requirement (also as a build requirement) in recipes in ConanCenter.

I'll try to link all related issues here, and please, try to centralize the conversation here as well.

Thanks!


@jgsogo jgsogo added the question Further information is requested label Jun 30, 2022
@Artalus
Copy link
Contributor

Artalus commented Oct 24, 2023

CUDA license doesn't allow us to redistribute the binaries

Does it literally mean that there can be no conan_package.tgz in Conan Center, because as soon it would contain nvcc and libcuda.so neatly tucked into bin/ and lib/ subfolders and become publicly available for every Conan user, it would count as redistribution?

If so - then how does Anaconda work? For example in their libcublas and cuda-nvcc packages there are literally tarballs containing the actual binaries:

$ aunpack libcublas-11.10.3.66-0.tar.bz2
$  ls -lh libcublas-11.10.3.66-0/lib
total 463M
   27 May 20  2022 libcublasLt.so.11 -> ./libcublasLt.so.11.10.3.66
 318M May 20  2022 libcublasLt.so.11.10.3.66
   25 May 20  2022 libcublas.so.11 -> ./libcublas.so.11.10.3.66
 145M May 20  2022 libcublas.so.11.10.3.66
   25 May 20  2022 libnvblas.so.11 -> ./libnvblas.so.11.10.3.66
 720K May 20  2022 libnvblas.so.11.10.3.66

(at least for Linux). And then their cuda-toolkit package is formed as a tree of requirements, with a plethora of smaller packages encapsulating different parts of CUDA.


IMO the main roadblock here would be the "policy to not package binaries that haven´t been built on our servers".

@jcar87
Copy link
Contributor

jcar87 commented Oct 24, 2023

Hi @Artalus - we are currently reviewing how to best proceed with this.
Unaware how Conda does it - either there are portions that are allowed to be distributed, or they have written permission by NVIDIA, or there are specific conditions that allow this to happen (e.g. I think the parts of the runtime are OK to distribute, but for Conan we need anything needed for development, e.g. headers, stub libraries, etc). There is some context conda-forge/staged-recipes#654 and conda-forge/conda-forge.github.io#63.

Distributing externally built binaries would not be a problem for us provided they come from a trusted, canonical, validated source, and the binary compatibility path is clear (prebuilt binaries means that we can't build from source for some configurations, so the recipe would need to encode the compatibility matrix since it can't build anything from source).

We are still working on this and reviewing how to best proceed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants