Support for 64-bit indptr data and multiple GPU？

[YH-Zheng]

Hi, thank you for your work, which has made our single cell analysis much faster.
However, when the nnz of the matrix exceeds the maximum limit of 32 bits, the preprocessing process needs to be performed again using native scanpy, which greatly increases our time. I saw in https://github.com/NVIDIA-Genomics-Research/rapids-single-cell-examples/tree/master this work uses dask to distribute large tasks in chunks to multiple GPUs, which makes large The GPU acceleration of the amount of data can be realized, do you have any ideas in this regard? Such as support for multiple GPUs?
Thanks again for your work!

[Intron7]

Dear @YH-Zheng,

Thank you for your interest in rapids_singlecell. I want to support 64bit indptr in rsc. However there is currently a limitation with cupy when it comes to the indptr arrays. I tested a bit with using uint32 which doubles the nnzs but that's sadly to unstable. I'm already testing ways to make rapids-singlecell multi gpu enabled. Which back end I'll use is still up in the air. cunumeric is also very promising.

[YH-Zheng]

Dear @Intron7 ,

Hi Severin! Thank you for your reply, and I am looking forward to your next work. At present, I will still use the regular scanpy analysis process until I use rsc for harmony integration.

I also noticed that the rsc of the Louvain and Leiden algorithm will generate more clusters, especially Leiden, even if the resolution is reduced, it will still generate hundreds of clusters, but it performs well in the sc (because the amount of data is too large, full iteration Time is too long, only iterate 2 times). What do you think about it?

I have not saved the Leiden result at present, the picture is the Louvain results of rsc, I think resolution=1 produces too many clusters than sc.

[Zethson]

See #44 for the clustering issue

[Intron7]

So the 64 bit support in cupy might be coming. They made it a high priority cupy/cupy#3513.

[SNOL2]

Dear @YH-Zheng,

Thank you for your interest in rapids_singlecell. I want to support 64bit indptr in rsc. However there is currently a limitation with cupy when it comes to the indptr arrays. I tested a bit with using uint32 which doubles the nnzs but that's sadly to unstable. I'm already testing ways to make rapids-singlecell multi gpu enabled. Which back end I'll use is still up in the air. cunumeric is also very promising.

Also waiting for supporting for multiple GPUs!

[Intron7]

@YH-Zheng @SNOL2 We started working on multi-GPU support with dask. It's still in early development but the performance looks really promising.

[SNOL2]

@YH-Zheng @SNOL2 We started working on multi-GPU support with dask. It's still in early development but the performance looks really promising.

Great! Thanks for the updating.