Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Prevent path conflict in builds #5821

Closed
wants to merge 2 commits into from

Conversation

AyodeAwe
Copy link
Contributor

@AyodeAwe AyodeAwe commented Mar 25, 2024

Contributes to rapidsai/build-planning#54

This will make builds fail when there are path conflicts

@AyodeAwe AyodeAwe requested a review from a team as a code owner March 25, 2024 19:35
@AyodeAwe AyodeAwe added non-breaking Non-breaking change improvement Improvement / enhancement to an existing function ci labels Mar 25, 2024
@AyodeAwe AyodeAwe changed the base branch from branch-24.04 to branch-24.06 April 3, 2024 18:39
@jameslamb jameslamb changed the title Prevent path conflict in builds WIP: Prevent path conflict in builds Apr 22, 2024
@jameslamb jameslamb marked this pull request as draft April 22, 2024 23:11
@jameslamb
Copy link
Member

jameslamb commented Apr 23, 2024

This is currently failing with the following conflicts.

(I've included just 1 example of each type below)

(CUDA 11.8 build log) (CUDA 12.2 build log)

  1. fmt and spdlog issues (conflicting packages: conda-forge/fmt, librmm, libraft-headers-only)

Described in rapidsai/rmm#1508 (comment).

  1. (CUDA 11.x only) CUDA toolkit packages packaged differently (conflicting packages: nvidia/cuda-{component}, conda-forge/cudatoolkit)
This transaction has incompatible packages due to a shared path.
  packages: nvidia/linux-64::libcublas-11.11.3.6-0, conda-forge/linux-64::cudatoolkit-11.8.0-h4ba93d1_13
  path: 'lib/libcublas.so.11'

And similar for all these:

lib/libcublas.so.11.11.3.6
lib/libcublasLt.so.11
lib/libcublasLt.so.11.11.3.6
lib/libnvblas.so.11
lib/libnvblas.so.11.11.3.6
lib/libcufft.so.10
lib/libcufft.so.10.9.0.58
lib/libcufftw.so.10
lib/libcufftw.so.10.9.0.58
lib/libcurand.so.10
lib/libcurand.so.10.3.0.86
lib/libcusolver.so.11
lib/libcusolver.so.11.4.1.48
lib/libcusolverMg.so.11
lib/libcusolverMg.so.11.4.1.48
lib/libcusparse.so.11
lib/libcusparse.so.11.7.5.86
lib/libcublas.so
lib/libcublasLt.so
lib/libnvblas.so
lib/libcufft.so
lib/libcufftw.so
lib/libcurand.so
lib/libcusolver.so
lib/libcusolverMg.so
lib/libcusparse.so
  1. (CUDA 11.x only) lib-{component} and lib-{component}-dev from the nvidia channel
This transaction has incompatible packages due to a shared path.
  packages: nvidia/linux-64::libcusparse-11.7.5.86-0, nvidia/linux-64::libcusparse-dev-11.7.5.86-0, conda-forge/linux-64::cudatoolkit-11.8.0-h4ba93d1_13
 path: 'lib/libcusparse.so.11.7.5.86'
  1. cub *(conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
 packages: rapidsai-nightly/linux-64::librmm-24.04.00a38-cuda11_240326_ga98931b9_38, rapidsai-nightly/linux-64::libraft-headers-only-24.04.00a93-cuda11_240326_g9637b3c2_93
 path: 'include/rapids/cub/agent/agent_radix_sort_upsweep.cuh'
  1. libcudacxx (conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
 packages: rapidsai-nightly/linux-64::librmm-24.04.00a38-cuda11_240326_ga98931b9_38, rapidsai-nightly/linux-64::libraft-headers-only-24.04.00a93-cuda11_240326_g9637b3c2_93
  path: 'include/rapids/libcudacxx/cuda/std/cassert'
  1. thrust (conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
 packages: rapidsai-nightly/linux-64::librmm-24.04.00a38-cuda11_240326_ga98931b9_38, rapidsai-nightly/linux-64::libraft-headers-only-24.04.00a93-cuda11_240326_g9637b3c2_93
 path: 'include/rapids/thrust/addressof.h'
  1. rmm (conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
  packages: rapidsai-nightly/linux-64::librmm-24.04.00a38-cuda11_240326_ga98931b9_38, rapidsai-nightly/linux-64::libraft-headers-only-24.04.00a93-cuda11_240326_g9637b3c2_93
  path: 'include/rmm/mr/device/arena_memory_resource.hpp'

@jameslamb jameslamb changed the base branch from branch-24.06 to branch-24.08 May 21, 2024 15:49
@jameslamb
Copy link
Member

This work is paused, in favor of pursuing a better long-term solution in the future. Closing this PR for now.

Subscribe to rapidsai/build-planning#54 and rapidsai/build-planning#56 for updates.

@jameslamb jameslamb closed this Jul 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants