Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed rye add for tensorflow[and-cuda] package. #745

Closed
Di-Is opened this issue Feb 23, 2024 · 13 comments
Closed

Failed rye add for tensorflow[and-cuda] package. #745

Di-Is opened this issue Feb 23, 2024 · 13 comments
Assignees
Labels
bug Something isn't working

Comments

@Di-Is
Copy link

Di-Is commented Feb 23, 2024

Steps to Reproduce

curl -sSf https://rye-up.com/get | bash
rye init 
rye pin 3.11
# use uv backend
rye config --set-bool behavior.use-uv=true
rye add tensorflow[and-cuda]
# use legacy backend
rye config --set-bool behavior.use-uv=false
rye add tensorflow[and-cuda]

Expected Result

Tensorflow[and-cuda] is successfully added.

Actual Result

Installation fails.

root@dcd0f3262eca:/tmp/test# curl -sSf https://rye-up.com/get | bash
root@dcd0f3262eca:/tmp/test# rye init 
success: Initialized project in /tmp/test/.
  Run `rye sync` to get started
root@dcd0f3262eca:/tmp/test# rye pin 3.11
pinned 3.11.7 in /tmp/test/.python-version
root@dcd0f3262eca:/tmp/test# rye config --set-bool behavior.use-uv=true
root@dcd0f3262eca:/tmp/test# rye add tensorflow[and-cuda]
error: failed to resolve packages:
thread 'main' panicked at crates/uv-resolver/src/resolution.rs:136:29:
Every package should have metadata: NameVersion(PackageName("tensorflow"), "2.15.0.post1")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

root@dcd0f3262eca:/tmp/test# rye config --set-bool behavior.use-uv=false
root@dcd0f3262eca:/tmp/test# rye add tensorflow[and-cuda]
error: did not find package 'tensorflow[and-cuda]' without using pre-releases.
root@dcd0f3262eca:/tmp/test# 

Version Info

rye 0.25.0
commit: 0.25.0 (d8e00ce 2024-02-19)
platform: linux (x86_64)
self-python: [email protected]
symlink support: true
uv enabled: false

Stacktrace

use uv backend

root@dcd0f3262eca:/tmp/test# rye config --set-bool behavior.use-uv=true
root@dcd0f3262eca:/tmp/test# RUST_BACKTRACE=1 rye add tensorflow[and-cuda]
error: failed to resolve packages:
thread 'main' panicked at crates/uv-resolver/src/resolution.rs:136:29:
Every package should have metadata: NameVersion(PackageName("tensorflow"), "2.15.0.post1")
stack backtrace:
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
   1: rye::cli::add::resolve_requirements_with_uv
   2: rye::cli::add::execute
   3: rye::cli::execute
   4: rye::main
   5: std::sys_common::backtrace::__rust_begin_short_backtrace
   6: std::rt::lang_start::{{closure}}
   7: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:284:13
   8: std::panicking::try::do_call
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
   9: std::panicking::try
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
  10: std::panic::catch_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
  11: std::rt::lang_start_internal::{{closure}}
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:48
  12: std::panicking::try::do_call
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
  13: std::panicking::try
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
  14: std::panic::catch_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
  15: std::rt::lang_start_internal
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:20
  16: main

use legacy backend

error: did not find package 'tensorflow[and-cuda]' without using pre-releases.

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
   1: rye::cli::add::execute
   2: rye::cli::execute
   3: rye::main
   4: std::sys_common::backtrace::__rust_begin_short_backtrace
   5: std::rt::lang_start::{{closure}}
   6: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:284:13
   7: std::panicking::try::do_call
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
   8: std::panicking::try
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
   9: std::panic::catch_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
  10: std::rt::lang_start_internal::{{closure}}
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:48
  11: std::panicking::try::do_call
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
  12: std::panicking::try
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
  13: std::panic::catch_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
  14: std::rt::lang_start_internal
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:20
  15: main
@ischaojie
Copy link
Contributor

ischaojie commented Feb 23, 2024

The latest commit looks like fixed this problem, you can fetch the latest code and build to try again.

➜  rye-test git:(main) ✗ rye add 'tensorflow[and-cuda]'
Detected outdated rye internals. Refreshing
Bootstrapping rye internals
Found a compatible Python version: [email protected]
Upgrading pip
Installing internal dependencies
Initializing new virtualenv in /Users/chaojie/projects/test/rye-test/.venv
Python version: [email protected]
Added tensorflow>=2.15.0 as regular dependency
Reusing already existing virtualenv
Generating production lockfile: /Users/chaojie/projects/test/rye-test/requirements.lock
warning: Requirements file /var/folders/6t/rml6nn4d0q99nrqv8wmwz2t80000gn/T/.tmp4yFLDF/requirements.txt does not contain any dependencies
   Built file:///Users/chaojie/projects/test/rye-test                                                                                                                           Built 1 editable in 2.98s
Resolved 43 packages in 15.97s
Generating dev lockfile: /Users/chaojie/projects/test/rye-test/requirements-dev.lock
warning: Requirements file /var/folders/6t/rml6nn4d0q99nrqv8wmwz2t80000gn/T/.tmpFgLEI5/requirements.txt does not contain any dependencies
   Built file:///Users/chaojie/projects/test/rye-test                                                                                                                           Built 1 editable in 162ms
Resolved 43 packages in 332ms
Installing dependencies
   Built file:///Users/chaojie/projects/test/rye-test                                                                                                                           Built 1 editable in 161ms
Resolved 41 packages in 16ms
Downloaded 41 packages in 2m 17s
Installed 43 packages in 312ms
 + absl-py==2.1.0
 + astunparse==1.6.3
 + cachetools==5.3.2
 + certifi==2024.2.2
........
Done!

@Di-Is
Copy link
Author

Di-Is commented Feb 23, 2024

@ischaojie
Thank you for checking.
However, "tensorflow[and-cuda]" seems to be installed as "tensorflow" without extra.
The requirements.lock generated as a result of rye add is missing library dependencies such as nvidia-* specific to [and-cuda].

Below is a comparison of the uv pip compile results and the requirements.lock generated by rye.

uv pip compile result

echo 'tensorflow[and-cuda]' | uv pip compile -

# This file was autogenerated by uv via the following command:
#    uv pip compile -
absl-py==2.1.0
    # via
    #   tensorboard
    #   tensorflow
astunparse==1.6.3
    # via tensorflow
cachetools==5.3.2
    # via google-auth
certifi==2024.2.2
    # via requests
charset-normalizer==3.3.2
    # via requests
flatbuffers==23.5.26
    # via tensorflow
gast==0.5.4
    # via tensorflow
google-auth==2.28.1
    # via
    #   google-auth-oauthlib
    #   tensorboard
google-auth-oauthlib==1.2.0
    # via tensorboard
google-pasta==0.2.0
    # via tensorflow
grpcio==1.62.0
    # via
    #   tensorboard
    #   tensorflow
h5py==3.10.0
    # via tensorflow
idna==3.6
    # via requests
keras==2.15.0
    # via tensorflow
libclang==16.0.6
    # via tensorflow
markdown==3.5.2
    # via tensorboard
markupsafe==2.1.5
    # via werkzeug
ml-dtypes==0.2.0
    # via tensorflow
numpy==1.26.4
    # via
    #   h5py
    #   ml-dtypes
    #   opt-einsum
    #   tensorboard
    #   tensorflow
nvidia-cublas-cu12==12.2.5.6
    # via
    #   nvidia-cudnn-cu12
    #   nvidia-cusolver-cu12
    #   tensorflow
nvidia-cuda-cupti-cu12==12.2.142
    # via tensorflow
nvidia-cuda-nvcc-cu12==12.2.140
    # via tensorflow
nvidia-cuda-nvrtc-cu12==12.2.140
    # via
    #   nvidia-cudnn-cu12
    #   tensorflow
nvidia-cuda-runtime-cu12==12.2.140
    # via tensorflow
nvidia-cudnn-cu12==8.9.4.25
    # via tensorflow
nvidia-cufft-cu12==11.0.8.103
    # via tensorflow
nvidia-curand-cu12==10.3.3.141
    # via tensorflow
nvidia-cusolver-cu12==11.5.2.141
    # via tensorflow
nvidia-cusparse-cu12==12.1.2.141
    # via
    #   nvidia-cusolver-cu12
    #   tensorflow
nvidia-nccl-cu12==2.16.5
    # via tensorflow
nvidia-nvjitlink-cu12==12.2.140
    # via
    #   nvidia-cusolver-cu12
    #   nvidia-cusparse-cu12
    #   tensorflow
oauthlib==3.2.2
    # via requests-oauthlib
opt-einsum==3.3.0
    # via tensorflow
packaging==23.2
    # via tensorflow
protobuf==4.25.3
    # via
    #   tensorboard
    #   tensorflow
pyasn1==0.5.1
    # via
    #   pyasn1-modules
    #   rsa
pyasn1-modules==0.3.0
    # via google-auth
requests==2.31.0
    # via
    #   requests-oauthlib
    #   tensorboard
requests-oauthlib==1.3.1
    # via google-auth-oauthlib
rsa==4.9
    # via google-auth
setuptools==69.1.0
    # via
    #   tensorboard
    #   tensorflow
six==1.16.0
    # via
    #   astunparse
    #   google-pasta
    #   tensorboard
    #   tensorflow
tensorboard==2.15.2
    # via tensorflow
tensorboard-data-server==0.7.2
    # via tensorboard
tensorflow==2.15.0.post1
tensorflow-estimator==2.15.0
    # via tensorflow
tensorflow-io-gcs-filesystem==0.36.0
    # via tensorflow
termcolor==2.4.0
    # via tensorflow
typing-extensions==4.9.0
    # via tensorflow
urllib3==2.2.1
    # via requests
werkzeug==3.0.1
    # via tensorboard
wheel==0.42.0
    # via astunparse
wrapt==1.14.1
    # via tensorflow

rye's requirements.lock

rye add 'tensorflow[and-cuda]'
cat requirements.lock

# generated by rye
# use `rye lock` or `rye sync` to update this lockfile
#
# last locked with the following flags:
#   pre: false
#   features: []
#   all-features: false
#   with-sources: false

-e file:.
absl-py==2.1.0
    # via tensorboard
    # via tensorflow
astunparse==1.6.3
    # via tensorflow
cachetools==5.3.2
    # via google-auth
certifi==2024.2.2
    # via requests
charset-normalizer==3.3.2
    # via requests
flatbuffers==23.5.26
    # via tensorflow
gast==0.5.4
    # via tensorflow
google-auth==2.28.1
    # via google-auth-oauthlib
    # via tensorboard
google-auth-oauthlib==1.2.0
    # via tensorboard
google-pasta==0.2.0
    # via tensorflow
grpcio==1.62.0
    # via tensorboard
    # via tensorflow
h5py==3.10.0
    # via tensorflow
idna==3.6
    # via requests
keras==2.15.0
    # via tensorflow
libclang==16.0.6
    # via tensorflow
markdown==3.5.2
    # via tensorboard
markupsafe==2.1.5
    # via werkzeug
ml-dtypes==0.2.0
    # via tensorflow
numpy==1.26.4
    # via h5py
    # via ml-dtypes
    # via opt-einsum
    # via tensorboard
    # via tensorflow
oauthlib==3.2.2
    # via requests-oauthlib
opt-einsum==3.3.0
    # via tensorflow
packaging==23.2
    # via tensorflow
protobuf==4.25.3
    # via tensorboard
    # via tensorflow
pyasn1==0.5.1
    # via pyasn1-modules
    # via rsa
pyasn1-modules==0.3.0
    # via google-auth
requests==2.31.0
    # via requests-oauthlib
    # via tensorboard
requests-oauthlib==1.3.1
    # via google-auth-oauthlib
rsa==4.9
    # via google-auth
setuptools==69.1.0
    # via tensorboard
    # via tensorflow
six==1.16.0
    # via astunparse
    # via google-pasta
    # via tensorboard
    # via tensorflow
tensorboard==2.15.2
    # via tensorflow
tensorboard-data-server==0.7.2
    # via tensorboard
tensorflow==2.15.0.post1
    # via test
tensorflow-estimator==2.15.0
    # via tensorflow
tensorflow-io-gcs-filesystem==0.36.0
    # via tensorflow
termcolor==2.4.0
    # via tensorflow
typing-extensions==4.9.0
    # via tensorflow
urllib3==2.2.1
    # via requests
werkzeug==3.0.1
    # via tensorboard
wheel==0.42.0
    # via astunparse
wrapt==1.14.1
    # via tensorflow

@charliermarsh
Copy link
Member

I think the issue here is that (e.g.) rye add "flask[dotenv]" only adds flask>=3.0.2, rather than flask[dotenv]>=3.0.2. So the extras are being lost in the pyproject.toml.

@charliermarsh
Copy link
Member

uv does not include extra annotations in the generated output, which is also pip-compile's default in the next version.

@charliermarsh
Copy link
Member

I can take a look.

@charliermarsh charliermarsh self-assigned this Feb 26, 2024
@charliermarsh charliermarsh added the bug Something isn't working label Feb 26, 2024
@mitsuhiko
Copy link
Collaborator

Pretty sure the bug is in rye. Unearth didn't retain that information either. I probably just retained it better.

@charliermarsh
Copy link
Member

I sent you a diff on Discord, but it's not fully correct.

@26awen
Copy link

26awen commented Feb 29, 2024

Wish this fix will be released sooner. 👀

@XBeg9
Copy link

XBeg9 commented Mar 4, 2024

facing the same issue with

rye add "ray[default]"    
error: did not find package 'ray[default]' without using pre-releases.

any ideas when it will be fixed in the release?

@shanilpanara
Copy link

shanilpanara commented Mar 5, 2024

Similar issue with "jax[cuda12_pip]". Install is successful, but only jax gets installed (i.e. the one from pypi), NOT jax with all the cuda12 dependencies.

$ rye --version
rye 0.27.0
commit: 0.27.0 (43ee4fce0 2024-02-26)
platform: linux (x86_64)
self-python: [email protected]
symlink support: true
uv enabled: true

And I am running this using WSL2 on a Dell Laptop with an Nvidia GPU (if any of this makes a difference...)

--

I first add the following sources to my pyproject.toml

[[tool.rye.sources]]
name = "google-jax"
url = "https://storage.googleapis.com/jax-releases/jax_cuda_releases.html"
type = "find-links"

Then this is what happens upon trying to add jax!

$ rye add jax[cuda12_pip] --no-sync # the behaviour is same with/without no-sync by the way!
Added jax>=0.4.25 as regular dependency

Expected pyproject.toml change

[project]
dependencies = [
    "jax[cuda12_pip]>=...",
]

Actual pyproject.toml change

[project]
dependencies = [
    "jax>=0.4.25", 
]

Other commands I've tried (with the exact same change/behaviour)

$ rye add jax --features cuda12_pip
$ rye add jax[cuda12_pip] --features cuda12_pip
$ rye add jax[cuda12_pip,cuda12_pip]
# And I tried adding the flags: `--pre` `--dev` for good measure too...

Workaround!

I manually edited the pyproject.toml like so:

[project]
dependencies = [
    "jax[cuda12_pip]>=0.4.25",
]

Then upon rye sync or the next rye add, IT WORKS

Example:

$ rye add numpy
Added numpy>=1.26.4 as regular dependency
Reusing already existing virtualenv
Generating production lockfile: <PATH>/requirements.lock
Generating dev lockfile: <PATH>/requirements-dev.lock
Installing dependencies
Installed 12 packages in 51ms
 + jaxlib==0.4.25+cuda12.cudnn89
 + nvidia-cublas-cu12==12.3.4.1
 + nvidia-cuda-cupti-cu12==12.3.101
 + nvidia-cuda-nvcc-cu12==12.3.107
 + nvidia-cuda-nvrtc-cu12==12.3.107
 + nvidia-cuda-runtime-cu12==12.3.101
 + nvidia-cudnn-cu12==8.9.7.29
 + nvidia-cufft-cu12==11.0.12.1
 + nvidia-cusolver-cu12==11.5.4.101
 + nvidia-cusparse-cu12==12.2.0.103
 + nvidia-nccl-cu12==2.19.3
 + nvidia-nvjitlink-cu12==12.3.101
Done!

(FYI, I dont have a 1TB internet speed, it just used previously downloaded nvidia packages haha! 😂)

Hope this is helpful!

Thanks to the Rye team!

Edit... just after writing this I tried one more thing

$ rye config --set-bool behavior.use-uv=false
$ rye add jax[cuda12_pip]

And it works... the pyproject.toml gets updated with jax[cuda12_pip]>=0.4.25 as a dependency!!!

Then everything gets installed (albeit much more slowly) after running

rye sync

How weird?!

@charliermarsh
Copy link
Member

I believe this is fixed in #807 (but not yet released).

@slashtechno
Copy link

Would it be possible to use tensorflow-intel on Windows but tensorflow[and-cuda] everywhere else? Or would this need dependency groups like in #421?

@tmm88
Copy link

tmm88 commented Mar 19, 2024

this seems to work for me

# Download CUDA pin file
#wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
#sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600

# Download CUDA repository package
#wget https://developer.download.nvidia.com/compute/cuda/12.4.0/local_installers/cuda-repo-wsl-ubuntu-12-4-local_12.4.0-1_amd64.deb

'''

# Update package lists
sudo apt update

# Upgrade packages
sudo apt upgrade -y
sudo apt dist-upgrade -y
sudo apt full-upgrade -y

sudo apt install software-properties-common
sudo add-apt-repository ppa:ubuntu-toolchain-r/test

sudo dpkg --configure -

sudo apt update

sudo apt install gcc-12 g++-12 gcc-13 g++-13 -y

sudo dpkg --configure -

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 12 --slave /usr/bin/g++ g++ /usr/bin/g++-12
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-13 13 --slave /usr/bin/g++ g++ /usr/bin/g++-13

sudo dpkg --configure -

sudo update-alternatives --config gcc

sudo dpkg --configure -

gcc --version

sudo apt install build-essential
sudo apt install libmpfr-dev libgmp3-dev libmpc-dev -y
wget http://ftp.gnu.org/gnu/gcc/gcc-13.2.0/gcc-13.2.0.tar.gz
#tar -xf gcc-13.2.0.tar.gz

sudo dpkg --configure -

cd gcc-13.2.0

sudo dpkg --configure -

./configure -v --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --prefix=/usr/local/gcc-13.2.0 --enable-checking=release --enable-languages=c,c++ --disable-multilib --program-suffix=-13.2.0

sudo dpkg --configure -

make -j3
sudo make install

sudo dpkg --configure -

/usr/local/gcc-13.2.0/bin/gcc-13.2.0 --version

sudo dpkg --configure -

# Install required packages
sudo apt install -y \
    libxkbcommon0=1.6.0-1 \
    fakeroot \
    libalgorithm-merge-perl \
    g++=4:13.2.0-7 \
    cpp-x86-64-linux-gnu=4:13.2.0-7 \
    g++-13>=13.2.0-11 \
    g++-x86-64-linux-gnu=4:13.2.0-7 \
    gcc-13>=13.2.0-11 \
    gcc=4:13.2.0-7 \
    gcc-x86-64-linux-gnu=4:13.2.0-7 \
    libglvnd0=1.7.0-1 \
    lto-disabled-list \
    openjdk-17-jre-headless=17.0.10+7-1~22.04.1 \
    libatk-wrapper-java-jni>=0.30.4-0ubuntu2 \
    cpp=4:13.2.0-7 \
    cpp-x86-64-linux-gnu=4:13.2.0-7 \
    g++-13>=13.2.0-11~ \
    g++-x86-64-linux-gnu=4:13.2.0-7 \ 
    gcc-13>=13.2.0-11~ \
    gcc=4:13.2.0-7 \
    cpp-x86-64-linux-gnu=4:13.2.0-7 \ 
    gcc-13>=13.2.0-11~ \
    gcc-x86-64-linux-gnu=4:13.2.0-7 \ 
    openjdk-17-jre-headless=17.0.10+7-1~22.04.1 \
    libatk-wrapper-java-jni>=0.30.4-0ubuntu2

'''

# Install CUDA repository
sudo dpkg -i cuda-repo-wsl-ubuntu-12-4-local_12.4.0-1_amd64

# Copy CUDA keyring
sudo cp /var/cuda-repo-wsl-ubuntu-12-4-local/cuda-*-keyring.gpg /usr/share/keyrings/

# Update package lists
sudo apt-get update

# Install CUDA toolkit
sudo apt-get -y install cuda-toolkit-12-4

# Install pycuda (optional)
#sudo pip3 install pycuda

@Di-Is Di-Is closed this as completed Jun 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

9 participants