Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

problems installing torch-geometric #1662

Closed
KukumavMozolo opened this issue Aug 2, 2022 · 6 comments · Fixed by #1723
Closed

problems installing torch-geometric #1662

KukumavMozolo opened this issue Aug 2, 2022 · 6 comments · Fixed by #1723
Labels
bug Something is not working hashes Related to hashes generated via --generate-hashes

Comments

@KukumavMozolo
Copy link

pip-sync requirements_lock.txt reports missing torch for any of these packages:

torch-cluster==1.6.0 \
    --hash=sha256:362076bd713268ba52ed31ac9517686e206084a50bc4ea50543bdeb992d25bfe
    # via -r requirements.in
torch-geometric==2.0.4 \
    --hash=sha256:d64e4c7486fcf0c7fa82f0acbf5dd52035855469708bf89f8bc7fc607671c8b7
    # via -r requirements.in
torch-scatter==2.0.9 \
    --hash=sha256:a3e90ca7d97b0269ba5c86995f93dedd9dba80c42a65aab51d0250581ef024b4
    # via -r requirements.in
torch-sparse==0.6.14 \
    --hash=sha256:1d19c337e4842f633aedc4e23ad27c15a7ac85ce1bee0b9b016e26a821ab5f85
    # via -r requirements.in
torch-spline-conv==1.2.1 \
    --hash=sha256:959287ca87f33b8c5d901d5dccf24b1744e9c10b60fee54958fc452302059426
    # via -r requirements.in

Environment Versions

  1. OS Type: Ubuntu 20.04
  2. Python version: $ python -V = Python 3.9.5
  3. pip version: $ pip --version = pip 22.2.1
  4. pip-tools version: $ pip-compile --version = version 6.8.0

Steps to replicate

  1. create requirements_lock.txt with:
--find-links https://download.pytorch.org/whl/torch_stable.html
--find-links https://data.pyg.org/whl/torch-1.12.0+cpu.html
torch==1.12.0+cpu \
    --hash=sha256:133023854509dd84d7ccefd99d1e9c07c392efef48c574ef6f798933c284dd6a \
    --hash=sha256:2f15f6bfcc3bca61e9b7acead9b47e6ee848c3523dedfcff3efbdd07ae6777ed \
    --hash=sha256:37f14f640b91effe41db244b932c2dd697ca2b51ae241a534259b9d9f7f51f6f \
    --hash=sha256:69afe17ea3f1bd3dd5c88ede3e275c7395463293739965e9b206f94c575d8c99 \
    --hash=sha256:6fa9e60fed5c7d7738e49e343aa1c3849ba4f3bd4da661a146d86f253e5e43cb \
    --hash=sha256:733ebe15f94edda67286b0284921b513940024d9c1943ba4122cdae4959de6e6 \
    --hash=sha256:98993de211242593626a236ee31686b2dc31b6fe564ce10a4047167b9c3eb2f5 \
    --hash=sha256:ed618d8aec7c4d46edd05fb064b2712f039b418e414a83acb959596cdf80423c
    # via
    #   -r requirements.in
    #   torchaudio
    #   torchvision
torch-cluster==1.6.0 \
    --hash=sha256:362076bd713268ba52ed31ac9517686e206084a50bc4ea50543bdeb992d25bfe
    # via -r requirements.in
torch-geometric==2.0.4 \
    --hash=sha256:d64e4c7486fcf0c7fa82f0acbf5dd52035855469708bf89f8bc7fc607671c8b7
    # via -r requirements.in
torch-scatter==2.0.9 \
    --hash=sha256:a3e90ca7d97b0269ba5c86995f93dedd9dba80c42a65aab51d0250581ef024b4
    # via -r requirements.in
torch-sparse==0.6.14 \
    --hash=sha256:1d19c337e4842f633aedc4e23ad27c15a7ac85ce1bee0b9b016e26a821ab5f85
    # via -r requirements.in
torch-spline-conv==1.2.1 \
    --hash=sha256:959287ca87f33b8c5d901d5dccf24b1744e9c10b60fee54958fc452302059426
    # via -r requirements.in
  1. pip-sync requirements_lock.txt

Expected result

pip-sync installed the packages

Actual result

Collecting torch-cluster==1.6.0
  Using cached torch_cluster-1.6.0.tar.gz (43 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-jpw__qkf/torch-cluster_04c100c202574c86820a68b9c8a6ad3f/setup.py", line 8, in <module>
          import torch
      ModuleNotFoundError: No module named 'torch'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.




Workaround:

 pip-sync --pip-args "--only-binary torch-cluster,torch-scatter,torch-sparse,torch-spline-conv" requirements_lock.txt

but then the hashes are different than those created by pip-compile and need to be adjusted manually... Maybe there is a way to avoid this?

@AndydeCleyre
Copy link
Contributor

I think an issue here is that torch-cluster doesn't use a pyproject.toml (or deprecated setup file stuff) to specify that torch is a build dependency, yet does in fact require torch to be pre-installed in order to build; see https://setuptools.pypa.io/en/latest/build_meta.html#dynamic-build-dependencies-and-other-build-meta-tweaks

@AndydeCleyre
Copy link
Contributor

I think you'll see the same problem installing with pip install -r requirements_lock.txt, as long as torch isn't preinstalled.

@KukumavMozolo
Copy link
Author

KukumavMozolo commented Aug 4, 2022

Another workaround I found is to install the torch deps like this:
pip-sync --pip-args "--only-binary :all --no-binary torch-cluster,torch-scatter,torch-sparse,torch-spline-conv" requirements_lock.txt
I think that works because the --no-binary deps will be installed last, therefore after torch is installed..?

@KukumavMozolo
Copy link
Author

KukumavMozolo commented Aug 5, 2022

@AndydeCleyre do you have an idea why the returned hashes might be different when using:
pip-sync --pip-args "--only-binary torch-cluster,torch-scatter,torch-sparse,torch-spline-conv" requirements_lock.txt
as compared to the hashes generated by pip-compile?
I double checked for example the hash for torch-scatter returned by pip-compile is correct as can be seen here:
(https://pypi.org/project/torch-scatter/#copy-hash-modal-d4d4dc20-c5c5-4f31-9fab-4994e36cc657):
but when running:
pip-sync --pip-args "--only-binary torch-cluster,torch-scatter,torch-sparse,torch-spline-conv" requirements_lock.txt
I get:

 (ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    torch-scatter==2.0.9 from https://data.pyg.org/whl/torch-1.12.0%2Bcpu/torch_scatter-2.0.9-cp39-cp39-linux_x86_64.whl (from -r /tmp/tmpyj7b66_c (line 1)):
        Expected sha256 08f5511d64473badf0a71d156b36dc2b09b9c2f00a7cd373b935b490c477a7f1
             Got        a3e90ca7d97b0269ba5c86995f93dedd9dba80c42a65aab51d0250581ef024b4

Could it be that the hashes are generated based on a binary in the pypi repo instead of this one?
https://data.pyg.org/whl/torch-1.12.0%2Bcpu/torch_scatter-2.0.9-cp39-cp39-linux_x86_64.whl

@AndydeCleyre
Copy link
Contributor

Yeah, I'm not too familiar with the hash logic, but I think when fetching remote hashes we currently rely on the --index-url, maybe not accounting for --find-links.

Pinging @atugushev for some wisdom here

Related: #1109

@atugushev
Copy link
Member

atugushev commented Aug 6, 2022

Oh, this might be an unknown bug where pip-compile wouldn't fetch hashes from packages listed in --find-links. See details below.

Details

Only a hash from PyPI

$ cat requirements.in
--find-links https://data.pyg.org/whl/torch-1.12.0+cpu.html
torch-cluster

$ pip-compile --generate-hashes --no-header
--find-links https://data.pyg.org/whl/torch-1.12.0+cpu.html

torch-cluster==1.6.0 \
    --hash=sha256:249c1bd8c33a887b22bf569a59d0868545804032123594dd8c76ba1885859c39
    # via -r requirements.in

Hashes from --find-link

$ cat requirements.in
--no-index
--find-links https://data.pyg.org/whl/torch-1.12.0+cpu.html
torch-cluster

$ pip-compile --generate-hashes --no-header
--find-links https://data.pyg.org/whl/torch-1.12.0+cpu.html

torch-cluster==1.6.0 \
    --hash=sha256:07cc15bcad6acab550d5ffcc573a72b36e0dba477153f06d449c3ec32d7ddd9d \
    --hash=sha256:362076bd713268ba52ed31ac9517686e206084a50bc4ea50543bdeb992d25bfe \
    --hash=sha256:3a7624cae4aee89b50ea04bbccfac89c5d4e945f5000cbc975b115369dfa2942 \
    --hash=sha256:3eea6b1cec9f19e41a8478968307cd8f2405643b375d6e1a2820d842dc59474c \
    --hash=sha256:5cf2ec9100f573a73e604fa36f1e2add53ad7441b591c0beac41817626b5e3ab \
    --hash=sha256:72b48c39dce2c92e78030b99e46b6bea5fac188680aaf05a4fb973a101d86bee \
    --hash=sha256:7765a7dad5ec4876d715112065ec162e0d78465f259bdd934907468721c495f6 \
    --hash=sha256:b1698105b728b0ae8628e7d94b7201244831d72c82fed90ade092288971dd066 \
    --hash=sha256:b8e5f20186fb384008514655193e336518366e26316f3292163cc0abe969cb9c \
    --hash=sha256:e92e990d3d63035e05cacdc7d460e202afc0b56a6f6551d91bdc8b48ccf03c59 \
    --hash=sha256:ffc8b1106c0cf03c9ee5c8d2ec6f1fad4bccb0f596c8e0bae560cd2663951793
    # via -r requirements.in

@atugushev atugushev added bug Something is not working hashes Related to hashes generated via --generate-hashes labels Aug 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something is not working hashes Related to hashes generated via --generate-hashes
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants