Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make test tries to download non-existing file #3544

Open
giordano opened this issue Jul 25, 2024 · 3 comments
Open

make test tries to download non-existing file #3544

giordano opened this issue Jul 25, 2024 · 3 comments
Labels

Comments

@giordano
Copy link
Contributor

giordano commented Jul 25, 2024

Describe the bug

Trying to run make test on 71d9c71 I get

parsl/executors/taskvine/install-taskvine.sh
+ [[ -z 7.8.0 ]]
+ TARBALL=cctools-7.8.0-x86_64-ubuntu20.04.tar.gz
+ [[ -f /etc/redhat-release ]]
+ TARBALL=cctools-7.8.0-x86_64-centos8.tar.gz
+ [[ ! -t 1 ]]
+ wget '' '' -O /tmp/cctools.tar.gz https://github.com/cooperative-computing-lab/cctools/releases/download/release/7.8.0/cctools-7.8.0-x86_64-centos8.tar.gz
http://: Invalid host name.
http://: Invalid host name.
--2024-07-25 20:58:39--  https://github.com/cooperative-computing-lab/cctools/releases/download/release/7.8.0/cctools-7.8.0-x86_64-centos8.tar.gz
Resolving github.com (github.com)... 20.26.156.215
Connecting to github.com (github.com)|20.26.156.215|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-07-25 20:58:39 ERROR 404: Not Found.

make: *** [/tmp/cctools] Error 8

The file doesn't exist among the assets of https://github.com/cooperative-computing-lab/cctools/releases/tag/release%2F7.8.0

To Reproduce

Follow instructions at

parsl/README.rst

Lines 95 to 107 in 71d9c71

1. Download Parsl::
$ git clone https://github.com/Parsl/parsl
2. Build and Test::
$ cd parsl # navigate to the root directory of the project
$ make # show all available makefile targets
$ make virtualenv # create a virtual environment
$ source .venv/bin/activate # activate the virtual environment
$ make deps # install python dependencies from test-requirements.txt
$ make test # make (all) tests. Run "make config_local_test" for a faster, smaller test set.

Expected behavior

Tests don't fail because they try to download non-existing files.

Environment

  • OS: RedHat 7.8
  • Python version: 3.11.3
  • Parsl version: 71d9c71

Distributed Environment

N/A

@giordano giordano added the bug label Jul 25, 2024
@dthain
Copy link
Contributor

dthain commented Jul 31, 2024

Hmm, it would appear that the centos version for 7.8.0 was never generated:
https://github.com/cooperative-computing-lab/cctools/releases/tag/release%2F7.8.0

Let me see what I can do to make it...

@benclifford
Copy link
Collaborator

to address this problem more generally, I'd like to move the parsl Makefile from being the worlds lamest package manager. Cooperative Computing Lab has been pretty clear that we should be installing this with conda, and recent work has started to do testing with more focused environments: see the flux tests introduced in #3159 and the kubernetes tests prototyped in PR #3482. Going in that direction we should be testing WQ/TaskVine stuff in a conda environment installed as it should be (and more importantly as our users should be installing the parsl+cctools stack)

@dthain
Copy link
Contributor

dthain commented Jul 31, 2024

Ah, it had slipped my mind that we moved diagonally from centos7 to almalinux8 a few versions back. So switching to that is likely the easiest solution within your CI.

That said, Conda is our preferred install path that gets the dependencies sorted out right. (And we nightly test conda install parsl ndcctools to make sure that is working.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants