Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove unneccessary workaround for missing version of OpenDataCube and Spyder for 2020+ toolchains #12757

Conversation

Flamefire
Copy link
Contributor

@Flamefire Flamefire commented Apr 27, 2021

(created using eb --new-pr)

I discovered this while checking #12522 for validity.
In short: for 2019b and earlier we need the workaround because setuptools_scm>=3.4 is required which we don't have there.
In 2020a we do have that, so all is fine.

We should generally check if either an updated setuptools (maybe as a build dep) solves problems like this or use the generic whl from PyPi as pip install foo is doing.

@Flamefire
Copy link
Contributor Author

Test report by @Flamefire
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
taurusa16 - Linux centos linux 7.7.1908, x86_64, Intel(R) Xeon(R) CPU E5-2603 v4 @ 1.70GHz (broadwell), Python 2.7.5
See https://gist.github.com/44ffaa94d1bf7caff9c40a338b57c289 for a full test report.

@branfosj branfosj added this to the 4.x milestone Apr 27, 2021
@Flamefire
Copy link
Contributor Author

@branfosj It's a small change and worth getting in soon so new ECs based on those existing ones won't have it.

@branfosj branfosj changed the title Remove unneccessary workaround for missing version of OpenDataCube and Spyer for 2020+ toolchains Remove unneccessary workaround for missing version of OpenDataCube and Spyder for 2020+ toolchains Apr 27, 2021
@branfosj
Copy link
Member

@boegelbot please test @ generoso

@branfosj branfosj dismissed their stale review April 27, 2021 11:03

change made

@boegelbot
Copy link
Collaborator

@branfosj: Request for testing this PR well received on generoso

PR test command 'EB_PR=12757 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_12757 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 16923

Test results coming soon (I hope)...

- notification for comment with ID 827516536 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@branfosj
Copy link
Member

Test report by @branfosj
FAILED
Build succeeded for 0 out of 2 (2 easyconfigs in total)
bear-pg0211u03a.bear.cluster - Linux RHEL 8.3, x86_64, Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz (cascadelake), Python 3.6.8
See https://gist.github.com/2e6edbea928f10661e1f7cb68827479f for a full test report.

@branfosj
Copy link
Member

@Flamefire failures are:

 The following Python packages were likely not installed correctly because they show a version of '0.0.0':
datacube

Did I need to rebuild something else first?

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
FAILED
Build succeeded for 0 out of 2 (2 easyconfigs in total)
generoso-c1-s-1 - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/1d29cbef0105455cc3e183a7461cf55b for a full test report.

@Flamefire
Copy link
Contributor Author

It might be possible that this is a bug with how we install pip&setuptools which I fixed at https://github.com/easybuilders/easybuild-easyconfigs/pull/12650/files#diff-f2778ba0368a4ffc9f51ce4387a33394d9558d72b8802f7d06199bb8e6c81243L53-L57

I'm testing that now again, what exactly that causes and will report back. If you got some time please check the EasyBlock PRs required for #12650

@Flamefire
Copy link
Contributor Author

@branfosj After lot's of trouble-shooting I found the culprit: While it looks like we installed a recent setuptools in core Python we did not. By using pip_ignore_installed:True (default) the (old) setuptools contained within core Python is not removed when installing the newer one. In fact it only copies new files, i.e. most of that package is left unchanged.

I'm currently testing a change to the Python EB to set pip_ignore_installed:False for all extensions of that: easybuilders/easybuild-easyblocks#2388

If you want to test that you could load your Python module (Python/3.8.2-GCCcore-9.3.0) and run pip install --force-reinstall setuptools==45.2.0 to fix up that package (mostly a no-op, just does what the EC should have done). You may need temporary write permissions for the install folder of course.
After that the tests should pass

@Flamefire
Copy link
Contributor Author

Test report by @Flamefire
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
taurusi6163.taurus.hrsk.tu-dresden.de - Linux RHEL 7.9, x86_64, Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz (haswell), Python 2.7.5
See https://gist.github.com/714b769565d97e1e4c1dae53f23dbd14 for a full test report.

@Flamefire
Copy link
Contributor Author

Test report by @Flamefire
SUCCESS
Build succeeded for 3 out of 3 (2 easyconfigs in total)
taurusi8028 - Linux centos linux 7.9.2009, x86_64, AMD EPYC 7352 24-Core Processor (zen2), Python 2.7.5
See https://gist.github.com/db8845441d8551d2bced53ba1d94f025 for a full test report.

@branfosj
Copy link
Member

Test report by @branfosj
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
bear-pg0211u03a.bear.cluster - Linux RHEL 8.3, x86_64, Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz (cascadelake), Python 3.6.8
See https://gist.github.com/9f6d2567a7f34a1a419773bb8ca4039e for a full test report.

@branfosj
Copy link
Member

Successful build was after rebuilds of Python with easybuilders/easybuild-easyblocks#2388 and #12650.

@branfosj
Copy link
Member

@boegelbot please test @ generoso

@boegelbot
Copy link
Collaborator

@branfosj: Request for testing this PR well received on generoso

PR test command 'EB_PR=12757 EB_ARGS= /apps/slurm/default/bin/sbatch --job-name test_PR_12757 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 17214

Test results coming soon (I hope)...

- notification for comment with ID 841101800 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
generoso-c1-s-1 - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/f601567c26ed9e184acd6ae2a7022363 for a full test report.

@branfosj
Copy link
Member

Going in, thanks @Flamefire!

@branfosj branfosj merged commit 368d3a4 into easybuilders:develop May 14, 2021
@Flamefire Flamefire deleted the 20210427084343_new_pr_Open-Data-Cube-Core183 branch May 15, 2021 08:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants