{devel,vis}[fosscuda/2019b] torchvision v0.8.1, PyTorch v1.7.0 w/ Python 3.7.4, with MKL #11786

terjekv · 2020-11-30T08:26:48Z

(created using eb --new-pr)

…orch-1.7.0-imkl.eb, PyTorch-1.7.0-fosscuda-2019b-Python-3.7.4-imkl.eb

terjekv · 2020-11-30T08:29:19Z

Requires #11636 for patches and fixes.

Flamefire · 2020-11-30T08:38:27Z

easybuild/easyconfigs/p/PyTorch/PyTorch-1.7.0-fosscuda-2019b-Python-3.7.4-imkl.eb

+builddependencies = [
+    ('CMake', '3.15.3'),
+    ('hypothesis', '4.44.2', local_pyversionsuffix),
+    ('libarchive', '3.4.2'),


This shouldn't be required. The error message from #11636 (comment) hints at cmake not pytorch so if at all it should be added to the cmake EC but first it should be checked why it uses the system cmake

Just to check, is it libarchive or cmake that shouldn't be required in builddependencies?

libarchive. The CMake 3.15 shouldn't need it. I see CMake-3.18.4-GCCcore-10.2.0.eb added it so I guess your system CMake is newer than 3.15 and something is wrong so the system cmake gets picked up

Hm, ick. No idea why this happens. My familiarity with CMake isn't great. :-(

Nothing with CMake here, only with the CMake module. As you posted the PyTorch build somehow uses /usr/bin/cmake3 which is clearly wrong. So try to load the CMake/3.15.3 module and run which cmake and which cmake3 to check if the module is broken.

And I checked the PyTorch code: CMAKE = find_executable('cmake3') or find_executable('cmake')
Looks like we need an update to the CMake easyblock

Yeah, good catch. I doubt I have time to sort it right now. Make an issue of it and first one gets a cookie?

Can I haz my cookie please: easybuilders/easybuild-easyblocks#2259

Updated in 117a9bd to not need libarchive, which I didn't include in the PR anyway... I was clearly ahead of myself this morning.

terjekv · 2020-11-30T09:20:03Z

This also requires the imkl modules from #11785.

boegelbot · 2020-11-30T17:23:16Z

@terjekv: Tests failed in GitHub Actions, see https://github.com/easybuilders/easybuild-easyconfigs/actions/runs/391563189
Output from first failing test suite run:

ERROR: test_dep_versions_per_toolchain_generation (test.easyconfigs.easyconfigs.EasyConfigTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 441, in test_dep_versions_per_toolchain_generation
    self.process_all_easyconfigs()
  File "test/easyconfigs/easyconfigs.py", line 121, in process_all_easyconfigs
    EasyConfigTest.parsed_easyconfigs, modules_tool(), retain_all_deps=True)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 460, in resolve_dependencies
    raise_error_missing_deps(totally_missing, extra_msg="no easyconfig file or existing module found")
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 323, in raise_error_missing_deps
    raise EasyBuildError(error_msg)
EasyBuildError: 'Missing dependencies: typing-extensions/3.7.4.3-fosscuda-2019b-Python-3.7.4, imkl/2019.5.281-fosscuda-2019b (no easyconfig file or existing module found)'

======================================================================
ERROR: test_sanity_check_paths (test.easyconfigs.easyconfigs.EasyConfigTest)
Make sure specified sanity check paths adher to the requirements.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 504, in test_sanity_check_paths
    self.process_all_easyconfigs()
  File "test/easyconfigs/easyconfigs.py", line 121, in process_all_easyconfigs
    EasyConfigTest.parsed_easyconfigs, modules_tool(), retain_all_deps=True)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 460, in resolve_dependencies
    raise_error_missing_deps(totally_missing, extra_msg="no easyconfig file or existing module found")
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 323, in raise_error_missing_deps
    raise EasyBuildError(error_msg)
EasyBuildError: 'Missing dependencies: typing-extensions/3.7.4.3-fosscuda-2019b-Python-3.7.4, imkl/2019.5.281-fosscuda-2019b (no easyconfig file or existing module found)'

======================================================================
FAIL: test__parse_easyconfig_PyTorch-1.7.0-fosscuda-2019b-Python-3.7.4-imkl.eb (test.easyconfigs.easyconfigs.EasyConfigTest)
Test for parsing of easyconfig PyTorch-1.7.0-fosscuda-2019b-Python-3.7.4-imkl.eb
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 1041, in innertest
    template_easyconfig_test(self, spec_path)
  File "test/easyconfigs/easyconfigs.py", line 939, in template_easyconfig_test
    self.assertTrue(os.path.isfile(patch_full), msg)
AssertionError: Patch file /home/runner/work/easybuild-easyconfigs/easybuild-easyconfigs/easybuild/easyconfigs/p/PyTorch/PyTorch-1.7.0_fix_altivec_defines.patch is available for PyTorch-1.7.0-fosscuda-2019b-Python-3.7.4-imkl.eb

======================================================================
FAIL: test_style_conformance (test.easyconfigs.styletests.StyleTest)
Check the easyconfigs for style
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/styletests.py", line 57, in test_style_conformance
    self.assertEqual(result, 0, "Found code style errors (and/or warnings): %s" % result)
AssertionError: Found code style errors (and/or warnings): 5

----------------------------------------------------------------------
Ran 10987 tests in 339.573s

FAILED (failures=2, errors=2)
ERROR: Not all tests were successful.

bleep, bloop, I'm just a bot (boegelbot v20200716.01)
Please talk to my owner @boegel if you notice you me acting stupid),
or submit a pull request to https://github.com/boegel/boegelbot fix the problem.

Flamefire · 2020-12-01T09:14:09Z

Can you add a ToDo to resync the 1.7 with the other PR once that is finished?

terjekv · 2020-12-01T09:18:04Z

Sync with {bio,devel}[fosscuda/2019b] PyTorch v1.7.1, typing-extensions v3.7.4.3 w/ Python 3.7.4 #11636 once that is merged!

Flamefire · 2021-06-10T10:10:37Z

@terjekv Ready to be synced and please update to use the easyblock as in #13102

boegel · 2024-01-13T17:50:15Z

closing this since fosscuda/2019b is no longer supported, see https://docs.easybuild.io/policies/toolchains

adding easyconfigs: torchvision-0.8.1-fosscuda-2019b-Python-3.7.4-PyT…

1793f79

…orch-1.7.0-imkl.eb, PyTorch-1.7.0-fosscuda-2019b-Python-3.7.4-imkl.eb

terjekv added the update label Nov 30, 2020

terjekv changed the title ~~{devel,vis}[fosscuda/2019b] torchvision v0.8.1, PyTorch v1.7.0 w/ Python 3.7.4~~ {devel,vis}[fosscuda/2019b] torchvision v0.8.1, PyTorch v1.7.0 w/ Python 3.7.4, with MKL Nov 30, 2020

Flamefire reviewed Nov 30, 2020

View reviewed changes

terjekv added 2 commits November 30, 2020 12:20

CMake fixed in easybuilders/easybuild-easyblocks#2259

117a9bd

Updated checksums.

7548450

boegel added this to the 4.x milestone Jan 13, 2024

boegel closed this Jan 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

{devel,vis}[fosscuda/2019b] torchvision v0.8.1, PyTorch v1.7.0 w/ Python 3.7.4, with MKL #11786

{devel,vis}[fosscuda/2019b] torchvision v0.8.1, PyTorch v1.7.0 w/ Python 3.7.4, with MKL #11786

terjekv commented Nov 30, 2020

terjekv commented Nov 30, 2020

Flamefire Nov 30, 2020

terjekv Nov 30, 2020

Flamefire Nov 30, 2020 •

edited

Loading

terjekv Nov 30, 2020

Flamefire Nov 30, 2020

Flamefire Nov 30, 2020

terjekv Nov 30, 2020

Flamefire Nov 30, 2020 •

edited

Loading

terjekv Nov 30, 2020

terjekv Nov 30, 2020

terjekv commented Nov 30, 2020

boegelbot commented Nov 30, 2020

Flamefire commented Dec 1, 2020

terjekv commented Dec 1, 2020

Flamefire commented Jun 10, 2021 •

edited

Loading

boegel commented Jan 13, 2024

{devel,vis}[fosscuda/2019b] torchvision v0.8.1, PyTorch v1.7.0 w/ Python 3.7.4, with MKL #11786

{devel,vis}[fosscuda/2019b] torchvision v0.8.1, PyTorch v1.7.0 w/ Python 3.7.4, with MKL #11786

Conversation

terjekv commented Nov 30, 2020

terjekv commented Nov 30, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Flamefire Nov 30, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Flamefire Nov 30, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

terjekv commented Nov 30, 2020

boegelbot commented Nov 30, 2020

Flamefire commented Dec 1, 2020

terjekv commented Dec 1, 2020

Flamefire commented Jun 10, 2021 • edited Loading

boegel commented Jan 13, 2024

Flamefire Nov 30, 2020 •

edited

Loading

Flamefire Nov 30, 2020 •

edited

Loading

Flamefire commented Jun 10, 2021 •

edited

Loading