Conference call notes 20200513

(back to Conference calls)

Notes on the 147th EasyBuild conference call, Wednesday May 13th 2020 (8:00 UTC - 10:00 CEST)

Attendees

Alphabetical list of attendees (XXX):

• Kenneth Hoste (HPC-UGent, Belgium)

• Alex Domingo (Vrije Universiteit Brussel, Belgium)

• Fotis Georgatos (SDSC, Switzerland)

• Adam Huffman (Big Data Institute, Oxford, UK)

• Terje Kvernes (University of Oslo, Norway)

• Mikael Öhman (Chalmers University of Technology, Sweden)

• Bart Oldeman (ComputeCanada)

• Victor Rusu (CSCS, Switzerland)

• Jörg Saßmannshausen (NIHR Biomedical Research Centre, UK)

• Åke Sandgren (Umeå University, Sweden)

• Shahzeb Siddiqui (DassaultSystemes)

• Alexandre Strube (JSC, Germany)

• Lars Viklund (Umeå University, Sweden)

Agenda

• overview of recent changes to be included with upcoming EasyBuild v4.2.1
• update on progress on 2020a generation of easyconfigs
• Q&A

Overview of recent changes

• project in GitHub with overview of things to be done for EasyBuild v4.2.1
  • maintainers: if any issues/PRs pop up that really must be included in next release, assign them to the 4.2.1 project!

framework: merged PRs in 4.2.1 milestone

• recently merged PRs:
  • automatic changing of directory by extract_file is deprecated (PR #3292)
    • also fixed in easyblocks (PR #2051)
  • proper fix for PyPI download problem, parse page served by PyPI's /simple API as HTML (not XML) (PR #3329)
    • it looks like the cause of the problems in EasyBuild will be fixed on the PyPI side too (see here)
  • cleaning up locks on signals (PR #3291)
    • this is not enough to auto-cleanup locks for EasyBuild sessions run in Slurm jobs that get killed though... (issue #3280)
    • requires looking into fcntl style looks; looks doable, maybe even should be the default locking mechanism?
  • stop using distutils.dir_util in copy_dir (PR #3310)
  • make ModulesTool.exist more robust w.r.t. module wrappers & co (PR #3216)
    • this introduced a regression (issue #3335)
  • fix problems with parsing easyconfigs using a Cray toolchain when there are no actual external modules (PR #3319)
    • this fixed --list-installed-software crash when using HMNS
  • add constants for common OS dependencies (PR #3309, PR #3334)
    • OS_PKG_IBVERBS_DEV, OS_PKG_OPENSSL_DEV, OS_PKG_PAM_DEV, ...
  • bug fixes for --try-update-deps experimental feature (PR #3325), (PR #3326), (PR #3330),
  • cleaned up output for --list-toolchains --output-format=rst (PR #3246)
    • see result in docs (develop only for now)
  • add --disable-job in 'eb' command used in jobs, to prevent infinite job cycle (PR #3328)
  • avoid empty entries in $LD_LIBRARY_PATH and other path-like environment variables (PR #)
• open PRs/issues that should be merged/fixed for EasyBuild 4.2.1 release:
  • bugs to fix:
    • Java/11 module wrapper is not seen as installed when using HierarchicalMNS (issue #3335, introduced by PR #3216)
  • bugfix PRs to review/merge:
    • expand glob paths when checking for multiple inclusion of easyblocks (bugfix for --include-easyblocks*) (PR #3311)
  • new features:
    • allow copying of tweaked easyconfigs when using --try-* (PR #3332)

easyblocks: merged PRs in 4.2.1 milestone

• recently merged PRs:
  • fixes/enhancements for PETSc easyblock (PR #2048)
  • use config.guess for GCC builds (PR #2033)
  • impi: execute minimal test in sanity check commands (PR #2045)
  • fix for ROOT easyblock (PR #2047)
  • include own lib dir in RPATH before system lib dirs in binutils easyblock (PR #2044)
• open PRs/issues that should be merged/fixed for EasyBuild 4.2.1 release:
  • correct download url for test data in WPS easyblock (PR #2055)
  • use updated config.guess for installing R packages (PR #1949)
  • fix for CMakeMake easyblock w.r.t. switch to building outside source tree (PR#2032)
  • rewrite GROMACS easyblock to build all four variations in the same build (PR #1991)
  • generically find the arch folder and add top-level bin/lib symlinks in PDT & TAU easyblocks (PR #2029)
  • new easyblock to install XALT (PR #1942)
  • fix problem with forced reinstallation of ROOT (issue #2023)

easyconfigs merged PRs in 4.2.1 milestone

• recently merged PRs:
  • patch for h5py to avoid workaround for MPI_Init at import h5py (PR #10626, original workaround in PR #10246)
    • will need more work, current approach of disabling auto-init by mpi4py isn't working out well
    • see also https://github.com/h5py/h5py/pull/1544
  • fix for R 3.6.2 + R 3.6.3 on POWER (PR #9830, PR #10531)
• open issues/PRs:
  • fix Mako easyconfigs using a 2016 & 2017 toolchain (PR #10627)
  • TensorFlow 2.2.0 w/ fosscuda/2019b (PR #10600)
  • updated/more extensions for R 4.0.0 & Bioconductor 3.11 (PR #10614, PR #10621)
  • bugfix for Arlequin (PR #10620)

Update on 2020a common toolchains

• gradually collapsing easyconfigs prepared in 2020a branch into develop
• TODO:
  • LAMMPS (blocked byMPI_Init issues with h5py) (PR #10191, PR #10537)
  • GROMACS (PR #10026)
  • ...
• CUDA compatible with GCC 9.3.0 (for fosscuda/2020a) is coming soon?

Q&A