Brainstorm meeting (Oct 9th 2020)

Attending

• Victor Holanda Rusu (CSCS)
• Kenneth Hoste (HPC-UGent)
• Terje Kvernes (UiO)
• Maxim Masterov (SURF)
• Alan O'Cais (JSC)
• Caspar van Leeuwen (SURF)
• Peter Stol (VU)
• Bob Dröge (RUG)

Notes

(by Kenneth)

• format?
  • Python (like .eb)
  • YAML (like .yml)
    • JSON
    • TOML
• what should spec file include?
  • EasyBuild version?
  • list of easyconfigs (wildcards)
  • "requirements": gpu-only, not on aarch64, ...
  • additional EasyBuild configuration options
    • --from-pr
• build script
  • processes spec file
  • filters, figures out what to build
  • resolves deps
  • installs stuff one by one (don't use --robot)
    • gives the option to tweak specific installations

Proposal

Usage

eb eessi-2020.10.yml

eb --spec [email protected]

• also support filtering on software name?

eessi-2020.09.yml

$ eb eessi-2020.09.yml --labels='arch:aarch64;system:gpu,ib;os:rhel8;cluster:lisa'

eessi-2020.09:
  easybuild_version: 4.3.0
  robot: True
  software:
    Bioconductor:
      toolchains:
        foss-2020a:
          versions:
            3.11:
              versionsuffix: '-R-4.0.0'
              exclude-when: arch:aarch64
    EasyBuild:
      toolchains:
        SYSTEM:
          versions: [4.3.1]
    GROMACS:
      toolchains:
        foss-2020a:
          exclude-labels: system:gpu
          versions:
            2020.1:
            2020.3:
              from_pr: 1234
        fosscuda-2020a:
          include-labels: system:gpu
          versions: [2020.1]
    OpenFOAM:
      toolchains:
        foss-2020a:
          versions: [8, v2006]
    R:
      toolchains:
        foss-2020a:
          versions: [4.0.0]

If this file becomes too big, we can split it up in multiple files:

$ eb eessi-2020.09.yml eessi-2020.09-gromacs.yml --labels='arch:aarch64;system:gpu,ib;os:rhel8;cluster:lisa'

Other use case: Rebuild stuff with new EB release

Stuff to rebuild with EasyBuild v4.3.1:

$ eb eb-4.3.1-rebuild.yml --trace

$ eb eb-4.3.1-rebuild.yml --dryrun
* R
  eb_args: --rebuild --robot --skip

eb-4.3.2-rebuild:
  easybuild_version: 4.3.2
  confirmation_needed: True
  rebuild-if-exists: True
  robot: True
  software:
    impi:
      toolchains:
        iccifort-2020.1.217:
           versions:
             2019.7.217:
              # fix module file for impi v2019.7.217
              module_only: True
    R:
      toolchains:
        foss-2020a:
          versions:
            4.0.0:
              # install additional extensions for R v4.0.0
              skip: True