Skip to content

Latest commit

 

History

History
74 lines (58 loc) · 1.86 KB

Running YANK on Summit.md

File metadata and controls

74 lines (58 loc) · 1.86 KB

Preliminaries

Set up some paths. This can be in your ~/.bash_profile:

unset PYTHONPATH

export USERNAME="`whoami`"
export EDITOR="emacs -nw"

# SUMMIT
alias summit="ssh summit.ccs.ornl.gov"
module add cuda/9.2.148
export PROJECT="bip178"
alias interactive="bsub -W 2:00 -nnodes 1 -P $PROJECT -Is /bin/bash"
export USER_ARCHIVE="/home/$USER"
export PROJECT_HOME="/ccs/proj/$PROJECT"
export MEMBER_WORK="/gpfs/alpine/scratch/$USER/$PROJECT/"
export PROJECT_WORK="/gpfs/alpine/proj-shared/$PROJECT"
export WORLD_WORK="/gpfs/alpine/world-shared/$PROJECT"
export PROJECT_ARCHIVE="/proj/$PROJECT"

# miniconda
export PATH=$MEMBER_WORK/miniconda/bin:$PATH

Miniconda

Install miniconda for ppc64le:

source ~/.bash_profile
cd $MEMBER_WORK
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-ppc64le.sh
bash Miniconda3-latest-Linux-ppc64le.sh -b -p miniconda
export PATH=$MEMBER_WORK/miniconda/bin:$PATH

YANK

Install YANK and its dependencies

conda install --yes yank

** NOTE: This is currently not working because some dependencies are still being built for ppc64le. **

mpi4py

Install mpi4py using Summit MPI-enabled compilers:

# Install gcc 8 and corresponding MPI compilers
module add gcc/8.1.1

# Make sure to remove glib, since it breaks `aprun`
conda remove --yes --force glib mpi mpich mpi4py

# Build and install special mpi4py for titan
cd $MEMBER_WORK
wget https://bitbucket.org/mpi4py/mpi4py/downloads/mpi4py-3.0.0.tar.gz -O mpi4py-3.0.0.tar.gz
tar zxf mpi4py-3.0.0.tar.gz
cd mpi4py-3.0.0

cat >> mpi.cfg <<EOF
# Summit spectrum MPI for gcc 8.1.1
# ---------------------------------
[summit]
mpi_dir              = $OLCF_SPECTRUM_MPI_ROOT
mpicc                = $OLCF_SPECTRUM_MPI_ROOT/bin/mpicc
mpicxx               = $OLCF_SPECTRUM_MPI_ROOT/bin/mpiCC
EOF

python setup.py build --mpi=summit
python setup.py install