7.5. WRF

Installation

system check

Create a Test folder

mkdir TESTS
cd TESTS

Download the test cases and start tests, the output of tests should include SUCCESS

wget https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/Fortran_C_tests.tar
tar -xf Fortran_C_tests.tar
gfortran TEST_1_fortran_only_fixed.f
./a.out
gfortran TEST_2_fortran_only_free.f90
./a.out
gcc TEST_3_c_only.c
./a.out
gcc -c -m64 TEST_4_fortran+c_c.c
gfortran -c -m64 TEST_4_fortran+c_f.f90
gfortran -m64 TEST_4_fortran+c_f.o TEST_4_fortran+c_c.o
./a.out
./TEST_csh.csh
./TEST_perl.pl
./TEST_sh.sh

Create a library folder and install the library

mkdir Build_WRF
cd Build_WRF
mkdir LIBRARIES
cd LIBRARIES

Download and unzip the required libraries.

nedcdf-c-4.9.2

wget -c https://downloads.unidata.ucar.edu/netcdf-c/4.9.2/netcdf-c-4.9.2.tar.gz
tar xzvf netcdf-c-4.9.2.tar.gz

hdf5(you can also install the newest version)

wget -c https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12/hdf5-1.12.2/src/hdf5-1.12.2.tar.gz
tar xzvf hdf5-1.12.2.tar.gz

zlib(1.2.11)

wget -c https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/zlib-1.2.11.tar.gz
tar xzvf zlib-1.2.11.tar.gz

NETCDF-Fortran(4.6.1)

wget https://downloads.unidata.ucar.edu/netcdf-fortran/4.6.1/netcdf-fortran-4.6.1.tar.gz
tar xzvf netcdf-fortran-4.6.1.tar.gz

PnetCDF(1.12.3)

wget https://parallel-netcdf.github.io/Release/pnetcdf-1.12.3.tar.gz
tar xzvf pnetcdf-1.12.3.tar.gz

LibPNG

wget -c https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/libpng-1.2.50.tar.gz
tar xzvf libpng-1.2.50.tar.gz

MPICH when load the adios2, mpi is automatically loaded in PATH

module load adios2/2.9.1-6fh7kh2

Choice 1: Install the library(with the HDF5 support and without PnetCDF support)

Set the environment variables

export DIR=/PATH/TO/Build_WRF/LIBRARIES
export CC=gcc
export CXX=g++
export FC=gfortran
export F77=gfortran
module load libxml2

Install the zlib

cd zlib-1.2.11/
./configure --prefix=$DIR
make
make install

Install the hdf5

cd hdf5-1.12.2/
./configure --prefix=$DIR --with-zlib=$DIR --enable-hl --enable-fortran
make check
make install
export HDF5=$DIR
export LD_LIBRARY_PATH=$DIR/lib:$LD_LIBRARY_PATH

Build NETCDF-C

cd netcdf-c-4.9.2
export CPPFLAGS=-I$DIR/include
export LDFLAGS=-L$DIR/lib
 ./configure --prefix=$DIR --disable-dap
make check
make install
export PATH=$DIR/bin:$PATH
export NETCDF=$DIR

Build NetCDF fortran library links with NETCDF-C, after successful installation, netcdf.inc will be ${NETCDF}/include file. For sometimes, it will show C compiler stop working, Please check the environment variable in LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz", make sure netcdf and hdf5 path in the environment variables.

export LD_LIBRARY_PATH=$DIR/lib:$LD_LIBRARY_PATH
export CPPFLAGS="-I${DIR}/include -I/usr/include"
export LDFLAGS="-L${DIR}/lib -L/usr/lib"
export LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz" 
./configure --prefix=$DIR --disable-shared
make check
make install

Library Compatibility Tests( the result should show SUCCESS)

mkdir LibTests
cd libTests
wget https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/Fortran_C_NETCDF_MPI_tests.tar
tar -xf Fortran_C_NETCDF_MPI_tests.tar
cp ${NETCDF}/include/netcdf.inc .
gfortran -c 01_fortran+c+netcdf_f.f
gcc -c 01_fortran+c+netcdf_c.c
gfortran 01_fortran+c+netcdf_f.o 01_fortran+c+netcdf_c.o \
     -L${NETCDF}/lib -lnetcdff -lnetcdf
./a.out

Install the WRF The adios2 need to be installed with c-blocs, mpi, and hdf5.

cd ..
wget https://github.com/wrf-model/WRF/releases/download/v4.5.1/v4.5.1.tar.gz
tar xvzf v4.5.1.tar.gz
cd WRFV4.5.1
module load adios2/2.9.1-6fh7kh2
export ADIOS2="/mnt/repo/software/spack/spack/opt/spack/linux-ubuntu22.04-skylake_avx512/gcc-11.3.0/adios2-2.9.1-6fh7kh2v3tombadg56kmop72sjhf23dg"
export HDF5=$DIR
export NETCDF=$DIR
export NETCDF_classic=1
./configure

Please select from among the following Linux x86_64 options: Choose 34(dmpar), Compile for nesting? choose 1. If configure is done, you can see a configure.wrf file in the folder.

./compile em_real
 #./compile em_real >& log.compile
export WRF_DIR=PATH/to/WRFV4.5.1

You can see the following information shows the installation is successful.

--->                  Executables successfully built                  <---                                                                                                                                                                      
 -rwxrwxr-x 1 hxu40 hxu40 51667784 Sep 15 12:21 main/ndown.exe                                                           
 -rwxrwxr-x 1 hxu40 hxu40 51794800 Sep 15 12:21 main/real.exe                                                           
 -rwxrwxr-x 1 hxu40 hxu40 50930720 Sep 15 12:21 main/tc.exe                                                              
 -rwxrwxr-x 1 hxu40 hxu40 59885752 Sep 15 12:21 main/wrf.exe                                                                                                                                                                                     

==========================================================================

To check check whether it was successful

ls -ls main/*.exe

If you compiled a real case, you should see:

wrf.exe (model executable)
real.exe (real data initialization)
ndown.exe (one-way nesting)
tc.exe (for tc bogusing--serial only)

Install WPS

Download and install required library

cd $DIR
wget https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/libpng-1.2.50.tar.gz
tar xzvf libpng-1.2.50.tar.gz
cd libpng-1.2.50
./configure --prefix=$DIR/grib2
make
make install
cd ..
wget https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compile_tutorial/tar_files/jasper-1.900.1.tar.gz
tar xzvf jasper-1.900.1.tar.gz    
cd jasper-1.900.1
./configure --prefix=$DIR/grib2
make
make install
cd ..
cd $DIR
cd zlib_1.22.1
./configure --prefix=$DIR/grib2
make
make install
export JASPERLIB=$DIR/grib2/lib
export JASPERINC=$DIR/grib2/include

Download WPS and install

#go to WRF folder
mkdir WPS
wget https://github.com/wrf-model/WPS/archive/v4.5.tar.gz
tar -xvzf v4.5.tar.gz
cd WPS-4.5
export JASPERLIB=$DIR/grib2/lib
export JASPERINC=$DIR/grib2/include
./configure --build-grib2-libs #Option 3 for gfortran and distributed memory
./compile

IF you see this message after ./configure, please ignore

Testing for NetCDF, C and Fortran compiler
This installation NetCDF is 64-bit
C compiler is 64-bit
Fortran compiler is 64-bit
Your versions of Fortran and NETCDF are not consistent.
```bash

If the compile successfully,
there should be 3 executables in the WPS top-level directory, that are linked to their corresponding src/ directories:
```bash
geogrid.exe -> geogrid/src/geogrid.exe
ungrib.exe -> ungrib/src/ungrib.exe
metgrid.exe -> metgrid/src/metgrid.exe

Usage

Download the benchmark dataset, here is the website for dataset: https://www2.mmm.ucar.edu/wrf/users/benchmark/v44/benchdata_v44.html Choose one of them for testing. Unzip the file. copy the wrfbdy_d01, wrfinput_d01 and namelist.input file to WRF/test/em_real file. change the namelist.input file, 14 means adios2 as the I/O for WRF.

io_form_history = 14
io_form_restart = 14
frames_per_outfile   = 1000000,
#if a specific location is wanted
history_outname = '/path/directory/wrfout_d<domain>_<date>'

Run wrf with adios2 add adios2.xml file in the same folder(wrf/test/em_real) Here is an example of an adios2.xml file:

<?xml version="1.0"?>
<adios-config>
  <io name="/Path/to/wrfout_d01_2019-11-26_12:00:00">
    <engine type="BP4">
      <parameter key="RendezvousReaderCount" value="0"/>
      <parameter key="QueueLimit" value="1"/>
      <parameter key="QueueFullPolicy" value="Discard"/>
      <parameter key="OpenTimeoutSecs" value="30.0"/>
    </engine>
  </io>
</adios-config>

module load adios2/2.9.1-6fh7kh2
export DIR=~/Build_WRF/LIBRARIES
export CC=gcc
export CXX=g++
export FC=gfortran
export F77=gfortran
export HDF5=$DIR
export LD_LIBRARY_PATH=$DIR/lib:$LD_LIBRARY_PATH
export NETCDF=$DIR
export PATH=$DIR/bin:$PATH
export LD_LIBRARY_PATH=/mnt/repo/software/spack/spack/opt/spack/linux-ubuntu22.04-skylake_avx512/gcc-11.3.0/adios2-2.9.1-6fh7kh2v3tombadg56kmop72sjhf23dg/lib:$LD_LIBRARY_PATH
./wrf.exe or(mpirun -np <number of process> ./wrf.exe

The BP5 output will have mmd.0 file and the BP4 does not have mmd.0 file THe output file has the same name for different machine: "wrfout_d01_2019-11-26_12:00:00"

Post processing part

Use the python script:
https://github.com/grc-iit/coeus-adapter/blob/ML-assist/test/jarvis/jarvis_coeus/jarvis_coeus/post_wrf/post_wrf.py

Run this Python script with the adios2.xml file in the same folder.

The procedure to run WRF in Jarvis

change the adios2.xml file to this

<?xml version="1.0"?>
<adios-config>

  <io name="wrfout_d01_2019-11-26_12:00:00">

    <engine type="Plugin">
      <parameter key="PluginName" value="hermes"/>
      <parameter key="PluginLibrary" value="hermes_engine"/>
      <parameter key="ppn" value="16"/>
      <parameter key="db_file" value="/mnt/nvme/hxu40/metadata.db"/>
    </engine>

  </io>
</adios-config>

Then export these system parameter

export LD_LIBRARY_PATH=~/adapter/coeus-adapter/build/bin:$LD_LIBRARY_PATH
export PATH=~/adapter/coeus-adapter/build/bin:$PATH
module load adios2/2.9.1-6fh7kh2
spack load hermes@master
export DIR=~/Build_WRF_2/LIBRARIES
export LD_LIBRARY_PATH=$DIR/lib:$LD_LIBRARY_PATH
export PATH=$DIR/bin:$PATH
export LD_LIBRARY_PATH=/mnt/repo/software/spack/spack/opt/spack/linux-ubuntu22.04-skylake_avx512/gcc-11.3.0/adios2-2.9.1-6fh7kh2v3tombadg56kmop72sjhf23dg/lib:$LD_LIBRARY_PATH

jarvis pipeline append hermes_run --sleep=5
jarvis pipeline append wrf wrf_location=~/Build_WRF_2/WRFV4.5.1/test/em_real nprocs=4 ppn=6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly