Phase out of Izumi for our testing

[ekluzek]

Izumi has been a great test platform for us that provides several benefits:

1. The Nag compiler which catches issues other compilers haven't (made it easier to find problems)
2. Machine available when NCAR HPC is not
3. Easy to run quickly without waiting in the batch queue
4. Different compiler versions than NCAR HPC which can helps us to identify problems with compiler versions
5. Same as above but doubles our compiler version testing environment
6. Helps us ensure inputdata are rimported
7. Similar environment to University users

However, we are running into these problems:

• Hardware is beyond it's expected life
• Not well supported by NRIT, with only one admin with experience on it
• NRIT wants us to move away from having a supported cluster in CGD
• We are having environment problems where it works for some, but not all of us

As such we need to have a plan to move our testing away from Izumi. We need to look at this as a group and decide what that will look like. Some tests might need to be moved to Derecho. Possibly we can change warnings to errors on other compilers (as being done in CMEPS/CDEPS) on NCAR HPC. Possibly we can get access to another machine to serve the purpose that Izumi does.

This came up at today's CSEG meeting and was discussed there.

@briandobbins

[ekluzek]

Tasks that come to mind for me:

• Run the izumi test list on Derecho and just see what it does
• Examine test list to see if there are tests only done on izumi (make sure they have a copy on Derecho)
• Add a test list for Casper to help replace izumi (but see Single Site Test Attempt on Casper (fails) #2293) (would have to only be single point sites)
• Move Derecho single point sites for testing to Casper
• Examine what CMEPS/CDEPS does for warnings and replicate it for CTSM
• Figure out a protocol for ensuring inputfiles added to XML get rimported

Longer term:

• Ask CISL to add a NAG license to Derecho (move Izumi nag tests to that)
• Look into options for another machine to fill this role
• Get the team access to that machine
• Move the Izumi test list over to that machine