UFS-Coastal Input File Generation Scripts for Workflow

[mansurjisan]

@uturuncoglu , @janahaddad

UFS-Coastal Input File Generation Scripts for Workflow

Collection of scripts for generating input files for UFS-Coastal

Repository Link

UFS-Coastal-Inputs on GitHub

This is the continuation of this ticket oceanmodeling/ufs-weather-model#127 (comment)

Script update:

SCHISM Boundary Condition Generator (gen_bctides.py)

I updated SCHISM's boundary condition generation script (gen_bctides.py) to be used
within the UFS-Coastal for Ike Shinnecock and Duck, NC regression test cases.

Workflow diagram:

workflow

Features

• Multiple Boundary Types:

  • Type 3: Tidal boundaries (using TPXO or FES2014)
  • Type 4: Time-elevation boundaries (from elev.th file or HYCOM data)

• Data Sources:

  • Local time series files (elev.th)
  • HYCOM oceanographic data for:
    • Water elevation (elev2D.th.nc)
    • Temperature (TEM_3D.th.nc)
    • Salinity (SAL_3D.th.nc)

• Automated Processing:

  • Automatic boundary detection from hgrid.ll
  • Flexible grid file format support
  • some error handling features

Usage

1. Tidal Boundary for Ike Shinnecock Testcase:

python gen_bctides.py ../../data/hgrid.ll 2008-08-23 20 \
    --bc_mode tidal \
    --bc_type 3 \
    --constituents Q1,O1,P1,K1,N2,M2,S2,K2,Mm,Mf,M4,MN4,MS4,2N2,S1 \
    --database tpxo \
    --cutoff_depth 40 \
    --earth_tidal_potential Y

2. Time Series Boundary for Duck, NC Testcase (requires TPXO tide database access):

python gen_bctides.py ../../data/hgrid.ll 2012-10-27 2.333 \
    --bc_mode time-elev \
    --bc_type 4 \
    --elev_th elev.th \
    --vgrid vgrid.in

3. HYCOM Boundary:

python gen_bctides.py hgrid.ll 2024-01-01 10 \
    --bc_mode time-elev \
    --bc_type 4 \
    --elev_source hycom \
    --gen_bc elev,temp,salt \
    --vgrid vgrid.in

Required Files

• hgrid.ll: SCHISM horizontal grid file
• vgrid.in: Vertical grid file (for time-elevation mode)
• elev.th: Time series file (if using time series source)

Dependencies

• pyschism
• netCDF4
• numpy

schism_bc_in_ufs_coastal.pdf

[mansurjisan]

@josephzhang8 , @feiye-vims , @saeed-moghimi-noaa , @SorooshMani-NOAA ,

As we're integrating PySCHISM functionality into UFS-Coastal and other operational systems, I believe we should discuss the future development and maintenance of PySCHISM.

Some discussion points:

1. PySCHISM Development:

• Long-term maintenance strategy
• Integration with UFS-Coastal workflow
• Version control and release management
• Feature requests for operational needs
• Documentation improvements
• Testing framework

2. Alternative Approaches:
   • Standalone operational tools
   • Existing Python package alternatives

Would appreciate your insights on the best path forward for development and operational support.

[uturuncoglu]

@mansurjisan Let me know if scripts are ready to port into workflow.

[josephzhang8]

Thx @mansurjisan.

Currently VIMS team is maintaining pySCHISM, but due to sever manpower shortage we were only able to do bug fixes in the past years or so. Development of new capabilities is out of question. I would appreciate it if other teams can take charge.

[SorooshMani-NOAA]

@josephzhang8 I remember we had this discussion about PySCHISM vs PyLibs a while ago. We understand that it's not feasible to maintain two separate packages which do the same thing in the long run, especially given the developer availability; so we wanted to make sure we understand what the plans are for PySCHISM and PyLibs, and if we, for example, need to switch to PyLibs more instead of SCHISM.

If I remember correctly, at some point you mentioned VIMS wants to keep PySCHISM for its downloading capabilities, while PyLibs is better for its setup capabilities.

We just wanted to double check before either side invest more time and energy (i.e. scripting) on one or the other tool.

[josephzhang8]

It's not easy to merge the 2 packages due to very different starting points:

1. pySCHISM: written to make common tasks as easy as one-button
2. pyLibs: written to minimize dependencies with other packages that are being constantly updated

I personally use both depending on what tasks I want to accomplish. For model setup using HYCOM, atmos forcing, I use pySCHISM as it's more developed in those areas. I use pyLibs (schismview, schismcheck, pload) to viz inputs/outputs and also load from DEMs and generating b.c. part for hgrid. I'm sure there are other usages for either tool that I have not used.

It'd be good to have a meeting involving developers of the two tools. I did that a few years ago and the consensus was that it's premature to discuss merging. pyLibs developer has been reluctant to add HYCOM and atmos capabilities if doing so would mean he has to add dependencies; at the moment pyLibs has a very small core lib that proved to be powerful for what we are doing. Maybe you all have good ideas in this regard.

[mansurjisan]

@mansurjisan Let me know if scripts are ready to port into the workflow.

Hi @uturuncoglu , the script to generate the boundary condition for SCHISM is ready to be used within UFS-Coastal workflow. The script is located here: https://github.com/mansurjisan/UFS-Coastal-Inputs/blob/main/scripts/coastal_ike_shinnecock_atm2sch/gen_bctides.py

Also, i attached a PDF file at the end of this comment: #11 (comment)

We can talk about it more during our Monday meeting.

[uturuncoglu]

@mansurjisan Okay. Let me try to use it under workflow. I'll update you soon.

[mansurjisan]

Thanks @uturuncoglu . I forgot to mention that the script requires to have the TPXO data located in /home/$username/.local/share/tpxo directory. Please give it a try and let me know if any additional changes need to be made.

[mansurjisan]

I've developed a script to generate meteorological forcing for UFS-Coastal data atmospheric cap using the pySCHISM library. While the script successfully downloads and processes GFS and HRRR data from AWS, I discovered a limitation for historical test cases:

• Ike Shinnecock (2008-08-23)
• Duck, NC (2012-10-27)

The AWS archive only contains:

• HRRR data from late 2016
• GFS data from early 2021

Alternative data sources for historical cases:

1. ERA5 Reanalysis (recommended)
2. NOMADS archive
3. NCAR Research Data Archive

I plan to extend the script to include ERA5 data download functionality for historical cases in the next update.

The current script is available at:

https://github.com/mansurjisan/UFS-Coastal-Inputs/blob/main/scripts/coastal_ike_shinnecock_atm2sch/download_met.py

Usage examples:

# HRRR data
python download_met.py --start-date "2025-01-03 12:00" --rnday 1 --model hrrr --var uwind vwind prmsl --hgrid ../../data/hgrid.gr3

# GFS data
python download_met.py --start-date "2025-01-03 12:00" --rnday 1 --model gfs --var uwind vwind prmsl --hgrid ../../data/hgrid.gr3

# For Grib2 data format 
python download_met.py --start-date "2025-01-03 12:00" --rnday 1 --model gfs --format grib2
 --var uwind vwind prmsl --hgrid ../../data/hgrid.gr3

[josephzhang8]

There is a script for ERA5 under examples/Sflux. It requires users to download a key from ECMWF site.