LULC: Define Verification Tasks

[georgemccabe]

Sub-issue of #31 - LULC use case

Describe the Task

For verification, we have a couple options. Ideally Xin would work with George & Brianne to define a METplus configuration for this use case (even if it's somewhat contrived, so we can keep demonstrating METplus for I-WRF users), which would include identifying what observation data to use for validation, determining what statistics to calculate, etc. An alternative (or supplemental) solution would be for Xin to containerize his own custom validation scripts that he already used for his paper; that could allow users to generate similar stats/plots to what Xin published in his JGR paper, which would be nice.

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the task down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

0.3 (December 6, 2024)

Funding Source

I-WRF

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Task Checklist

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[levinzx]

The radar measurements are now available on GLADE:
/glade/campaign/ral/wsap/i-wrf/science_case2_LULC_modification/nexrad

For the verification, we may compare the the following between simulation and NEXRAD:
1) spatial pattern of 24-hour accumulated precipitation
2) spatial pattern of composite radar reflectivity at the time when the deep convective systems passing over the urban area.
3) evolution of domain averaged rain rate.

[JohnHalleyGotway]

Based on the description in the comment above, recommend configuring a METplus use case to:

1. Evaluate 24-hour precip and radar reflectivity.
2. For each, run the MET Grid-Stat tool to compute...

• contingency table counts and statistics
• consider computing fractions skill score
• consider computing the distance map line type

3. For each, run the MET MODE tool to perform and object-based evaluation.

Recommend meeting with @georgemccabe, @briannen, @JohnHalleyGotway, and @jaredalee to further discuss and refine the desired configuration.