Feature 2003 briercl

[JohnHalleyGotway]

Expected Differences

Note that this PR includes 3 main changes:

1. Enable the computation of BRIERCL and BSS in Series-Analysis by reading climo mean/stdev and deriving probabilities from them.
2. Enhance the climo_cdf dictionary to support the direct computation of climo probabilities rather than sampling from the climo distribution. The former is more intuitive, while the latter is in place to mimic logic used by NOAA/EMC.
3. Enables Series-Analysis to define block_size on the fly as the full size of the grid. Setting block_size <= 0 tells Series-Analysis to do this.

• Do these changes introduce new tools, command line arguments, or configuration file options? [Yes]
  
  If yes, please describe:
  
  Adds climo_cdf.direct_prob = TRUE/FALSE. Please see this user's guide section for a description of the functionality.

• Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [No]
  
  If yes, please describe:
  

Pull Request Testing

• Describe testing already performed for these changes:
  

Manually tested to confirm the changes work as expected. Added a new unit test and modified and existing one.

• Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
  

@georgemccabe, please confirm that all tests pass, review the code changes, and use of the climo_cdf.direct_prob option.

@j-opatz, please inspect the new Series-Analysis unit test, along with the output file it creates. Please review documentation updates to ensure they're accurate and complete.

• Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]

• Do these changes include sufficient testing updates? [Yes]

• Will this PR result in changes to the test suite? [Yes]
  
  If yes, describe the new output and/or changes to the existing output:
  

Adds a new test to unit_climatology_1.0deg.xml which creates a new output file:
climatology_1.0deg/series_analysis_PROB_CLIMO_1.0DEG.nc

This test verifies SREF probability of TMP > 273 vs GFS-Analysis and compared to NCEP's 1.0 degree climatology.

The new test sets block_size = -9999 and I modified another to set block_size = 0. The latter saves about 20 seconds of runtime since it doesn't have to loop through the files twice. The new test also sets climo_cdf.direct_prob = TRUE, which is the only example of that.

• Please complete this pull request review by [Wed 2/2/2022].
  

Pull Request Checklist

See the METplus Workflow for details.

• Complete the PR definition above.
• Ensure the PR title matches the feature or bugfix branch name.
• Define the PR metadata, as permissions allow.
  Select: Reviewer(s)
  Select: Organization level software support Project or Repository level development cycle Project
  Select: Milestone as the version that will include these changes
• After submitting the PR, select Linked issues with the original issue number.
• After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
• Close the linked issue and delete your feature or bugfix branch from GitHub.

[JohnHalleyGotway]

@georgemccabe notes that the unit_climatology_1.0deg.xml run failed because the input data is not present in the subset of data used for GHA. Here are instructions for updating the input dataset:

The data is on mohawk here:
/d2/www/dtcenter/dfiles/code/METplus/MET/MET_unit_test/main_v10.1

There are directories with the data and a tarfile that contains all of the data called unit_test-v10.1.tgz. You can add the new data you need, then recreate the tarfile. I would recommend copying the existing tarfile and adding today's date in case something gets messed up.
You'll have to run these commands with Docker Desktop started in the METplus develop branch:

cd METplus
docker build -t dtcenter/met-data-dev:develop ci/docker/docker_data --build-arg TARFILE_URL=https://dtcenter.ucar.edu/dfiles/code/METplus/MET/MET_unit_test/develop/unit_test.tgz --build-arg MOUNTPT=/data/input/MET_test_data/unit_test --build-arg DATA_DIR=/data/input/MET_test_data/unit_test --no-cache
docker push dtcenter/met-data-dev:develop

There is a sym link in the MET_unit_test/develop directory to the tarfile in main_v10.1 directory, but the link is named unit_test.tgz. If you use the same name for the tarfile in the main_v10.1 directory, then the above command should still work fine.

I'd really like to adjust this so that the tarfile contains a top level directory named "unit_test" so that it is easier to update the files and create a new tarfile. The DATA_DIR build arg would have to change to exclude the unit_test directory. I think that would be the only change but we should test that the automated tests still work after changing the format.

I suppose one of us could make those changes now and use this as an opportunity to make sure it works. Without this change it is kind of a pain to create the tgz file without including other existing tgz files in it.

Also you have to be the met_test user to modify the files.

[JohnHalleyGotway]

I added the necessary model_data/grib2/gfsanl and model_data/grib2/sref_pr files, and recreated a new tarfile, but using a top-level unit_test directory. Then I ran the following commands to regenerate the unit test data volume:

cd METplus
docker build -t dtcenter/met-data-dev:develop ci/docker/docker_data --build-arg TARFILE_URL=https://dtcenter.ucar.edu/dfiles/code/METplus/MET/MET_unit_test/develop/unit_test.tgz --build-arg MOUNTPT=/data/input/MET_test_data/unit_test --build-arg DATA_DIR=/data/input/MET_test_data --no-cache
docker push dtcenter/met-data-dev:develop

Editing to use MOUNTPT=/data/input/MET_test_data/unit_test as instructed by @georgemccabe

[JohnHalleyGotway]

@georgemccabe and @j-opatz, my docker push command completed. And I just pushed up a minor pb2nc log message change to this feature branch, which re-triggered the unit tests.

Good news, the automated testing worked really well! If you download the "logs" artifact, you'll see the only difference is the addition of this new output file:

ERROR: folder /data/output/met_test_truth missing 1 files
    climatology_1.0deg/series_analysis_PROB_CLIMO_1.0DEG.nc 

@j-opatz please also download the diff artifact so you can actually inspect the contents of this new output file.

[georgemccabe]

These changes include adding the climo_cdf.direct_prob option to GridStat and PointStat. It also adds the climo_cdf dictionary with cdf_bins, center_bins, and direct_prob to SeriesAnalysis.

I reviewed the automated test output and confirmed that the only difference is the newly added test output:

• climatology_1.0deg/series_analysis_PROB_CLIMO_1.0DEG.nc

I suggest updating the automated test logic to print an excerpt of the comp_dir.log file when the string "ERROR:" is found in it so it can be reviewed at a glance in the GitHub Actions log output instead of requiring the user to download the log files and review them. That is not mandatory and unrelated to this PR.

I approve.

[j-opatz]

The proposed documentation updates look good for the inclusion of direct_prob and make the user much more aware (in my opinion) of the circumstances surrounding the calculation of briercl.
Test output looks good (as George already noted), and I approve this PR for merge.