Update main_v10.0-ref after #1837. (#1847)

* Per #1789, remove duplicate plot_point_obs configuration section. (#1790) * Per #1798, fix up the read_tmpe_dataplane.py script to handle a grid string or dictionary. (#1799) * Github Issue #1801: Comment out code that checks for BEST track to support extra-tropical cyclone tracks not verified against BEST tracks. * Bugfix 1395 main_v10.0 comp script (#1796) * Updating compile script and associated config files for met-10.0.0 * Removing old config files * install_met_env.het * Updated orion file * Added stampede2 config file * Adding modulefiles for various machines * Feature main_v10.0 bugfix_checklist (#1803) * Recommended updates to clarify the use of projects and milestones for bugfixes. * Making details consistent across all issue templates. * Per met-help question (https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99964) clarify the description of the obs_thresh option. * Update README.md Adding text for GitHub Discussions * changed non-unicode apostrophe and fixed typo in URL (#1813) * Correct formatting errors in release notes. * #1817 Added to_north argument at two_to_one() * #1817 Calls two_to_one() with false for the third argument, if latitude moves north to south * #1508 checking stat() methoid returns no error before looking at FILE flag Co-authored-by: bikegeek <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: George McCabe <[email protected]> Co-authored-by: Howard Soh <[email protected]> Co-authored-by: hsoh-u <[email protected]>
dtcenter · Jul 6, 2021 · 70bf06b · 70bf06b
1 parent e903f8f
commit 70bf06b
Show file tree

Hide file tree

Showing 37 changed files with 499 additions and 226 deletions.
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -48,8 +48,9 @@ Describe the steps to reproduce the behavior:
 - [ ] Select **requestor(s)**
 
 ### Projects and Milestone ###
-- [ ] Review **projects** and select relevant **Repository** and **Organization** ones or add "alert:NEED PROJECT ASSIGNMENT" label
-- [ ] Select **milestone** to relevant bugfix version
+- [ ] Select **Organization** level **Project** for support of the current coordinated release
+- [ ] Select **Repository** level **Project** for development toward the next official release or add **alert: NEED PROJECT ASSIGNMENT** label
+- [ ] Select **Milestone** as the next bugfix version
 
 ## Define Related Issue(s) ##
 Consider the impact to the other METplus components.
@@ -68,10 +69,15 @@ Branch name: `bugfix_<Issue Number>_main_<Version>_<Description>`
 - [ ] Submit a pull request to merge into **main_\<Version>**.
 Pull request: `bugfix <Issue Number> main_<Version> <Description>`
 - [ ] Define the pull request metadata, as permissions allow.
-Select: **Reviewer(s)**, **Project(s)**, **Milestone**, and **Linked issues**
+Select: **Reviewer(s)** and **Linked issues**
+Select: **Organization** level software support **Project** for the current coordinated release
+Select: **Milestone** as the next bugfix version
 - [ ] Iterate until the reviewer(s) accept and merge your changes.
 - [ ] Delete your fork or branch.
 - [ ] Complete the steps above to fix the bug on the **develop** branch.
 Branch name:  `bugfix_<Issue Number>_develop_<Description>`
 Pull request: `bugfix <Issue Number> develop <Description>`
+Select: **Reviewer(s)** and **Linked issues**
+Select: **Repository** level development cycle **Project** for the next official release
+Select: **Milestone** as the next official version
 - [ ] Close this issue.
diff --git a/.github/ISSUE_TEMPLATE/enhancement_request.md b/.github/ISSUE_TEMPLATE/enhancement_request.md
@@ -38,8 +38,8 @@ Consider breaking the enhancement down into sub-issues.
 - [ ] Select **requestor(s)**
 
 ### Projects and Milestone ###
-- [ ] Review **projects** and select relevant **Repository** and **Organization** ones or add "alert:NEED PROJECT ASSIGNMENT" label
-- [ ] Select **milestone** to next major version milestone or "Future Versions"
+- [ ] Select **Repository** and/or **Organization** level **Project(s)** or add **alert: NEED PROJECT ASSIGNMENT** label
+- [ ] Select **Milestone** as the next official version or **Future Versions**
 
 ## Define Related Issue(s) ##
 Consider the impact to the other METplus components.
@@ -58,7 +58,9 @@ Branch name: `feature_<Issue Number>_<Description>`
 - [ ] Submit a pull request to merge into **develop**.
 Pull request: `feature <Issue Number> <Description>`
 - [ ] Define the pull request metadata, as permissions allow.
-Select: **Reviewer(s)**, **Project(s)**, **Milestone**, and **Linked issues**
+Select: **Reviewer(s)** and **Linked issues**
+Select: **Repository** level development cycle **Project** for the next official release
+Select: **Milestone** as the next official version
 - [ ] Iterate until the reviewer(s) accept and merge your changes.
 - [ ] Delete your fork or branch.
 - [ ] Close this issue.
diff --git a/.github/ISSUE_TEMPLATE/new_feature_request.md b/.github/ISSUE_TEMPLATE/new_feature_request.md
@@ -42,8 +42,8 @@ Consider breaking the new feature down into sub-issues.
 - [ ] Select **requestor(s)**
 
 ### Projects and Milestone ###
-- [ ] Review **projects** and select relevant **Repository** and **Organization** ones or add "alert:NEED PROJECT ASSIGNMENT" label
-- [ ] Select **milestone** to next major version milestone or "Future Versions"
+- [ ] Select **Repository** and/or **Organization** level **Project(s)** or add **alert: NEED PROJECT ASSIGNMENT** label
+- [ ] Select **Milestone** as the next official version or **Future Versions**
 
 ## Define Related Issue(s) ##
 Consider the impact to the other METplus components.
@@ -62,7 +62,9 @@ Branch name: `feature_<Issue Number>_<Description>`
 - [ ] Submit a pull request to merge into **develop**.
 Pull request: `feature <Issue Number> <Description>`
 - [ ] Define the pull request metadata, as permissions allow.
-Select: **Reviewer(s)**, **Project(s)**, **Milestone**, and **Linked issues**
+Select: **Reviewer(s)** and **Linked issues**
+Select: **Repository** level development cycle **Project** for the next official release
+Select: **Milestone** as the next official version
 - [ ] Iterate until the reviewer(s) accept and merge your changes.
 - [ ] Delete your fork or branch.
 - [ ] Close this issue.
diff --git a/.github/ISSUE_TEMPLATE/sub-issue.md b/.github/ISSUE_TEMPLATE/sub-issue.md
@@ -28,5 +28,5 @@ This is a sub-issue of #*List the parent issue number here*.
 - [ ] Select **requestor(s)**
 
 ### Projects and Milestone ###
-- [ ] Review **projects** and select relevant **Repository** and **Organization** ones or add "alert:NEED PROJECT ASSIGNMENT" label
-- [ ] Select **milestone** to next major version milestone or "Future Versions"
+- [ ] Select **Repository** and/or **Organization** level **Project(s)** or add **alert: NEED PROJECT ASSIGNMENT** label
+- [ ] Select **Milestone** as the next official version or **Future Versions**
diff --git a/.github/ISSUE_TEMPLATE/task.md b/.github/ISSUE_TEMPLATE/task.md
@@ -38,8 +38,8 @@ Consider breaking the task down into sub-issues.
 - [ ] Select **requestor(s)**
 
 ### Projects and Milestone ###
-- [ ] Review **projects** and select relevant **Repository** and **Organization** ones or add "alert:NEED PROJECT ASSIGNMENT" label
-- [ ] Select **milestone** to next major version milestone or "Future Versions"
+- [ ] Select **Repository** and/or **Organization** level **Project(s)** or add **alert: NEED PROJECT ASSIGNMENT** label
+- [ ] Select **Milestone** as the next official version or **Future Versions**
 
 ## Define Related Issue(s) ##
 Consider the impact to the other METplus components.
@@ -58,7 +58,9 @@ Branch name: `feature_<Issue Number>_<Description>`
 - [ ] Submit a pull request to merge into **develop**.
 Pull request: `feature <Issue Number> <Description>`
 - [ ] Define the pull request metadata, as permissions allow.
-Select: **Reviewer(s)**, **Project(s)**, **Milestone**, and **Linked issues**
+Select: **Reviewer(s)** and **Linked issues**
+Select: **Repository** level development cycle **Project** for the next official release
+Select: **Milestone** as the next official version
 - [ ] Iterate until the reviewer(s) accept and merge your changes.
 - [ ] Delete your fork or branch.
 - [ ] Close this issue.
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -18,7 +18,9 @@ See the [METplus Workflow](https://dtcenter.github.io/METplus/Contributors_Guide
 - [ ] Complete the PR definition above.
 - [ ] Ensure the PR title matches the feature or bugfix branch name.
 - [ ] Define the PR metadata, as permissions allow.
-Select: **Reviewer(s)**, **Project(s)**, and **Milestone**
-- [ ] After submitting the PR, select **Linked Issues** with the original issue number.
+Select: **Reviewer(s)**
+Select: **Organization** level software support **Project** or **Repository** level development cycle **Project**
+Select: **Milestone** as the version that will include these changes
+- [ ] After submitting the PR, select **Linked issues** with the original issue number.
 - [ ] After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
 - [ ] Close the linked issue and delete your feature or bugfix branch from GitHub.
diff --git a/README.md b/README.md
@@ -2,4 +2,4 @@
 
 This repository contains the source code for the Model Evaluation Tools package (met), unit test code (test), and scripts used to build and test the code (scripts).
 
-Please see the MET website (https://dtcenter.org/community-code/model-evaluation-tools-met) and direct questions to [email protected].
+Please see the [MET website](https://dtcenter.org/community-code/model-evaluation-tools-met) for more information.  Support	for the	METplus	components is provided through the [METplus Discussions](https://github.com/dtcenter/METplus/discussions) forum.  Users are welcome and encouraged to answer or address each other's questions there!  For more information, please read "[Welcome to the METplus Components Discussions](https://github.com/dtcenter/METplus/discussions/939)".
diff --git a/met/data/wrappers/read_tmp_dataplane.py b/met/data/wrappers/read_tmp_dataplane.py
@@ -17,14 +17,19 @@
 ds = nc.Dataset(netcdf_filename, 'r')
 met_data = ds['met_data'][:]
 met_attrs = {}
+
+# grid is defined as a dictionary or string
 grid = {}
 for attr, attr_val in ds.__dict__.items():
-    if 'grid' in attr:
+    if 'grid.' in attr:
         grid_attr = attr.split('.')[1]
         grid[grid_attr] = attr_val
     else:
         met_attrs[attr] = attr_val
-met_attrs['grid'] = grid
+
+if grid:
+    met_attrs['grid'] = grid
+
 met_attrs['name'] = met_attrs['name_str']
 del met_attrs['name_str']
 met_info['met_data'] = met_data

diff --git a/met/docs/Users_Guide/config_options.rst b/met/docs/Users_Guide/config_options.rst
@@ -2342,10 +2342,11 @@ e.g. nc_var_str = "MIN";
 **obs_thresh**
 
 The "obs_thresh" entry is an array of thresholds for filtering observation
-values prior to applying ensemble verification logic. The default setting
-of NA means that no observations should be filtered out. Verification output
-will be computed separately for each threshold specified. This option may be
-set separately for each obs.field entry.
+values prior to applying ensemble verification logic. They specify the values
+to be included in the verification, not excluded. The default setting of NA,
+which always evaluates to true, means that all observations should be used.
+Verification output will be computed separately for each threshold specified.
+This option may be set separately for each obs.field entry.
 
 .. code-block:: none
 		

diff --git a/met/docs/Users_Guide/plotting.rst b/met/docs/Users_Guide/plotting.rst
@@ -201,131 +201,6 @@ Users are encouraged to define as many **point_data** array entries as needed to
 
 For each observation, this tool stores the observation latitude, longitude, and value. However, unless the **dotsize(x)** function is not constant or the **fill_plot_info.flag** entry is set to true, the observation value is simply set to a flag value. For each **plot_data** array entry, the tool stores and plots only the unique combination of observation latitude, longitude, and value. Therefore multiple obsevations at the same location will typically be plotted as a single circle.
 
-plot_point_obs configuration file
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The default configuration file for the Plot-Point-Obs tool named **PlotPointObsConfig_default** can be found in the installed *share/met/config* directory. The contents of the configuration file are described in the subsections below.
-
-Note that environment variables may be used when editing configuration files, as described in :numref:`pb2nc configuration file` for the PB2NC tool.
-
-______________________
-
-.. code-block:: none
-
-  grid_data = {
-
-     field = [];
-
-     grid_plot_info = {
-        color_table   = "MET_BASE/colortables/met_default.ctable";
-        plot_min      = 0.0;
-        plot_max      = 0.0;
-        colorbar_flag = TRUE;
-     }
-  }
-
-The **grid_data** dictionary defines a gridded field of data to be plotted as a base image prior to plotting point locations on top of it. The data to be plotted is specified by the **field** array. If **field** is empty, no base image will be plotted. If **field** has length one, the requested data will be read from the input file specified by the **-plot_grid** command line argument.
-
-The **grid_plot_info** dictionary inside **grid_data** specifies the options for for plotting the gridded data. The options within **grid_plot_info** are described in :numref:`config_options`.
-
-______________________
-
-.. code-block:: none
-
-  point_data = [
-    { fill_color = [ 255, 0, 0 ]; }
-  ];
-
-The **point_data** entry is an array of dictionaries. Each dictionary may include a list of filtering, data processing, and plotting options, described below. For each input point observation, the tool checks the **point_data** filtering options in the order specified. The point information is added to the first matching array entry. The default entry simply specifies that all points be plotted red.
-
-______________________
-
-.. code-block:: none
-
-  msg_typ     = [];
-  sid_inc     = [];
-  sid_exc     = [];
-  obs_var     = [];
-  obs_quality = [];
-  
-The options listed above define filtering criteria for the input point observation strings. If empty, no filtering logic is applied. If a comma-separated list of strings is provided, only those observations meeting all of the criteria are included. The **msg_typ** entry specifies the message type. The **sid_inc** and **sid_exc** entries explicitly specify station id's to be included or excluded. The **obs_var** entry specifies the observation variable names, and **obs_quality** specifies quality control strings.
-
-______________________
-
-.. code-block:: none
-
-  obs_gc      = [];
-
-When using older point observation files which have GRIB codes, the **obs_gc** entry specifies a list of integer GRIB codes to be included.
-
-______________________
-
-.. code-block:: none
-
-  valid_beg   = "";
-  valid_end   = "";
-
-The **valid_beg** and **valid_end** options are time strings which specify a range of dates to be included. When left to their default empty strings no time filtering is applied.
-
-______________________
-
-.. code-block:: none
-
-  lat_thresh  = NA;
-  lon_thresh  = NA;
-  elv_thresh  = NA;
-  hgt_thresh  = NA;
-  prs_thresh  = NA;
-  obs_thresh  = NA;
-
-The options listed above define filtering thresholds for the input point observation values. The default NA thresholds always evaluate to true and therefore apply no filtering. The **lat_thresh** and **lon_thresh** thresholds filter the latitude and longitude of the point observations, respectively. The **elv_thresh** threshold filters by the station elevation. The **hgt_thresh** and **prs_thresh** thresholds filter by the observation height and pressure level. The **obs_thresh** threshold filters by the observation value.
-
-______________________
-
-.. code-block:: none
-
-  convert(x)    = x;
-  censor_thresh = [];
-  censor_val    = [];
-  
-The **convert(x)** function, **censor_thresh** option, and **censor_val** option may be specified separately for each **point_data** array entry to transform the observation values prior to plotting. These options are further described in :numref:`config_options`.
-
-______________________
-
-.. code-block:: none
-
-   dotsize(x) = 10;
-
-The **dotsize(x)** function defines the size of the circle to be plotted as a function of the observation value. The default setting shown above defines the dot size as a constant value.
-
-______________________
-
-.. code-block:: none
-
-  line_color = [];
-  line_width = 1;
-
-The **line_color** and **line_width** entries define the color and thickness of the outline for each circle plotted. When **line_color** is left as an empty array, no outline is drawn. Otherwise, **line_color** should be specified using 3 intergers between 0 and 255 to define the red, green, and blue components of the color.
-
-______________________
-
-.. code-block:: none
-
-  fill_color = [];
-  fill_plot_info = { // Overrides fill_color
-    flag          = FALSE;
-    color_table   = "MET_BASE/colortables/met_default.ctable";
-    plot_min      = 0.0;
-    plot_max      = 0.0;
-    colorbar_flag = TRUE;
-  }
-
-The circles are filled in based on the setting of the **fill_color** and **fill_plot_info** entries. As described above for **line_color**, if **fill_color** is empty, the points are not filled in. Otherwise, **fill_color** must be specified using 3 integers between 0 and 255. If **fill_plot_info.flag** is set to true, then its settings override **fill_color**. The **fill_plot_info** dictionary defines a colortable which is used to determine the color to be used based on the observation value.
-
-Users are encouraged to define as many **point_data** array entries as needed to filter and plot the input observations in the way they would like. Each point observation is plotted using the options specified in the first matching array entry. Note that the filtering, processing, and plotting options specified inside each **point_data** array entry take precedence over ones specified at the higher level of configuration file context.
-
-For each observation, this tool stores the observation latitude, longitude, and value. However, unless the **dotsize(x)** function is not constant or the **fill_plot_info.flag** entry is set to true, the observation value is simply set to a flag value. For each **plot_data** array entry, the tool stores and plots only the unique combination of observation latitude, longitude, and value. Therefore multiple obsevations at the same location will typically be plotted as a single circle.
-
 .. _plot_data_plane-usage:
 
 plot_data_plane usage

diff --git a/met/docs/Users_Guide/release-notes.rst b/met/docs/Users_Guide/release-notes.rst
@@ -21,8 +21,8 @@ MET Version |version| release notes (|release_date|)
    * Correct the time offset for tests in unit_plot_data_plane.xml (`#1677 <http://github.com/dtcenter/MET/issues/1677>`_).
    * Enhance the sample plotting R-script to read output from different versions of MET (`#1653 <http://github.com/dtcenter/MET/issues/1653>`_).
    * Update the default configuration options to compile the development code with the debug (-g) option and the production code without it (`#1788 <http://github.com/dtcenter/MET/issues/1788>`_).
-   * Update MET to compile using GCC version 10 (`#1552 https://github.com/dtcenter/MET/issues/1552`_).
-   * Update MET to compile using PGI version 20 (`#1317 https://github.com/dtcenter/MET/issues/1317`_).
+   * Update MET to compile using GCC version 10 (`#1552 <https://github.com/dtcenter/MET/issues/1552>`_).
+   * Update MET to compile using PGI version 20 (`#1317 <https://github.com/dtcenter/MET/issues/1317>`_).
 
 * Documentation:
 

diff --git a/met/scripts/Rscripts/plot_tcmpr.R b/met/scripts/Rscripts/plot_tcmpr.R
@@ -477,12 +477,14 @@ for(i in 1:length(info_list)) {
   cat("Found ", length(uniq_list), " unique entries for ", info_list[i], ": ",
       paste(uniq_list, collapse=", "), "\n", sep='');
 
+  # Comment out to support plotting extra-tropical cyclone tracks not
+  # verified against BEST tracks
   # Check for a single BDECK model
-  if(info_list[i] == "BMODEL" & length(uniq_list) != 1) {
-    cat("ERROR: Must have exactly 1 BDECK model name.  ",
-        "Try setting \"-bmodel name\" in the \"-filter\" option.\n");
-    quit(status=1);
-  }
+  #if(info_list[i] == "BMODEL" & length(uniq_list) != 1) {
+  #  cat("ERROR: Must have exactly 1 BDECK model name.  ",
+  #      "Try setting \"-bmodel name\" in the \"-filter\" option.\n");
+  #  quit(status=1);
+  #}
 }
 
 ########################################################################

diff --git a/met/src/basic/vx_log/file_fxns.cc b/met/src/basic/vx_log/file_fxns.cc
@@ -28,8 +28,7 @@ using namespace std;
 
 bool is_regular_file(const char *path) {
    struct stat path_stat;
-   stat(path, &path_stat);
-   return(S_ISREG(path_stat.st_mode));
+   return(!stat(path, &path_stat) && S_ISREG(path_stat.st_mode));
 }
 
 ////////////////////////////////////////////////////////////////////////

diff --git a/met/src/basic/vx_util/data_plane.cc b/met/src/basic/vx_util/data_plane.cc
@@ -410,7 +410,7 @@ return;
 
 ///////////////////////////////////////////////////////////////////////////////
 
-int DataPlane::two_to_one(int x, int y) const {
+int DataPlane::two_to_one(int x, int y, bool to_north) const {
    int n;
 
    if((x < 0) || (x >= Nx) || (y < 0) || (y >= Ny)) {
@@ -420,7 +420,7 @@ int DataPlane::two_to_one(int x, int y) const {
       exit(1);
    }
 
-   n = y*Nx + x;   //  don't change this!  lots of downstream code depends on this!
+   n = (to_north ? y : (Ny-1-y))*Nx + x;    //  don't change this!  lots of downstream code depends on this!
 
    return(n);
 }

diff --git a/met/src/basic/vx_util/data_plane.h b/met/src/basic/vx_util/data_plane.h
@@ -109,7 +109,7 @@ class DataPlane {
 
       void replace_bad_data(const double value);
 
-      int  two_to_one(int x, int y) const;
+      int  two_to_one(int x, int y, bool to_north=true) const;
       void one_to_two(int n, int &x, int &y) const;
 
       bool s_is_on(int x, int y) const;