From 98d1f17d9b69880b33fb095fbcf0e0b4aa988502 Mon Sep 17 00:00:00 2001 From: John Halley Gotway Date: Fri, 28 May 2021 12:46:36 -0600 Subject: [PATCH] Per #1749, update the documentation about the HSS_EC statistic. --- met/docs/Users_Guide/appendixC.rst | 17 ++++++++++++++++- met/docs/Users_Guide/config_options.rst | 16 ++++++++++++++++ met/docs/Users_Guide/grid-stat.rst | 1 + met/docs/Users_Guide/point-stat.rst | 7 +++++++ 4 files changed, 40 insertions(+), 1 deletion(-) diff --git a/met/docs/Users_Guide/appendixC.rst b/met/docs/Users_Guide/appendixC.rst index 05905e9c67..6d8c81efb7 100644 --- a/met/docs/Users_Guide/appendixC.rst +++ b/met/docs/Users_Guide/appendixC.rst @@ -240,7 +240,7 @@ HK is also known as the True Skill Statistic (TSS) and less commonly (although p Heidke Skill Score (HSS) ~~~~~~~~~~~~~~~~~~~~~~~~ -Called "HSS" in CTS output :numref:`table_PS_format_info_CTS` +Called "HSS" in CTS output :numref:`table_PS_format_info_CTS` and "HSS" in MCTS output :numref:`table_PS_format_info_MCTS` HSS is a skill score based on Accuracy, where the Accuracy is corrected by the number of correct forecasts that would be expected by chance. In particular, @@ -252,6 +252,21 @@ where HSS can range from minus infinity to 1. A perfect forecast would have HSS = 1. +Heidke Skill Score - Expected Correct (HSS_EC) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Called "HSS_EC" in MCTS output :numref:`table_PS_format_info_MCTS` + +HSS_EC is a skill score based on Accuracy, where the Accuracy is corrected by the number of correct forecasts that would be expected by chance. In particular, + +.. math:: \text{HSS } = \frac{n_{11} + n_{00} - C_2}{T - C_2}, + +where + +The C_2 value is user-configurable with a default value of T divided by the number of contingency table categories. + +HSS_EC can range from minus infinity to 1. A perfect forecast would have HSS_EC = 1. + Odds Ratio (OR) ~~~~~~~~~~~~~~~ diff --git a/met/docs/Users_Guide/config_options.rst b/met/docs/Users_Guide/config_options.rst index b27e819908..b17f214e2b 100644 --- a/met/docs/Users_Guide/config_options.rst +++ b/met/docs/Users_Guide/config_options.rst @@ -1966,6 +1966,22 @@ intensive and slows down the runtime significantly. rank_corr_flag = FALSE; +**hss_ec_value** + +The "hss_ec_value" entry is a floating point number used in the computation +of the HSS_EC statistic in the MCTS line type. It specifies the expected +correct (EC) rate by chance for multi-category contingency tables. If set +to its default value of NA, it will automatically be replaced with 1.0 +divided by the MCTC table dimension. For example, for a 4x4 table, the +default hss_ec_value is 1.0 / 4 = 0.25. + +It set, it must greater than or equal to 0.0 and less than 1.0. A value of +0.0 produces an HSS_EC statistic equal to the Accuracy statistic. + +.. code-block:: none + + hss_ec_value = NA; + **duplicate_flag** The "duplicate_flag" entry specifies how to handle duplicate point diff --git a/met/docs/Users_Guide/grid-stat.rst b/met/docs/Users_Guide/grid-stat.rst index ff5808df9c..fbafab867a 100644 --- a/met/docs/Users_Guide/grid-stat.rst +++ b/met/docs/Users_Guide/grid-stat.rst @@ -225,6 +225,7 @@ __________________________ mpr_thresh = []; eclv_points = 0.05; rank_corr_flag = TRUE; + hss_ec_value = NA; tmp_dir = "/tmp"; output_prefix = ""; version = "VN.N"; diff --git a/met/docs/Users_Guide/point-stat.rst b/met/docs/Users_Guide/point-stat.rst index bfbbc53fba..6a6ae40198 100644 --- a/met/docs/Users_Guide/point-stat.rst +++ b/met/docs/Users_Guide/point-stat.rst @@ -338,6 +338,7 @@ ________________________ mpr_thresh = []; eclv_points = 0.05; rank_corr_flag = TRUE; + hss_ec_value = NA; sid_inc = []; sid_exc = []; duplicate_flag = NONE; @@ -914,6 +915,12 @@ The first set of header columns are common to all of the output files generated * - 38-40 - GER, :raw-html:`
` GER_BCL, :raw-html:`
` GER_BCU - Gerrity Score and bootstrap confidence limits + * - 41-43 + - HSS_EC, :raw-html:`
` HSS_EC_BCL, :raw-html:`
` HSS_EC_BCU + - Heidke Skill Score with user-specific expected correct and bootstrap confidence limits + * - 44 + - EC_VALUE + - Expected correct rate for HSS_EC .. _table_PS_format_info_PCT: