From ee11fc1d24620cb5729926c8510ee88741c6bc52 Mon Sep 17 00:00:00 2001 From: John Halley Gotway Date: Wed, 4 Aug 2021 10:20:38 -0600 Subject: [PATCH] Per #1673, making the formatting of Gilleland-2020 part I and II references consistent. --- met/docs/Users_Guide/appendixD.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/met/docs/Users_Guide/appendixD.rst b/met/docs/Users_Guide/appendixD.rst index 2c370817a2..66c578e91a 100644 --- a/met/docs/Users_Guide/appendixD.rst +++ b/met/docs/Users_Guide/appendixD.rst @@ -77,9 +77,9 @@ All other verification scores with CIs in MET must be obtained through bootstrap 5. Calculate CIs for the parameters directly from the sample (see text below for more details) -Typically, a simple random sample is taken for step 2, and that is how it is done in MET. As an example of what happens in this step, suppose our sample is :math:`X_1,X_2,X_3,X_4`. Then, one possible replicate might be :math:`X_2,X_2,X_2,X_4`. Usually one samples :math:`m = n` points in this step, but there are cases where one should use :math:`m < n`. For example, when the underlying distribution is heavy-tailed, one should use a smaller size m than n (e.g., the closest integer value to the square root of the original sample size). See :ref:`Gilleland (2020 part II) ` for considerably more information about the issues with estimators that follow a heavy tailed distribution and the closely related issue of bootstrapping extreme-valued estimators, such as the maximum, in the atmospheric science domain. +Typically, a simple random sample is taken for step 2, and that is how it is done in MET. As an example of what happens in this step, suppose our sample is :math:`X_1,X_2,X_3,X_4`. Then, one possible replicate might be :math:`X_2,X_2,X_2,X_4`. Usually one samples :math:`m = n` points in this step, but there are cases where one should use :math:`m < n`. For example, when the underlying distribution is heavy-tailed, one should use a smaller size m than n (e.g., the closest integer value to the square root of the original sample size). See :ref:`Gilleland (2020, part II) ` for considerably more information about the issues with estimators that follow a heavy tailed distribution and the closely related issue of bootstrapping extreme-valued estimators, such as the maximum, in the atmospheric science domain. There are numerous ways to construct CIs from the sample obtained in step 4. MET allows for two of these procedures: the percentile and the BCa. The percentile is the most commonly known method, and the simplest to understand. It is merely the :math:`\alpha / 2` and :math:`1 - \alpha / 2` percentiles from the sample of statistics. Unfortunately, however, it has been shown that this interval is too optimistic in practice (i.e., it doesn't have accurate coverage). One solution is to use the BCa method, which is very accurate, but it is also computationally intensive. This method adjusts for bias and non-constant variance, and yields the percentile interval in the event that the sample is unbiased with constant variance. -If there is dependency in the sample, then it is prudent to account for this dependency in some way. :ref:`Gilleland (2010) ` describes the bootstrap procedure, along with the above-mentioned parametric methods, in more detail specifically for the verification application. If there is dependency in the sample, then it is prudent to account for this dependency in some way (see :ref:`Gilleland (2020) ` part I for an in-depth discussion of bootstrapping in the competing forecast verification domain). One method that is particularly appropriate for serially dependent data is the circular block resampling procedure for step 2. +If there is dependency in the sample, then it is prudent to account for this dependency in some way. :ref:`Gilleland (2010) ` describes the bootstrap procedure, along with the above-mentioned parametric methods, in more detail specifically for the verification application. If there is dependency in the sample, then it is prudent to account for this dependency in some way (see :ref:`Gilleland (2020, part I) ` part I for an in-depth discussion of bootstrapping in the competing forecast verification domain). One method that is particularly appropriate for serially dependent data is the circular block resampling procedure for step 2.