Feature #2757 allow missing docs

docs/Users_Guide/glossary.rst

@@ -12990,3 +12990,280 @@ METplus Configuration Glossary
      Specify the value for 'obtype_as_group_val_flag' in the MET configuration file for EnsembleStat.
 
      | *Used by:* EnsembleStat
+
+   ALLOW_MISSING_INPUTS
+     If True, report a warning instead of an error if a wrapper cannot find all
+     of its required input files. Only report an error if the number of runs
+     that successfully found input files does not meet the threshold defined
+     with :term:`INPUT_THRESH`.
+     Wrapper-specific versions of this variable are supported for many of the
+     wrappers. Defaults to False.
+
+     | *Used by:* Many
+
+   INPUT_THRESH
+     A decimal number between 0 and 1 that defines the threshold of successful
+     run times where all of the required input files were found.
+     If INPUT_THRESH is set to 0.5 and 3 out of the 4 run times (75% or 0.75)
+     successfully found the required input files, no errors will be reported.
+     However, if INPUT_THRESH is set to 0.5 and only 1 out of 4 run times
+     (25% or 0.25) found the required input files, an error will be reported
+     at the end of the run. The threshold is inclusive, so 2 out of 4 successful
+     run times (50% or 0.5) will not report an error.
+     This is only used if :term:`ALLOW_MISSING_INPUTS` is True.
+     Wrapper-specific versions of this variable are supported for many of the
+     wrappers.
+
+     | *Used by:* Many
+
+   ASCII2NC_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for ASCII2NC only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* ASCII2NC
+
+   ASCII2NC_INPUT_THRESH
+     Defines input threshold for ASCII2NC only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* ASCII2NC
+
+   ENSEMBLE_STAT_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for EnsembleStat only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* EnsembleStat
+
+   ENSEMBLE_STAT_INPUT_THRESH
+     Defines input threshold for EnsembleStat only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* EnsembleStat
+
+   GEN_ENS_PROD_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for GenEnsProd only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* GenEnsProd
+
+   GEN_ENS_PROD_INPUT_THRESH
+     Defines input threshold for GenEnsProd only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* GenEnsProd
+
+   GEN_VX_MASK_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for GenVxMask only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* GenVxMask
+
+   GEN_VX_MASK_INPUT_THRESH
+     Defines input threshold for GenVxMask only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* GenVxMask
+
+   GRID_DIAG_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for GridDiag only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* GridDiag
+
+   GRID_DIAG_INPUT_THRESH
+     Defines input threshold for GridDiag only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* GridDiag
+
+   GRID_STAT_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for GridStat only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* GridStat
+
+   GRID_STAT_INPUT_THRESH
+     Defines input threshold for GridStat only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* GridStat
+
+   IODA2NC_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for IODA2NC only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* IODA2NC
+
+   IODA2NC_INPUT_THRESH
+     Defines input threshold for IODA2NC only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* IODA2NC
+
+   MADIS2NC_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for MADIS2NC only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* MADIS2NC
+
+   MADIS2NC_INPUT_THRESH
+     Defines input threshold for MADIS2NC only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* MADIS2NC
+
+   MODE_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for MODE only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* MODE
+
+   MODE_INPUT_THRESH
+     Defines input threshold for MODE only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* MODE
+
+   MTD_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for MTD only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* MTD
+
+   MTD_INPUT_THRESH
+     Defines input threshold for MTD only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* MTD
+
+   PB2NC_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for PB2NC only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* PB2NC
+
+   PB2NC_INPUT_THRESH
+     Defines input threshold for PB2NC only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* PB2NC
+
+   PCP_COMBINE_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for PCPCombine only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* PCPCombine
+
+   PCP_COMBINE_INPUT_THRESH
+     Defines input threshold for PCPCombine only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* PCPCombine
+
+   PLOT_DATA_PLANE_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for PlotDataPlane only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* PlotDataPlane
+
+   PLOT_DATA_PLANE_INPUT_THRESH
+     Defines input threshold for PlotDataPlane only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* PlotDataPlane
+
+   PLOT_POINT_OBS_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for PlotPointObs only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* PlotPointObs
+
+   PLOT_POINT_OBS_INPUT_THRESH
+     Defines input threshold for PlotPointObs only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* PlotPointObs
+
+   POINT2GRID_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for Point2Grid only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* Point2Grid
+
+   POINT2GRID_INPUT_THRESH
+     Defines input threshold for Point2Grid only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* Point2Grid
+
+   POINT_STAT_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for PointStat only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* PointStat
+
+   POINT_STAT_INPUT_THRESH
+     Defines input threshold for PointStat only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* PointStat
+
+   REGRID_DATA_PLANE_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for RegridDataPlane only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* RegridDataPlane
+
+   REGRID_DATA_PLANE_INPUT_THRESH
+     Defines input threshold for RegridDataPlane only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* RegridDataPlane
+
+   SERIES_ANALYSIS_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for SeriesAnalysis only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* SeriesAnalysis
+
+   SERIES_ANALYSIS_INPUT_THRESH
+     Defines input threshold for SeriesAnalysis only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* SeriesAnalysis
+
+   TC_DIAG_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for TCDiag only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* TCDiag
+
+   TC_DIAG_INPUT_THRESH
+     Defines input threshold for TCDiag only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* TCDiag
+
+   TC_GEN_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for TCGen only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* TCGen
+
+   TC_GEN_INPUT_THRESH
+     Defines input threshold for TCGen only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* TCGen
+
+   WAVELET_STAT_ALLOW_MISSING_INPUTS
+     Activates allow missing inputs logic for WaveletStat only.
+     See :term:`ALLOW_MISSING_INPUTS` for details.
+
+     | *Used by:* WaveletStat
+
+   WAVELET_STAT_INPUT_THRESH
+     Defines input threshold for WaveletStat only.
+     See :term:`INPUT_THRESH` for details.
+
+     | *Used by:* WaveletStat

docs/Users_Guide/systemconfiguration.rst

@@ -2138,6 +2138,96 @@ can be simplified as::
     INPUT_TEMPLATE = ensbegin_end_incr(1,8,1,2).nc
 
 
+.. _allow-missing-inputs:
+
+Allow Missing Inputs
+--------------------
+
+When any of the required input files for a given METplus run time are not found,
+an error is reported. In result, the entire METplus run fails.
+In some cases, users may expect a certain number of inputs to be unavailable
+and do not want the entire run to fail when this happens.
+
+The :term:`ALLOW_MISSING_INPUTS` config variable can be set to **True** to
+report a warning when required inputs are not found for a given run time.
+An error at the end of the METplus run will only be reported if the number
+of successful runs does not meet the value defined by :term:`INPUT_THRESH`.
+The value of :term:`INPUT_THRESH` should be a decimal number between 0 and 1.
+The default value is 0.0, so any missing input files in a run will still report
+an error unless this value is changed.
+
+The threshold is compared to the results of each item in the
+:ref:`Process_List`, so each wrapper listed in the **PROCESS_LIST** must meet
+the threshold to prevent an error.
+
+There are wrapper-specific versions of both :term:`ALLOW_MISSING_INPUTS` and
+:term:`INPUT_THRESH` for most of the wrappers,
+e.g. :term:`GRID_STAT_ALLOW_MISSING_INPUTS` and :term:`GRID_STAT_INPUT_THRESH`.
+Refer to the :ref:`python_wrappers` chapter or the :ref:`METplus_glossary`
+to see which variables are supported.
+
+**Example 1**::
+
+    [config]
+    PROCESS_LIST = RegridDataPlane, GridStat
+    VALID_TIME_FMT = %Y%m%d%H
+    VALID_BEG = 2024020301
+    VALID_BEG = 2024020310
+    VALID_INCREMENT = 1H
+    LEAD_SEQ = 0
+
+    ALLOW_MISSING_INPUTS = True
+    INPUT_THRESH = 0.6
+
+In this example, 10 valid times will be run, so there will be 10 calls to
+RegridDataPlane and 10 calls to GridStat. The input threshold is set
+to 60%, so if 6 or more of the RegridDataPlane runs *and*
+6 or more of the GridStat runs successfully find all of the required files,
+an error will not be reported. If 5 or fewer runs for either wrapper succeed,
+then an error will be reported.
+
+**Example 2**::
+
+
+    [config]
+    PROCESS_LIST = RegridDataPlane, GridStat
+    VALID_TIME_FMT = %Y%m%d%H
+    VALID_BEG = 2024020301
+    VALID_BEG = 2024020310
+    VALID_INCREMENT = 1H
+    LEAD_SEQ = 0
+
+    GRID_STAT_ALLOW_MISSING_INPUTS = True
+    INPUT_THRESH = 0.6
+
+In this case, only GridStat wrapper will allow missing inputs.
+At least 60% of the GridStat runs
+must successfully find the required input files to prevent an error.
+Any missing inputs for RegridDataPlane will result in an error.
+
+**Example 3**::
+
+
+    [config]
+    PROCESS_LIST = RegridDataPlane, GridStat
+    VALID_TIME_FMT = %Y%m%d%H
+    VALID_BEG = 2024020301
+    VALID_BEG = 2024020310
+    VALID_INCREMENT = 1H
+    LEAD_SEQ = 0
+
+    ALLOW_MISSING_INPUTS = True
+    REGRID_DATA_PLANE_INPUT_THRESH = 0.9
+    GRID_STAT_INPUT_THRESH = 0.6
+
+In this case, both GridStat and RegridDataPlane wrappers allow missing inputs,
+but the threshold to prevent an error differs between wrappers.
+At least 90% of the RegridDataPlane runs
+must successfully find the required input files to prevent an error.
+At least 60% of the GridStat runs
+must successfully find the required input files to prevent an error.
+Any missing inputs for RegridDataPlane will result in an error.
+
 .. _metplus-control-met:
 
 How METplus controls MET configuration variables