Per #2092, update the overview section to better describe the pre-pro…

…cessing tools and mention SonarQube. Also swap in the 10.1.0 flowchart.

met/docs/Users_Guide/overview.rst

@@ -36,7 +36,7 @@ MET components
 
 The major components of the MET package are represented in :numref:`overview-figure`. The main stages represented are input, reformatting, plotting, intermediate output, statistical analyses, and output and aggregation/analysis. The MET-TC package functions independently of the other MET modules, as indicated in the Figure. Each of these stages is described further in later sections. For example, the input and output formats are discussed in :numref:`data_io` as well as in the sections associated with each of the statistics modules. MET input files are represented on the far left.
 
-The reformatting stage of MET consists of the Gen-Vx-Mask, PB2NC, ASCII2NC, Pcp-Combine, MADIS2NC, MODIS regrid, WWMCA Regrid, and Ensemble-Stat tools. The PB2NC tool is used to create NetCDF files from input PrepBUFR files containing point observations. Likewise, the ASCII2NC tool is used to create NetCDF files from input ASCII point observations. Many types of data from the MADIS network can be formatted for use in MET by the MADIS2NC tool. MODIS and WWMCA files are regridded and formatted into NetCDF files by their respective tools. These NetCDF files are then used in the statistical analysis step. The Gen-Vx-Mask and Pcp-Combine tools are optional. The Gen-Vx-Mask tool will create a bitmapped masking area in a variety of ways. The output mask can then be used to efficiently limit verification to the interior of a user specified region. The Pcp-Combine tool can be used to add, subtract, or derive fields across multiple time steps. Often it is run to accumulate precipitation amounts into the time interval selected by the user - if a user would like to verify over a different time interval than is included in their forecast or observational dataset. The Ensemble-Stat tool will combine many forecasts into an ensemble mean or probability forecast. Additionally, if gridded or point observations are included, ensemble verification statistics are produced.
+The reformatting stage of MET consists of several tools which perform a variety of functions. The ASCII2NC, PB2NC, MADIS2NC, LIDAR2NC, and IODA2NC tools read a variety of point observation input file formats and, optionally, derive time summaries for each observing location. They all write to a common NetCDF point observation file format which can be read by the other MET tools. The Point2Grid tool reads that common NetCDF point observation file format and interpolates the point data onto a user-specified grid. The Regrid-Data-Plane, Shift-Data-Plane, MODIS-Regrid, and WWMCA-Regrid tools read a variety of gridded input file formats and interpolate user-requested input fields to a user-defined output grid. While the MET statistics tools can interpolate many input file formats in-memory and on-the-fly, manually regridding upstream is sometimes useful. The Pcp-Combine tool adds, subtracts, or derives fields across multiple time steps. It is often run to accumulate precipitation amounts into a user-specified time interval - if a user would like to verify over a different time interval than is included in their forecast or observational dataset. The Gen-Vx-Mask tool provides a variety of methods for creating bitmapped masking areas. Those masks can then be used to efficiently limit verification to the interior of a user-specified region in the downstream statistics tools. The Gen-Ens-Prod tool derives basic ensemble products (mean, spread, probabilities) from multiple gridded input ensemble members. The GSI tools reformat binary GSI diagnostic data to be read by the Stat-Analysis tool.
 
 .. _overview-figure:
 
@@ -52,7 +52,7 @@ Sometimes it may be useful to verify a forecast against gridded fields (e.g., St
 
 Users wishing to accumulate statistics over a time, height, or other series separately for each grid location should use the Series-Analysis tool. Series-Analysis can read any gridded matched pair data produced by the other MET tools and accumulate them, keeping each spatial location separate. Maps of these statistics can be useful for diagnosing spatial differences in forecast quality. 
 
-Ensemble-Stat is a hybrid tool that provides based post-processing capability of the ensemble members as well as computing measures of ensemble characteristics.  Basic post-processing capability includes computing the ensemble mean, min, max, standard deviation, and ensemble relative frequency or probability.  These fields can then be used in other MET tools for additional evaluation.  The ensemble characteristics include computation of rank and probability integral transform (PIT) histograms, the end-points for receiver operator curve (ROC) and reliability diagrams, and ranked probabilities scores (RPS) and the continuous version (CRPS).
+Ensemble-Stat is a hybrid tool that provides based post-processing capability of the ensemble members as well as computing measures of ensemble characteristics. Basic post-processing capability includes computing the ensemble mean, min, max, standard deviation, and ensemble relative frequency or probability. These fields can then be used in other MET tools for additional evaluation. Note however that the Gen-Ens-Prod tool also performs ensemble product generation, and this functionality will be removed from Ensemble-Stat in future versions. The ensemble characteristics include computation of rank and probability integral transform (PIT) histograms, the end-points for receiver operator curve (ROC) and reliability diagrams, and ranked probabilities scores (RPS) and the continuous version (CRPS).
 
 The MODE (Method for Object-based Diagnostic Evaluation) tool also uses gridded fields as observational datasets. However, unlike the Grid-Stat tool, which applies traditional forecast verification techniques, MODE applies the object-based spatial verification technique described in :ref:`Davis et al. (2006a,b) <Davis-2006>` and :ref:`Brown et al. (2007) <Brown-2007>`. This technique was developed in response to the "double penalty" problem in forecast verification. A forecast missed by even a small distance is effectively penalized twice by standard categorical verification scores: once for missing the event and a second time for producing a false alarm of the event elsewhere. As an alternative, MODE defines objects in both the forecast and observation fields. The objects in the forecast and observation fields are then matched and compared to one another. Applying this technique also provides diagnostic verification information that is difficult or even impossible to obtain using traditional verification measures. For example, the MODE tool can provide information about errors in location, size, and intensity.
 
@@ -90,9 +90,9 @@ We welcome comments and suggestions for improvements to MET, especially informat
 
 The MET package is a "living" set of tools. Our goal is to continually enhance it and add to its capabilities. Because our time, resources, and talents are limited, we welcome contributed code for future versions of MET. These contributions may represent new verification methodologies, new analysis tools, or new plotting functions. For more information on contributing code to MET, please create a post in the `METplus GitHub Discussions Forum <https://github.com/dtcenter/METplus/discussions>`_.
 
-Fortify
-=======
+Fortify and SonarQube
+=====================
 
-Requirements from various government agencies that use MET have resulted in our code being analyzed by Fortify, a proprietary static source code analyzer owned by HP Enterprise Security Products. Fortify analyzes source code to identify for security risks, memory leaks, uninitialized variables, and other such weaknesses and bad coding practices. Fortify categorizes any issues it finds as low priority, high priority, or critical, and reports these issues back to the developers for them to address. A development cycle is thus established, with Fortify analyzing code and reporting back to the developers, who then make changes in the source code to address these issues, and hand the new code off to Fortify again for re-analysis. The goal is to drive the counts of both high priority and critical issues down to zero.
+Requirements from various government agencies that use MET have resulted in our code being analyzed by both the Fortify and SonarQube static source code analysis tools. Fortify and SonarQube analyze source code to identify for security risks, memory leaks, uninitialized variables, and other such weaknesses and bad coding practices. They categorize issue as low priority, high priority, or critical, and report these issues back to the developers for them to address. The goal is to drive the counts of both high priority and critical issues down to zero.
 
 The MET developers are pleased to report that Fortify reports zero critical issues in the MET code. Users of the MET tools who work in high security environments can rest assured about the possibility of security risks when using MET, since the quality of the code has now been vetted by unbiased third-party experts. The MET developers continue using Fortify routinely to ensure that the critical counts remain at zero and to further reduce the counts for lower priority issues.