Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update develop-ref after #2779 #2804

Merged
merged 67 commits into from
Jan 30, 2024
Merged

Update develop-ref after #2779 #2804

merged 67 commits into from
Jan 30, 2024

Conversation

github-actions[bot]
Copy link
Contributor

@github-actions github-actions bot commented Jan 30, 2024

Adds new MODE multivar unit tests and output:

dir1: /data/output/met_test_truth contains 1136 files
dir2: /data/output/met_test_output contains 1152 files

ERROR: folder /data/output/met_test_truth missing 16 files
    mode_multivar/mode_Fcst_Snow_LO_Obs_Precip_LO_210000L_20210201_210000V_000000A.ps 
    mode_multivar/mode_Fcst_Snow_LO_Obs_Precip_LO_210000L_20210201_210000V_000000A_cts.txt 
    mode_multivar/mode_Fcst_Snow_LO_Obs_Precip_LO_210000L_20210201_210000V_000000A_obj.nc 
    mode_multivar/mode_Fcst_Snow_LO_Obs_Precip_LO_210000L_20210201_210000V_000000A_obj.txt 
    mode_multivar/mode_Fcst_VIS_L0_Obs_VIS_L0_210000L_20210201_210000V_000000A.ps 
    mode_multivar/mode_Fcst_VIS_L0_Obs_VIS_L0_210000L_20210201_210000V_000000A_cts.txt 
    mode_multivar/mode_Fcst_VIS_L0_Obs_VIS_L0_210000L_20210201_210000V_000000A_obj.nc 
    mode_multivar/mode_Fcst_VIS_L0_Obs_VIS_L0_210000L_20210201_210000V_000000A_obj.txt 
    mode_multivar/mode_Fcst_VIS_L0_Obs_WIND_Z10_210000L_20210201_210000V_000000A.ps 
    mode_multivar/mode_Fcst_VIS_L0_Obs_WIND_Z10_210000L_20210201_210000V_000000A_cts.txt 
    mode_multivar/mode_Fcst_VIS_L0_Obs_WIND_Z10_210000L_20210201_210000V_000000A_obj.nc 
    mode_multivar/mode_Fcst_VIS_L0_Obs_WIND_Z10_210000L_20210201_210000V_000000A_obj.txt 
    mode_multivar/mode_Fcst_WIND_Z10_Obs_WIND_Z10_210000L_20210201_210000V_000000A.ps 
    mode_multivar/mode_Fcst_WIND_Z10_Obs_WIND_Z10_210000L_20210201_210000V_000000A_cts.txt 
    mode_multivar/mode_Fcst_WIND_Z10_Obs_WIND_Z10_210000L_20210201_210000V_000000A_obj.nc 
    mode_multivar/mode_Fcst_WIND_Z10_Obs_WIND_Z10_210000L_20210201_210000V_000000A_obj.txt 

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # #

COMPARING mode_multivar/mode_Fcst_Super_LO_Obs_Super_LO_300000L_20120410_180000V_060000A.ps
file1: /data/output/met_test_truth/mode_multivar/mode_Fcst_Super_LO_Obs_Super_LO_300000L_20120410_180000V_060000A.ps
file2: /data/output/met_test_output/mode_multivar/mode_Fcst_Super_LO_Obs_Super_LO_300000L_20120410_180000V_060000A.ps
ERROR: diff error:
1837,1840c1837,1840
< 81.6 230.0 m (*,*) show
< [156](https://github.com/dtcenter/MET/actions/runs/7703030125/job/20993498058?pr=2779#step:5:157).6 230.0 m (*,*) show
< 81.8 215.0 m (NA) show
< 156.8 215.0 m (NA) show
---
> 81.7 230.0 m (SFC) show
> 156.7 230.0 m (SFC) show
> 81.8 215.0 m (kg/m^2) show
> 156.8 215.0 m (kg/m^2) show


Created by @JohnHalleyGotway

MET Tools Test Account and others added 30 commits November 19, 2023 19:42
…ated when running bootstrap inside the dtcenter/met-base:v3.1 Docker image. Updates autoconf 2.69 output to 2.71.
Fixed incorrect path to obs error table
…g the default version which no longer exists in /usr/local
… ncdiff. I did test running comp_dir.sh with this change and confirmed that it now runs to completion.
…ent variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL
…trings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL
* Per #2776, delete stale prob_pair_info.h/.cc, wwmca_utils.h/.cc, and rmw_analysis_utils.h/.cc files that were not being compiled by their Makefiles.

* Per #2776, update Makefile to compile the test_read_rmw utility.

* Per #2776, add Makefile for the vx_python3_utils test utility directory.

* add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca

* only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies

* Per #2776, just whitespace and capitalization

---------

Co-authored-by: George McCabe <[email protected]>
* Per #2782, port over fixes from the bugfix_2782_main_v11.1_MASSDEN branch over to the bugfix branch for develop. Will also add new GRIB2 filtering options in this branch.

* Per #2782, add support for 4 new GRIB2 filtering options for GRIB2_aerosol_type, GRIB2_aerosol_interval_type, GRIB2_aerosol_size_lower, and GRIB2_aerosol_size_upper. These are useful in filtering the MASSDEN records in the RRFS smoke output files.

* Per #2782, fix aerosol_size_lower() and aersol_size_upper() inline definitions.

* Per #2782, add a unit test for GRIB2 table 4.48.

* Per #2782, switch from strict equality to using the is_eq() function when checking the GRIB2_aerosol_size_lower and upper values since they're doubles and not ints.
* Add support for ISMN soil moisture data

* Per #2701, update ascii2nc.cc with the ismn file option.

* Per #2701, update ascii2nc docs

* Per #2701, store the depth for precip as 0.

* Per #2701, parse the ISMN observation value from the correct column.

* Per #2701, every time I run ascii2nc to I see a log message for 'DEBUG 1: Number of NDBC skipped files due to no lookup 0'. This is printed by the NdbcHandler destructor. Update the logic to only print a debug level 3 log message when the number of missing locations is greater than 0, 'DEBUG 3: Skipped 5 NDBC files whose locations are not defined in 'MET_BASE/table_files/ndbc_stations.xml'. Set the MET_NDBC_STATIONS environment variable to override this file.'

* Per #2701, as instructed by @anewman89, store the average of the depth values rather than the maximum value.

* Per #2701, add a test of processing the ISMN data through ascii2nc.

* Per #2701, fix parsing of year and month to subtract 1900 and 1, respectively.

* Per #2701, set use_var_id to true for ISMN inputs

* Per #2701, even though we're mapping obs data to GRIB code names and units, we want to encode them with use_var_id = true and so we need to keep track of those var_id values.

* Per #2701, fix typo in the name of the ascii2nc netcdf output file.

---------

Co-authored-by: MET Tools Test Account <[email protected]>
Howard Soh and others added 28 commits January 18, 2024 21:35
* Per #2547, add the file_type = NETCDF_WRF configuration option.

* Per #2547, rename vx_data2d_nc_interp as vx_data2d_nc_wrf and vx_data2d_nccf as vx_data2d_nc_cf.

* Per #2547, rename nccf files as nc_cf for consistency throughout

* Per #2547, rename Pinterp classes as Wrf

* only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies

* add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca

* ignore directory automatically created by CLion

* rename pinterp variables and classes to Wrf

* more rename and fix typos

* per #2547, add support for reading WRF files by adding support for different X/Y/Z dimensions, read X/Y/Z dimension names when reading a field instead of storing them per file, track if data is staggered in X/Y/Z dimension (still need to add DataPlane function to destagger data), handle pressure field that also includes time dimension

* per #2547, add DataPlane function to handle staggering of grid

* per #2547, change file type returned from pinterp to wrf

* per #2547, call function to stagger DataPlane if necessary

* per #2547, keep store if variable is on pressure levels and only read pressure field if it is

* ignore cmake directory

* per #2547, support U_PL, U_ZL, U, and U<x> where <x> is an integer in is_u_wind and the same for is_v_wind

* change regex to match zero or more instances of a number after U or V instead of 1 or more to include the fields named U and V

* error and exit if var with unsupported subgrid dimension is processed

* clean up logging

* only destagger Z dimension for bottom_top_stag because other _stag Z dimensions don't need to be destaggered, clean up logic to read X/Y/Z dimension info

* Per #2547, instead of checking dimensions of all vars in file, check dimensions for the var that is being read. Only read pressure var if the z dimension matches the var to read. Set class variable DimNames instead of local gDimNames so dump function will work as expected

* per #2547, support reading variable that corresponds to dimension with different name as dimension and includes a time dimension, e.g. P_PL(Time, num_press_levels_stag)

* per #2547, added unit tests to process WRF out data with plot_data_plane and grid_stat

* merged develop and resolved conflicts

* rename lib from pinterp to wrf

* rename lib

* fixed incorrect merge changes

* fixed broken unit test format

* Per #2547, restore the mention of NAK.poly to Appendix B.

* Change model to WRF

Co-authored-by: John Halley Gotway <[email protected]>

* change obtype to WRF

Co-authored-by: John Halley Gotway <[email protected]>

* plot higher vertical level to get non-zero output

Co-authored-by: John Halley Gotway <[email protected]>

* indentation

Co-authored-by: John Halley Gotway <[email protected]>

---------

Co-authored-by: John Halley Gotway <[email protected]>
…rf branch into develop. Rerunning bootstrap on the project machine seneca produces 2 small differences that should also be incorporated into develop.
* updated first 3 sections

* updated spacing

* adding the Environment Variables to Run Script section

* adding in the remainder of the google doc changes

* trying to fix duplicate explicit target name issue

* fixing required_external_libraries_to_build_met links

* HDF4 testing double underscore

* adding double underscores for HDF5 and HDF-EOS2 and changing library back

* fixing HDF5 web link name

* Added formatting, made minor edits, removed manual build instructions sections

* Removed reference to sample-test-case which no longer exists

* Resolving errors

* Minor formatting change

* Updated apptainer instructions, new version references

* Formatting updates and removal of first person language

* fixed code blocks

* Resolving error

* Removed first person references

* Removed reference to met_directory_structure

* updated bolding, removed end section

* Changing X.Y.X references to X.Y.Z

* updating thru Using the compile_MET_all.sh Script

* fixing spacing for ATLAS web address

* Fixed broken internal reference to installation due to typo

* adding version numbers

* fixing Recommended-Components MET

* 3.4.2 section updates

* fixing spacing

* adding a period and more dropdown menus

* fixing spacing

* loose ends

* removing space

* removing bold

* External Library handling in compile_MET_all.sh section updates

* installation directory view

* installation directory after

* attempting to add figures via the web

updating Executing the compile_MET_all.sh script section

* fixing bolds, italics etc.

* updating last half of doc

* loose ends

* loose ends

* adding more dropdown menus

* fixing dropdown menu and testing automatic version

* testing automatic version

* confirmed automatic version cannot be added to italics or code blocks

* another round of updates

* fixing indents

* creating a table

* fixing table

* fixing table 2

* more loose end changes

* more changes

* hopefully final updates

* trying a fancy new csv table

* troubleshooting csv table

* correcting table name

* adding a grid table

* fixing grid table

* removing csv table

* Per #2588, update to the environment variables table.

* Delete docs/Users_Guide/environment_variables_table.csv

* Per #2785, add new example installation files

* updating small changes in the first half of the documentation

* new dropdown menu and fixing some links

* trying to fix line spacing in dropdown

* take 2 fixing spacing

* take 3 fix spacing

* env var order

* removing "s"

* small changes

* un capitalizing S

* A few minor changes

---------

Co-authored-by: j-opatz <[email protected]>
Co-authored-by: Julie Prestopnik <[email protected]>
Co-authored-by: j-opatz <[email protected]>
Co-authored-by: John Halley Gotway <[email protected]>
…gle_point_obs

Bugfix #2755 header counter develop
* Per #2796, update actions/checkout@v3 to actions/checkout@v4.

* Per #2796, update to actions/upload-artifact@v4 and actions/setup-python@v5.

* Per #2796, update to download-artifact@v4 and actions/github-script@v7
* Per #2796, update actions/checkout@v3 to actions/checkout@v4.

* Per #2796, update to actions/upload-artifact@v4 and actions/setup-python@v5.

* Per #2796, update to download-artifact@v4 and actions/github-script@v7

* Per #2796, update testing workflow for update-artifact v4 version which requires unique artifact names. ci-run-unit

* Per #2796, merge the logs into a single artifact. ci-run-unit

* Per #2796, renaming rc_leads and rc as 1c and 2c, respectively for consistency purposes. Note that this commit should result in just a logs_compile artifact being created... and not doing the log merging step.

* Per #2796, update the compilation workflow to make the log artifact names unique.
* Per #2801, add new time_offset_warning config option.

* Per #2810, update MetConfig class to parse and check the time_offset_warning config option.

* Per #2801, update Grid-Stat, MODE, PCP-Combine, Series-Analysis, Wavelet-Stat, and the WWMCA tool to call MetConfig::time_offset_warning() to decide whether to print a Warning or Debug(3) log message about initialization or valid time differences.

* Per #2801, no change to content. Just removing unneeded whitespace.

* Per #2801, remove extra empty line foromw m warning message.

* Per #2801, remove extra empty line from MODE warning messages.

---------

Co-authored-by: MET Tools Test Account <[email protected]>
* Working mvmode with percentile thresholds and grid_type fixes, plus more unit tests

* separated multivar and traditional mode frontend classes to simplify the mode executive class design

* documentation changes to go with changes to percentiles and data_type

* Cleanup and bugfix to doc

* Bug fix to doc

* Another bugfix attempt

* synching up development.seneca with what is in the develop branch

* added clone() method to all the vx_data2d var_info classes

* Fix multivar_name/level definitions in UG

* bug fix regarding accessing the config to create a verification grid

* Update docs/Users_Guide/mode.rst

* Update docs/Users_Guide/mode.rst

* Per #2745, commit changes to Makefile.am/.in files after running bootstrap on this feature branch on seneca.

* Per #2745, no real code changes. Just whitespace.

---------

Co-authored-by: MET Tools Test Account <[email protected]>
Co-authored-by: Tracy Hertneky <[email protected]>
Co-authored-by: John Halley Gotway <[email protected]>
@JohnHalleyGotway JohnHalleyGotway added this to the MET 12.0.0 milestone Jan 30, 2024
@JohnHalleyGotway JohnHalleyGotway merged commit 1141a18 into develop-ref Jan 30, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🏁 Done
Development

Successfully merging this pull request may close these issues.

8 participants