GridStat WARNING when FCST and OBS data files record slightly different valid times

[MarcelCaron-NOAA]

Hi:

[Please move this discussion to the appropriate channel if needed]

The issue:

I am running a job that produces grid-to-grid Composite Reflectivity statistics, and while the job runs successfully, it consistently puts out a "WARNING" log indicating that the valid times of the forecast file and obs file do not exactly match (e.g.):

DEBUG 1: Forecast File: /lfs/h1/ops/prod/com/hiresw/v8.1/hiresw.20240123/hiresw.t12z.arw_5km.f03.conus.grib2
DEBUG 1: Observation File: /lfs/h2/emc/vpppg/noscrub/emc.vpppg/evs/v1.0/prep/cam/mrms.20240123/conus/MergedReflectivityQCComposite_00.50_20240123-150000.G227.nc
WARNING:
WARNING: process_scores() -> Forecast and observation valid times do not match 20240123_150000 != 20240123_150038 for REFC/L0 versus MergedReflectivityQCComposite/Z500.
WARNING:

All such WARNINGs I could find indicate a difference of only +60 seconds or less.

Expected Behavior:

I'd like to signal to METplus that such a difference (~1m) in valid times is permissible. Is it possible to mute these WARNINGs somehow?

My attempts to fix:

I tried setting "OBS_GRID_STAT_FILE_WINDOW_END = 60" but I realized that this configuration corresponds with the time used in the file name and not the valid time stored in the file's metadata, so this change did not mute the warning.

Other info:

sample_conf.txt
sample_log.txt

I've attached a sample conf file and corresponding log file for context. Please let me know if this discussion requires more data/information and I'll provide it.

[JohnHalleyGotway]

Hi @MarcelCaron-NOAA, I see you have a question about the writing of warning messages in the MET tools when the forecast and observation valid timestamps differ. We discussed this during our internal MET development meeting yesterday afternoon.

I realize that we need to be very judicious about when MET writes WARNING messages when being run operationally on WCOSS2. We want to only write them when there is a significant problem that requires attention.

The easiest solution would be simply changing the valid time mismatch warning messages to debug messages. But I do think it's worthwhile to maintain these checks and warn users about "large differences" to avoid big mistakes. While those mistakes are unlikely in the context of the highly controlled EVS runs, they're much more likely when running MET in research settings.

I propose that we add a new configuration option to define a maximum allowable time difference and only trigger a warning when that time difference is exceeded. I'll plan to default that difference to 0 to maintain the existing behavior. But increasing it would silence the warning in this situation... or setting it to a bad data value would silence all such warnings.

Is that the sort of change you're looking for? If so, I'll write up a GitHub issue to describe this change.

[MarcelCaron-NOAA]

Hi @JohnHalleyGotway:

Thanks for the help with this.

Is that the sort of change you're looking for? If so, I'll write up a GitHub issue to describe this change.

Yes, adding a configuration option to silence warnings based on a max allowable time difference is exactly what I was looking for.

The easiest solution would be simply changing the valid time mismatch warning messages to debug messages. But I do think it's worthwhile to maintain these checks and warn users about "large differences" to avoid big mistakes. While those mistakes are unlikely in the context of the highly controlled EVS runs, they're much more likely when running MET in research settings.

Agreed. I found these helpful as warning messages to avoid mistakes during development, so I agree that a config option to silence them is preferable to changing them to debug messages. It would be helpful to distinguish between expected and unexpected time differences in operations, the unexpected differences to be signaled by the warning messages.

I’ll mark your response as an answer to this question and will follow the Github issue.

Best,
-Marcel

[JohnHalleyGotway]

Thanks @MarcelCaron-NOAA, FYI, here's the MET issue: dtcenter/MET#2801