-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fearture/jjin3/allow for missing ob type b #415
base: develop
Are you sure you want to change the base?
Conversation
So far, this PR works for missing convectional, ozone, and microwave radiance data. It crashes swell-3dfgat_atmos run when any of AIRS, IASI, CRIS-FSR, and AVHRR input files is missing. |
@gmao-wgu It turns out that JEDI crashes with this correlated IR observational error inflation filter when there aren't observational input files:
UserError: Invalid prior filter requested I added a check of number of observations in /discover/nobackup/jjin3/jedi/jedi-work2/ufo/src/ufo/errors/ObsErrorCrossVarCov.cc.jj . However, that doesn't solve the issue. JEDI still returns an error of "Invalid prior filter". Any idea? |
@gmao-jjin3 I will try to replicate your case and proceed from there. |
@gmao-wgu It turns out there is an issue to handle IASI files in JEDI/ioda. The filter mentioned above is fine. My other tests go through with missing AIRS and CRIS-FSR input observational files. Tests only fail with missing IASI data. UFO filters are fine. |
It turns out I had a minor update in UFO before I traced down the above issue in IODA. Here is a UFO PR for the update: https://github.com/JCSDA-internal/ufo/pull/3455. |
@gmao-jjin3 and @rtodling, is this ready to merge? I see the relevant PR in JEDI is merged. |
@Dooruk No, the JEDI team is still looking into the issue that arises with missing IASI data. |
@gmao-jjin3, please notice that the actual error message in your excerpt above is: UserError: Invalid prior filter requested which means you are probably adding a filter in the wrong section (pre, prior, post, please see https://github.com/JCSDA-internal/oops/blob/d77217323bbbd02ea41049fe1bd339f24a64f177/src/oops/base/ObsFilters.h#L41-L58 for more info), while ObsErrorCrossVarCov: Obs error correlations not provided for 324 of 431 channels/variables in file: fv3-jedi/rcov/cris-fsr_108_jedi_rcov.nc4. To see which channels, turn on OOPS_TRACE
OOPS Ending 2024-09-03 12:25:59 (UTC-0400) is a warning message that the matrix you provided for CrIS-FSR doesn't have covariances for some channels (likely the ones you don't use). |
@fabiolrdiniz I noticed that before. However, I don't think it is the real reason. All the filters and their order are fine when the obs input file is available. |
Allow filter for missing satellite radiance files in the final SWELL configurations, following @rtodling PR #400. However, radiance observations with zero-sized input files are omitted in the final configurations because JEDI/IODA cannot read those zero-sized files. In addition, filters for missing conventional and ozone observational files are omitted since an action of "error" is now assigned for those missing observations.