[BUG/ISSUE] Error in resuspension for wet deposition #501
Comments
yantosca
changed the title
|
Hi @lfreese, thanks for writing. There are a couple of things going on here. From the stderr output we have this traceback: which seems to indicate a floating-point invalid (perhaps underflow) in reading data from netCDF (the m_netcdf_io_read.F90 is the module that reads data from disk. You might want to check your emissions files to make sure that there isn't a stray NaN in the data. Or if there isn't a NaN in the data, make sure that the data values are all representable by REAL*4 precision (i.e. nothing smaller than ~1e-38). The error in wetdep may be a side-effect of the netCDF error. Species #57 is HI, one of the halogens, and it seems it is going to a very small negative value (-5.100540729979787E-047). |
yantosca
changed the title
|
Hi Bob, Thanks for the help! I've checked the emission files, especially those that I modified for these runs from the normal NEI, and am not finding any NaNs in the data. Is it possible that it is an issue with the meteorology, or that I should find a way to work around such a small negative number? Lyssa |
|
Hi @lfreese. I wonder if the initial conditions may be causing this. If you use the GEOS-Chem restart file that ships with the run directory, that is quite old (we are replacing these in 13.0.0). It probably does not have accurate initial conditions for e.g. the isoprene chemistry species. Any species that isn't found in the restart file gets set to a very small number (e.g. 1e-30), so that can increase the chance of a value going negative. You can try to take the restart file for your start month from the last 1-yr benchmark (download from: ftp.as.harvard.edu/gcgrid/geos-chem/1yr_benchmarks/GC_12/12.9.0/FullChem/GCC/restarts.tar.gz) and use that for your initial conditions. That might help. |
|
Hi @lfreese, just checking if you are still encountering problems with this issue. If not let us know and we can close this issue out. |
|
Sorry--our server was down for the week, so I just got to trouble shooting again. It does look like it is an issue with the emissions files as when I try running it with the 1-year benchmark and it still stops with the error of reading in NaNs from emissions files. I'm looking into the files that should be read in after my inventory (which looks like its the last one being read into HEMCO) to see if that is where the error occurs, or if my emissions inventory does have an issue. |
|
Thanks for the update. One thing I've found is that if you use Python xarray to create files, then it seems to fill missing values with NaN's. I wonder if that is the cause of the error. You can also use xarray to strip out the NaNs and to convert those to zeros if that helps. |
|
Also you might want to try using the HEMCO standalone to see if you can read the data properly. That might let you narrow down where the error is in the data files. |
|
@eamarais I haven't solved it yet. Initially, this was a confluence of two errors--I had HEMCO files I made myself that looked to be pulling in some NaNs on this date, but once I fixed those the wetdep error continues to appear. I've re-run things using the restart file that was created for 20151230 on 20151231, and it works (so it's not the issue of a restart), so it must be an issue with a HEMCO emissions file for that specific day. I'm going to plug through looking at the different emissions inputs today and will update you when I find the source of NaNs. I didn't do my run in March 2015 (I started in June), so it would make sense that I didn't hit this error on the date you did. |
|
Short update--the only input files that seem to have a NaN are: I'm testing runs with the NaNs replaced in each of these files to see if this is the source. |
|
I've performed tests turning off different emission inventories. When I turn off MEGAN I move past the point where the error occurs. I'm using online MEGAN emissions. |
|
@lfreese I think it's a meteorology issue, as you suspected earlier in the thread. I haven't identified the cause, but I've run a few tests now. When I run the model in 2015 using 2014 met fields I don't get the wet deposition error. |
|
The error in MEGAN might be this. I recently fixed this for 13.0.0: The problem in that where the FLUXACET array is allocated in the MEGAN extension, we had a typo right below that that assigned FLUXACETmo to zero. The problem being that at this point, FLUXACETmo was not allocated (it gets allocated below). If you have that in your 12.9.3 code, try making the same fix there and then recompiling. NOTE: In 13.0.0 and later, HEMCO will be a completely separate repository than GEOS-Chem. |
|
Thanks @yantosca. The version of the code I'm using doesn't include FLUXACET, just the three components: FLUXACETmo, FLUXACETmb, and FLUXACETbg, so I don't think this is the issue. |
|
I'm having similar occurrences--I used my 20151229 restart file to start a run on 20160101 to see if it ran forward fine, and it does, while it breaks if I run it through 20151230. I'm wondering if it is the gradients of the PRECIP fields as none of mine have NaNs either, but I wonder if a sharp gradient in combination with already very small halogen levels may cause negative values to arise. I'm looking into the gradients for the different precip fields to see if this could be the cause and will report back later. I'm looking at the total precip for this, but if you have suggestions on a better one to look at I'll do that as well. |
|
I get error messages for all the precip fields, so looking at total precip to diagnose the error should be sufficient. |
|
Quick update--the precip fields do not have any really large gradients and I'm unclear where the error would be coming from for them. I am now looking into the linearized stratospheric chem as that is looking like a potential source of the error which is then propagating into the wetdep. |
|
Thanks @lfreese for the output. It seems like there are weird values in the restart file: those might be either NaN or underflow values. You might want to scan the restart file with Python and see what those values really are. If they are very low you can reset them to zero. |
|
@yantosca thanks for the suggestion! I've fixed those and it is still giving a negative value at level 41 in the atmosphere for HI. My current understanding of the track of the errors is that:
Is there a way to in the SAFETY to just set anything that is really small and negative to a zero when it finds a negative value? |
|
Hi @lfreese. Yes you could do this within SAFETY if you wish. The specific modifications you would need is to change the read-only attribute of the Spc vector in the argument list, i.e. change: REAL(fp), INTENT(IN) :: Spc(State_Grid%NZ)to REAL(fp), INTENT(INOUT) :: Spc(State_Grid%NZ)and then the easiest way to change negative values would be to do something like this: WHERE ( Spc < 0.0_fp )
Spc = 0.0_fp
ENDWHERE |
|
Hi @lfreese. Just checking in if there you have found a solution for this. If so, we can close out this issue. |
|
I did (although it is still somewhat a hack of setting this to zero, so if someone comes across this issue with these dates of meteo again, it might be worth revisiting). I ended up editing do_complete_reevap as that was the root of the error: |
Similar to the bug reported in #259 I am coming across issues with the wet dep on 12/30/15. In my scenario I am running 12.9.3, am using all of the regular input files and have not modified the code, other than to add emissions that will run in 2016 to replace the usual NEI emissions. This shouldn't be the problem, as those emissions files have worked in 2016 runs, and this error occurs in 2015. I am running a 6 month spinup, and this error is occuring right at the end of it.
Error messages
Required information:
Your GEOS-Chem version and runtime environment:
Input and log files to attach
I've run it with the debug, bounds, traceback and fpex on so that I could debug it, but it isn't clear to me what is causing the sudden issue on this date after almost 6 months of successful runs.
my_runerr.txt
my_runout.txt
The text was updated successfully, but these errors were encountered: