Update regrid.pl for GOCART2G #228
Comments
|
CC @weiyuan-jiang, @tclune, @aoloso in re the regrid refactoring effort |
|
Adding the ability for interp_restarts.x to handle these new cases an ungridded dimension + level is straightforward from a scientific perspective (just treat each index of the ungridded dimension as a 3-D variable in it's own right and do the usual horizontal and vertical regridding. However, the code has gotten to the point that to add new capabilities like this calls for a refactoring. At least splitting out the binary and netcdf into separate codes so I can focus on adding these capabilities to the netcdf restarts without breaking the binary path. Splitting this apart and adding new capabilities is a non-trivial exercise so lead time and dedicated time to do this will be necessary. |
|
@mmanyin @pcolarco It had been a long time since I looked at the code; I split interp_restarts.x into a binary and NetCDF code. That was fairly straightforward. Adding support for these 4-D variables in the NetCDF version of the program is actually more straightforward than I thought. Although this will mean a multiple repo PR for regrid.pl and the underlying regridding ... |
|
@mmanyin @pcolarco @gmao-jstassi @mathomp4 The issue now is regrid.pl itself. It needs to know about the restart names but I think we are getting to the point where it needs some more flexibility. Since each species in gocart and each instance can have it's own restart like |
|
Does this only affect the input name ? Is there any special option? For the future, the python regrider only cares about the names that are listed in the yaml file. |
|
Sounds like good progress. Will it eventually be possible to regrid from the old GOCART 1G format to the GOCART 2G? |
|
I would second @mmanyin. I think we would want to be able to regrid to GOCART2G from MERRA-2 for instance. |
@mmanyin Can you elaborate what that would entail or what that even means? Is it just a matter of splitting an old gocart 1g restart into separate restarts? That's beyond the scope of what the underlying regridding code (interp_restarts.x) would handle, it just regrids what is there already. It would have to be some other script but someone who understands this would have to write it or give me a precise recipe for what that operation would entail. |
As I said, I will not support this in interp_restarts.x, that's outside the scope of that program. If is as simple as spliting the fields into separate restarts that is another programs job, it would be a trivial python script. As far as MERRA2 This is a can of worms. Regridding directly from MERRA2 to GOCART2G is complicated by the fast that MERRA2 was binary, so there is no metadata in the file. I painstakingly figured out the order of the fields in the MERRA2 binary restarts a long time ago and wrote a converter here to convert them from binary to NetCDF using descriptor files that document the variable order in each file: https://github.com/bena-nasa/GEOS5_restart_converter If you want to do something with MERRA2 in this hypothetical GOCART1G to GOCART2G operation, you would need to use my tool to convert this to NetCDF then, if there is a solution to go from GOCART1G to GOCART2G with the NetCDF file. |
|
I understand. A boy can dream. |
|
@bena-nasa I have been impressed in the past when needing to convert legacy restarts to a more recent version, that regrid.pl could identify what was missing and provide at least a bare bones set of restarts. Without really knowing what the limitations are, I was posing the general question -- can the program generate a set of G2G restarts that approximates an older G1G set? Sounds like this is not the tool for doing it. |
No, regrid.pl just calls other program that regrid the restarts that are there using the boundary conditions it thinks are appropriate based on your answers to the questions; no more, no less. If you have a gocart_internal_rst in, you get one out. @mmanyin @pcolarco This does bring up point, what are all the "base" component names it need to be aware of. I'm going to try to implement a wild card feature in regrid.pl |
|
@bena-nasa Your list looks complete for GOCART2G, although note it is "cabr," "cabc," and "caoc". You might anticipate an eventual refactoring of the remainder of the legacy GOCART which would split what is still left in gocart_interal_rst into subsequent things like co, co2, ch4, ... _interal_rst. |
@pcolarco Oh, did misread that? I thought, cabr, cabc, caoc were separate instances of one species, but now I'm reading again, that is just brown carbon, black carbon, and organic carbon. So per species there is only one restart no matter how many instances? If so then I can just hard code the names and I was making a problem out of nothing. |
|
@bena-nasa Hmm... Each instance has its own restart. But default we are running three carbonaceous instances: brown, black, and organic carbon. We need to be able to regrid each such instance. Some guidance then for how to handle multiple instances for later (i.e., so far not tried out) cases will be helpful. Does that make sense? |
|
@pcolarco @jstassi I was only asking about the instances because currently the way regrid.pl works is that is has hard coded restart names it looks for. So if the name is not in the list, it won't regrid it. So this could be a problem if someone runs a new gocart case with multiple instances for example and wants to regrid those but the script is unaware.
It sounds like for now the PR I've made handles the current uses but still needs another extension. Any thoughts on which method sounds better as an end user? |
|
@bena-nasa Hey, Ben, is this functionality available now in some more recent model version for me to try out? Thanks |
|
@pcolarco I think it should be in GEOSgcm v10.21.0 for sure. |
Per @pcolarco and @mmanyin in email, there is a desire to regrid the new GOCART2G restarts. Looking at a set of them, the (possible) new files seem to be:
achem_internalcabc_internalcabr_internalcaoc_internaldu_internalhemco_internalni_internalss_internalsu_internalNow, normally we could just add these to The List™ that's in
regrid.plbut it's not that simple.I looked at these restarts with @bena-nasa and we noticed that:
hemco_internalis a 2D restart (no levels)du_internalandss_internalare 4D restarts (ungridded dims)The main issue is the underlying regridding code was not set up for files like these.
So, I suppose the first question for @pcolarco or @amdasilva or @christophkeller is: Do we need to worry about
hemco_internal? If not, we just just always bootstrap it and focus on the 4d restarts?The text was updated successfully, but these errors were encountered: