FATES LUH2 data curation tool #1032
Conversation
This incorporates an update to handle a pandas deprication issue when using cftime objects. See pydata/xarray#5359
Having too many of the dependencies in the yml file was causing issue with the dependency order and limiting xesmf to an older version.
This is necessary due to a ncd_io read requirement set in dynsubgrid.
I was able to successfully run the luh2.sh script on cheyenne using a custom conda environment with @slevis-lmwg I think this is good to review now. I'm assuming we'll need to work out what update to the "standard" conda environment will need to be on the ctsm5.2 development branch. |
There was a problem hiding this comment.
@glemieux feels this PR is ready to merge, after some final clean-up. Thank you @glemieux for going over the PR with me.
I am approving with a few minor comments to be addressed:
- Most important is the addition of comments + README for users unfamiliar with the tool
- Clean-up of obsolete commented out code
- One or two other minor things
Otherwise, as written the tool is easy to follow, and I do not feel expert enough in Python to recommend any type of refactor.
Adding 'default' and 'required' option as necessary to the argparse. This helps remove some of the luh2mod code that was previously handling default options. This also adds code to handle non-historical luh2 data file input arguments. Prior to this, it was assumed that only historical data files starting at 0850-01-01 were to be used. Now this can handle luh2 scenario files. It also adds time bounds error handling.
I didn't make this very clear in this comment, but I drafted a pull request on the ctsm side to capture the custom conda environment: ESCOMP/CTSM#2075 Per an google chat discussion with Ryan, I think we should address #1033 in the near future with another PR to make sure we can guarantee this and other fates python tools work for a particular set of package versions. |
…porting Land use change version 1 The patch-level land use labels are extended from their prior values of only primary or secondary lands to also include pasture, rangeland, and crops. Land use change in FATES is treated as a disturbance rate (in area/time units) from existing patches with a given land use label to new patches with the new land use label. Thus a fourtg distinct type of disturbances is added to the existing treefall, fire, and logging types. The driving data for this run mode is from the Land Use Harmonization 2 state and transition datasets. The workflow tool for processing this input was previously developed with pull request NGEET#1032.
Land use change version 1 The patch-level land use labels are extended from their prior values of only primary or secondary lands to also include pasture, rangeland, and crops. Land use change in FATES is treated as a disturbance rate (in area/time units) from existing patches with a given land use label to new patches with the new land use label. Thus a fourtg distinct type of disturbances is added to the existing treefall, fire, and logging types. The driving data for this run mode is from the Land Use Harmonization 2 state and transition datasets. The workflow tool for processing this input was previously developed with pull request NGEET#1032. # Conflicts: # biogeochem/FatesPatchMod.F90 # main/EDInitMod.F90 # main/EDTypesMod.F90
Land use change version 1 The patch-level land use labels are extended from their prior values of only primary or secondary lands to also include pasture, rangeland, and crops. Land use change in FATES is treated as a disturbance rate (in area/time units) from existing patches with a given land use label to new patches with the new land use label. Thus a fourtg distinct type of disturbances is added to the existing treefall, fire, and logging types. The driving data for this run mode is from the Land Use Harmonization 2 state and transition datasets. The workflow tool for processing this input was previously developed with pull request NGEET#1032. # Conflicts: # biogeochem/FatesPatchMod.F90 # main/EDInitMod.F90 # main/EDTypesMod.F90
Description:
This pull request adds the initial version of the fates luh2 data curation pipeline. The tool consists of four parts: 1) a module, luh2mod.py, that condenses a majority of the luh2 updates into a set of functions, 2) a python script, luh2.py, that calls the module functions and takes a set of command line arguments, 3) a bash script, luh2.sh, that sets up the expected arguments and calls the python script and 4) a conda environment
ymlfile to provide the user with the necessary dependencies to run the tool.This partially addresses #450. The script was heavily adapted from work done by @ckoven.
Raw LUH2 data from https://luh.umd.edu/ is available on cheyenne in the input data directory under
/lnd/clm2/rawdata/LUH2.Collaborators:
@ckoven @adrifoster
Expectation of Answer Changes:
No changes as this only impacts tooling
Checklist:
Test Results:
Testing not necessary
CTSM (or) E3SM (specify which) test hash-tag:
CTSM (or) E3SM (specify which) baseline hash-tag:
FATES baseline hash-tag:
Test Output: