-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update API for CaseClass #30
Commits on Sep 25, 2020
-
This first commit is done in anticipation of supporting time series files in addition to history files, and also using intake-esm catalogs behind the scenes. 1. start_date & end_date are not part of the class constructor -- basically all the __init__ routine should do is set up a catalog -- _find_hist_files() also doesn't rely on these variables either; this routine is essentially building the catalog, and we don't restrict the files listed until we are ready to call open_mfdataset() 2. _open_history_files() -> gen_dataset(); this routine is where start_date and end_date are specified, and it returns a dataset rather than updating a class member variable. Logic to apply start_date and end_date at this stage is much simpler than doing it in _find_hist_files() (or I figured out a much easier way to do it). This routine also now expects a list of variable names to include in the dataset. 3. compare_fields_at_lat_lon expects array of DataArrays, not array of cases 4. Added get_varnames_from_metadata_list() to utils/utils.py because I need this function to generate the list of variable names for gen_dataset() in several notebooks. I also created a script that submits jobs to the slurm queue to re-run notebooks, though it does not work for the trend_maps or plot_suite_00[34] notebooks (I think because of the reliance on dask and NCAR_jobqueue).
Configuration menu - View commit details
-
Copy full SHA for 361cd79 - Browse repository at this point
Copy the full SHA 361cd79View commit details
Commits on Sep 28, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 390ecc2 - Browse repository at this point
Copy the full SHA 390ecc2View commit details
Commits on Oct 5, 2020
-
Configuration menu - View commit details
-
Copy full SHA for 5309e40 - Browse repository at this point
Copy the full SHA 5309e40View commit details
Commits on Oct 6, 2020
-
Add ability to read data from time series
gen_dataset() first reads time series from campaign, then reads any additional data from history files (from archive or run directory). There's an API change where, instead of expecting start_date and end_date (strings formatted as 'YYYY-MM'), the function wants integers start_year and end_year. Figuring out better default values will be necessary to extend this to other CESM runs but it's a good start. I split plot_suite_004.ipynb into 4-year segments, as it was choking on 8 years of output. I also added plot_suite_map notebooks for 004 for years 0006 and 0007 (we're one month shy of 0008).
Configuration menu - View commit details
-
Copy full SHA for 4c43880 - Browse repository at this point
Copy the full SHA 4c43880View commit details -
I had left the "import yaml" statement in from when I toyed with the idea of passing the metadata yaml file to __init__
Configuration menu - View commit details
-
Copy full SHA for 3ee0d33 - Browse repository at this point
Copy the full SHA 3ee0d33View commit details
Commits on Oct 7, 2020
-
Concatenating time series dataset with history file dataset now works as expected
Configuration menu - View commit details
-
Copy full SHA for b3d38bb - Browse repository at this point
Copy the full SHA b3d38bbView commit details
Commits on Oct 8, 2020
-
clean up variable names and logic in CaseClass
Also reran plot_suite notebooks in an environment that will match previous master (note that this changes plots in Sanity Check). Next commit will be updated trend_maps, and the plan is to squash-merge this PR to remove the commits that changed the plots and then changed them back.
Configuration menu - View commit details
-
Copy full SHA for eed1035 - Browse repository at this point
Copy the full SHA eed1035View commit details -
Re-ran in an environment that matches the previous master to minimize diffs in the PR.
Configuration menu - View commit details
-
Copy full SHA for ebcdf70 - Browse repository at this point
Copy the full SHA ebcdf70View commit details -
I didn't realize this notebook hadn't been run in previous commits
Configuration menu - View commit details
-
Copy full SHA for f9657d8 - Browse repository at this point
Copy the full SHA f9657d8View commit details -
Reran plot_suite_maps notebooks
Some of these were plotting the wrong year; I refactored all of them to be more clear about what is being plotted.
Configuration menu - View commit details
-
Copy full SHA for 3ae8225 - Browse repository at this point
Copy the full SHA 3ae8225View commit details
Commits on Oct 9, 2020
-
New: compare time series to history files
For now the comparisons are done based on variables in diag_metadata.yaml. Required a flag to suppress some CaseClass output.
Configuration menu - View commit details
-
Copy full SHA for cf9c15a - Browse repository at this point
Copy the full SHA cf9c15aView commit details -
Re-run notebooks with xarray 0.16.1
All plot_suite_maps now match master
Configuration menu - View commit details
-
Copy full SHA for 53c1f9f - Browse repository at this point
Copy the full SHA 53c1f9fView commit details -
Code clean-up following review
* run_notebook.sh launches a single notebook via slurm, and run_all.sh is now run_all.py (python-script) that calls run_notebook.sh * _find_timeseries_files() now includes pop.h.nyear1 files * gen_dataset(): 1. no longer assume time_bound is the name of the variable (look at bounds property of ds["time"] 2. if bounds are not decoded, still update start_year before looking for history files (via decoding time, which may not be best solution) * use list.extend() instead of += operator
Configuration menu - View commit details
-
Copy full SHA for 17d34a3 - Browse repository at this point
Copy the full SHA 17d34a3View commit details -
Configuration menu - View commit details
-
Copy full SHA for c013454 - Browse repository at this point
Copy the full SHA c013454View commit details -
Configuration menu - View commit details
-
Copy full SHA for e2c58b7 - Browse repository at this point
Copy the full SHA e2c58b7View commit details