-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Water demand #226
Water demand #226
Changes from 15 commits
8358da0
a396d35
7e4bbcc
e7c536e
74806a0
d384e3d
55e09f3
aa5a113
549a28f
2533bfb
bdf17b6
a5fc679
37f13b4
353bcf8
04d8ee4
e4c0305
03d89fb
b578d62
c908394
6ea8279
82adf41
1e54efd
695d378
249812b
6cd825e
829aaf2
c253e06
ec3380f
51556cb
bee017b
3644316
1c97eaf
77ed235
575d333
9472ab6
3ead783
1d7059e
a175dad
98b1fd4
f55c5f6
7d0b659
0d70610
96a1306
524d9b3
c5c33cc
69cf45b
800e997
51da860
20072fa
be5d78a
5642d08
7a8e76c
762f0fd
89f65a7
dcd8d31
f895ae3
7590e3c
49a175f
f121e43
bf3b7c8
8602bea
5241ce6
c8edb34
f1598d0
a5c6901
aa77c25
d667974
f371ed5
3cf4cc4
d0552a5
9708f40
aa0509f
0d36ce6
e09953e
29222e0
9f624ca
57145a7
b607c98
04acdd0
0d1cb97
4d4e737
221e20f
f06a89e
6be61e6
fc9d7df
40a723d
2c65702
7720625
021f0f5
78a602e
a6182f8
527e2f1
4f1ac6a
94498e9
a9c4283
809db48
59e6bed
02060ef
cb9b74e
8c3c84d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -5,6 +5,7 @@ | |||||
import glob | ||||||
import logging | ||||||
import os | ||||||
from itertools import product | ||||||
from os.path import basename, dirname, isdir, isfile, join | ||||||
from pathlib import Path | ||||||
from typing import List, Optional, Union | ||||||
|
@@ -65,6 +66,9 @@ class WflowModel(GridModel): | |||||
"glacareas": "wflow_glacierareas", | ||||||
"glacfracs": "wflow_glacierfrac", | ||||||
"glacstore": "wflow_glacierstore", | ||||||
"paddy_area": "paddy_irrigation_areas", | ||||||
"nonpaddy_area": "nonpaddy_irrigation_areas", | ||||||
"crop_factor": "crop_factor", | ||||||
} | ||||||
_FOLDERS = [ | ||||||
"staticgeoms", | ||||||
|
@@ -2494,6 +2498,292 @@ def setup_rootzoneclim( | |||||
# update config | ||||||
self.set_config("input.vertical.rootingdepth", update_toml_rootingdepth) | ||||||
|
||||||
def setup_allocation( | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
To match better wflow names. Makes it also clearer for me that you are only preparing areas here not demand or actual allocation data. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes. |
||||||
self, | ||||||
min_area: float | int = 0, | ||||||
admin_bounds_fn: str = "gadm", | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We are cleaning this up still for wflow but we do not want to assume a default dataset value anymore, they are just mandatory arguments :) the example from artifact_data can be mentioned in the examples. Which might be good to add here! (update water demands example to showcase the new methods) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed! |
||||||
admin_level: int = None, | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is too specific to gadm and people should be able to use their own geometry. I think in the data catalog, it's actually gadm_level1 that you can call so just use this in admin_bounds_fn (in testing / examples). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fair enough. |
||||||
): | ||||||
"""_summary_. | ||||||
|
||||||
_extended_summary_ | ||||||
|
||||||
Parameters | ||||||
---------- | ||||||
min_area : float | int | ||||||
_description_ | ||||||
admin_bounds_fn : str, optional | ||||||
_description_, by default "gadm" | ||||||
admin_level : int, optional | ||||||
_description_, by default 0 | ||||||
""" | ||||||
self.logger.info("Preparing water demand allocation map.") | ||||||
# Will be fixes but for know this is done like this | ||||||
# TODO fix in the future | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this comment still needed or can be removed? |
||||||
admin_bounds = None | ||||||
hboisgon marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
if admin_bounds_fn is not None: | ||||||
if admin_level is not None: | ||||||
admin_bounds_fn += f"_level{admin_level}" | ||||||
admin_bounds = self.data_catalog.get_geodataframe( | ||||||
admin_bounds_fn, | ||||||
geom=self.region, | ||||||
) | ||||||
# Add this identifier for usage in the workflow | ||||||
admin_bounds["admin_id"] = range(len(admin_bounds)) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can be moved to the workflow function then to keep wflow.py clean There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed |
||||||
|
||||||
# Create the allocation grid | ||||||
alloc = workflows.demand.allocate( | ||||||
ds_like=self.grid, | ||||||
min_area=min_area, | ||||||
admin_bounds=admin_bounds, | ||||||
basins=self.geoms["basins"], | ||||||
hboisgon marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
rivers=self.geoms["rivers"], | ||||||
) | ||||||
self.set_grid(alloc) | ||||||
|
||||||
# Update the settings toml | ||||||
self.set_config("input.vertical.waterallocation.areas", "Allocation_id") | ||||||
|
||||||
def setup_non_irigation( | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Almost everywhere irrigation with two rr not irigation There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Everything should have 'rr' now |
||||||
self, | ||||||
non_irigation_fn: str = "pcr_globwb", | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Either None if optional or required argument
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed. |
||||||
non_irigation_vars: list = ["dom", "ind", "lsk"], | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Simplify arguments? They are already in the non_irrigation method
Suggested change
Same for below There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed. |
||||||
non_irigation_year: int = None, | ||||||
non_irigation_method: str = "nearest", | ||||||
population_fn: str = "worldpop_2020_constrained", | ||||||
population_method: str = "sum", | ||||||
hboisgon marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
): | ||||||
"""_summary_. | ||||||
|
||||||
_extended_summary_ | ||||||
|
||||||
hboisgon marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
Parameters | ||||||
---------- | ||||||
non_irigation_fn : str, optional | ||||||
_description_, by default "pcr_globwb" | ||||||
non_irigation_vars : list, optional | ||||||
_description_, by default ["dom", "ind", "lsk"] | ||||||
non_irigation_year : int, optional | ||||||
_description_, by default None | ||||||
non_irigation_method : str, optional | ||||||
_description_, by default "nearest" | ||||||
population_fn : str, optional | ||||||
_description_, by default "worldpop_2020_constrained" | ||||||
population_method : str, optional | ||||||
hboisgon marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
_description_, by default "sum" | ||||||
|
||||||
Raises | ||||||
------ | ||||||
ValueError | ||||||
_description_ | ||||||
""" | ||||||
self.logger.info("Preparing non irigation demand maps.") | ||||||
if not all([item in ["dom", "ind", "lsk"] for item in non_irigation_vars]): | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure if we want to keep this, maybe the user wants to use pcrglob for industry and a better dataset for domestic. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That is possible, this is just a check whether all supplied vars are either 'dom', 'ind', 'lsk' or a combination of them. |
||||||
raise ValueError("") | ||||||
|
||||||
# Set flag for cyclic data | ||||||
_cyclic = False | ||||||
|
||||||
# Selecting data | ||||||
if non_irigation_year is None: | ||||||
non_irigation_year = 2005 | ||||||
non_irigation_raw = self.data_catalog.get_rasterdataset( | ||||||
non_irigation_fn, | ||||||
geom=self.region, | ||||||
buffer=2, | ||||||
variables=[ | ||||||
f"{var}_{mode}" | ||||||
for var, mode in product(non_irigation_vars, ["gross", "net"]) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess gross and net should come from the same dataset. Please document the required variables names in your docstrings that it's actually not dom like in non_irigation_vars but dom_net and dom_gross There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes. |
||||||
], | ||||||
version=non_irigation_year, | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So 2005 is not the time_tuple but actually the version selection for the data catalog? Then this is too specific to pcrglobwb in deltares_data catalog. You can specify the version in the source name instead:
Which a user can specify in their yaml config file:
If you do not specify the version, then the first is used as default so I would definitely leave it out from here. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Allright! |
||||||
) | ||||||
if "time" in non_irigation_raw.coords: | ||||||
hboisgon marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
_cyclic = True | ||||||
non_irigation_raw["time"] = non_irigation_raw.time.astype("int32") | ||||||
|
||||||
if _cyclic and self.get_config("input.cyclic") is None: | ||||||
self.set_config("input.cyclic", []) | ||||||
|
||||||
# Get population data | ||||||
pop_raw = self.data_catalog.get_rasterdataset( | ||||||
population_fn, | ||||||
geom=self.region, | ||||||
buffer=2, | ||||||
).raster.mask_nodata() | ||||||
|
||||||
# Create static water demand rasters | ||||||
non_irigation, popu = workflows.demand.non_irigation( | ||||||
non_irigation_raw, | ||||||
ds_like=self.grid, | ||||||
ds_method=non_irigation_method, | ||||||
popu=pop_raw, | ||||||
popu_method=population_method, | ||||||
) | ||||||
self.set_grid(non_irigation) | ||||||
self.set_grid(popu) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does wflow need population? It was not in the list of variables mentioned by Willem in the issue. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Its an extra map, might be useful for some custom processing/ debugging by users |
||||||
|
||||||
# Update the settings toml with non irigation stuff | ||||||
for var in non_irigation.data_vars: | ||||||
hboisgon marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
sname, suffix = var.split("_") | ||||||
lname = workflows.demand.map_vars[sname] | ||||||
self.set_config( | ||||||
f"input.vertical.{lname}.demand_{suffix}", | ||||||
var, | ||||||
) | ||||||
|
||||||
# Also for the fact that these parameters are cyclic | ||||||
if _cyclic: | ||||||
self.config["input"]["cyclic"].append( | ||||||
f"input.vertical.{lname}.demand_{suffix}", | ||||||
) | ||||||
|
||||||
def setup_irrigation( | ||||||
hboisgon marked this conversation as resolved.
Show resolved
Hide resolved
hboisgon marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
self, | ||||||
irrigated_area_fn: str, | ||||||
landuse_fn: str, | ||||||
paddy_class: int = 12, | ||||||
area_threshold: float = 0.6, | ||||||
crop_irrigated_fn: str = "mirca_irrigated_data", | ||||||
crop_rainfed_fn: str = "mirca_rainfed_data", | ||||||
crop_info_fn: str = "mirca_crop_info" | ||||||
# crop_info_fn: str, TODO, required when adding the support for rootingdepth and | ||||||
# cropfactor | ||||||
): | ||||||
""" | ||||||
Add required information to simulate irrigation water demand. | ||||||
|
||||||
The function requires data that contains information about the location of the | ||||||
irrigated areas (`irrigated_area_fn`). This, combined with a landuse data that | ||||||
contains a class for paddy (rice) land use (`landuse_fn`), determines which | ||||||
locations are considered to be paddy irrigation (based on the `paddy_class`), | ||||||
and which locations are considered to be non-paddy irrigation. | ||||||
|
||||||
Next, these maps are reprojected to the model resolution, where a threshold | ||||||
(`area_threshold`) determines when pixels are considered to be classified as | ||||||
irrigation cells (both paddy and non-paddy). It adds the resulting maps to the | ||||||
input data. | ||||||
|
||||||
|
||||||
Adds model layers: | ||||||
|
||||||
* **paddy_irrigation_areas**: Irrigated (paddy) mask [-] | ||||||
* **nonpaddy_irrigation_areas**: Irrigated (non-paddy) mask [-] | ||||||
|
||||||
Parameters | ||||||
---------- | ||||||
irrigated_area_fn: str | ||||||
Name of the (gridded) dataset that contains the location of irrigated areas | ||||||
(as a mask), `irrigated_area` for example | ||||||
landuse_fn: str | ||||||
Name of the landuse dataset that contains a classification for paddy/rice, | ||||||
use `glcnmo` for example | ||||||
paddy_class: int | ||||||
Class in the landuse data that is considered as paddy or rice, by default 12 | ||||||
(matching the glcmno landuse data) | ||||||
area_threshold: float | ||||||
Fractional area of a (wflow) pixel before it gets classified as an irrigated | ||||||
pixel, by default 0.6 | ||||||
crop_irrigated_fn: str | ||||||
Name of dataset that contains information about irrigated crops, by default | ||||||
"mirca_irrigated_data", | ||||||
crop_rainfed_fn: str | ||||||
Name of dataset that contains information about rainfed crops, by default | ||||||
"mirca_rainfed_data", | ||||||
crop_info_fn: str | ||||||
Name of dataframe that contains rootingdepth and cropfactor values for the | ||||||
rice/paddy crop class"mirca_crop_info" | ||||||
|
||||||
|
||||||
See Also | ||||||
-------- | ||||||
workflows.demand.find_paddy | ||||||
workflows.demand.classify_pixels | ||||||
""" | ||||||
self.logger.info("Preparing irrigation maps.") | ||||||
# Extract irrigated area dataset | ||||||
irrigated_area = self.data_catalog.get_rasterdataset( | ||||||
irrigated_area_fn, bbox=self.grid.raster.bounds, buffer=3 | ||||||
) | ||||||
|
||||||
# Extract landcover dataset (that includes a paddy/rice class) | ||||||
landuse_da = self.data_catalog.get_rasterdataset( | ||||||
landuse_fn, bbox=self.grid.raster.bounds, buffer=3 | ||||||
) | ||||||
|
||||||
# Get paddy and nonpaddy masks | ||||||
paddy, nonpaddy = workflows.demand.find_paddy( | ||||||
landuse_da=landuse_da, | ||||||
irrigated_area=irrigated_area, | ||||||
paddy_class=paddy_class, | ||||||
) | ||||||
|
||||||
# Get paddy and non paddy pixels at model resolution | ||||||
wflow_paddy = workflows.demand.classify_pixels( | ||||||
da_crop=paddy, | ||||||
da_model=self.grid[self._MAPS["basins"]].raster.mask_nodata(), | ||||||
threshold=area_threshold, | ||||||
) | ||||||
wflow_nonpaddy = workflows.demand.classify_pixels( | ||||||
da_crop=nonpaddy, | ||||||
da_model=self.grid[self._MAPS["basins"]].raster.mask_nodata(), | ||||||
threshold=area_threshold, | ||||||
) | ||||||
|
||||||
# Add maps to grid | ||||||
self.set_grid(wflow_paddy, name=self._MAPS["paddy_area"]) | ||||||
self.set_grid(wflow_nonpaddy, name=self._MAPS["nonpaddy_area"]) | ||||||
|
||||||
# Update config | ||||||
JoostBuitink marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
self.set_config( | ||||||
"input.vertical.paddy.irrigation_areas", self._MAPS["paddy_area"] | ||||||
) | ||||||
self.set_config( | ||||||
"input.vertical.nonpaddy.irrigation_areas", self._MAPS["nonpaddy_area"] | ||||||
) | ||||||
|
||||||
# TODO: Include this support for adjusted crop_factor and rooting depth maps, or | ||||||
# move to seperate function? | ||||||
self.logger.info("Preparing crop factor maps.") | ||||||
crop_rainfed_ds = self.data_catalog.get_rasterdataset( | ||||||
crop_rainfed_fn, | ||||||
bbox=self.grid.raster.bounds, | ||||||
buffer=3, | ||||||
) | ||||||
crop_irrigated_ds = self.data_catalog.get_rasterdataset( | ||||||
crop_irrigated_fn, | ||||||
bbox=self.grid.raster.bounds, | ||||||
buffer=3, | ||||||
) | ||||||
|
||||||
df = self.data_catalog.get_dataframe(crop_info_fn) | ||||||
# TODO: Make more flexible | ||||||
rice_value = df.loc["Rice", "kc_mid"] | ||||||
hboisgon marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
cropfactor = workflows.demand.add_crop_maps( | ||||||
ds_rain=crop_rainfed_ds, | ||||||
ds_irri=crop_irrigated_ds, | ||||||
paddy_value=rice_value, | ||||||
mod=self, | ||||||
default_value=1.0, | ||||||
map_type="crop_factor", | ||||||
) | ||||||
|
||||||
# Add maps to grid and update config | ||||||
self.set_grid(cropfactor, name=self._MAPS["crop_factor"]) | ||||||
self.set_config("input.vertical.kc", self._MAPS["crop_factor"]) | ||||||
|
||||||
# # TODO: Make more flexible? | ||||||
# rice_value = df.loc["Rice", "rootingdepth_irrigated"] | ||||||
|
||||||
# rootingdepth = workflows.demand.add_crop_maps( | ||||||
# ds_rain=mirca_rain_ds, | ||||||
# ds_irri=mirca_irri_ds, | ||||||
# paddy_value=rice_value, | ||||||
# mod=self, | ||||||
# default_value=self.grid["RootingDepth"], | ||||||
# map_type="rootingdepth", | ||||||
# ) | ||||||
|
||||||
# I/O | ||||||
def read(self): | ||||||
"""Read the complete model schematization and configuration from file.""" | ||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes from this PR should move to the unreleased section