Skip to content

Commit

Permalink
Add cubed notebook for hourly climatology example using "map-reduce" …
Browse files Browse the repository at this point in the history
…method (#356)

* Add cubed notebook for hourly climatology example using "map-reduce" method

* Add cubed dependencies to docs build

* Use Cubed version with HTML repr fix
  • Loading branch information
tomwhite authored Apr 29, 2024
1 parent 627bf2b commit efd88e1
Show file tree
Hide file tree
Showing 3 changed files with 109 additions and 0 deletions.
2 changes: 2 additions & 0 deletions ci/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ name: flox-doc
channels:
- conda-forge
dependencies:
- cubed>=0.14.3
- cubed-xarray
- dask-core
- pip
- xarray
Expand Down
1 change: 1 addition & 0 deletions docs/source/user-stories.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
user-stories/overlaps.md
user-stories/climatology.ipynb
user-stories/climatology-hourly.ipynb
user-stories/climatology-hourly-cubed.ipynb
user-stories/custom-aggregations.ipynb
user-stories/nD-bins.ipynb
```
106 changes: 106 additions & 0 deletions docs/source/user-stories/climatology-hourly-cubed.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "0",
"metadata": {},
"source": [
"# More climatology reductions using Cubed\n",
"\n",
"This is the Cubed equivalent of [More climatology reductions](climatology-hourly.ipynb).\n",
"\n",
"The task is to compute an hourly climatology from an hourly dataset with 744 hours in each chunk, using the \"map-reduce\" strategy."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1",
"metadata": {},
"outputs": [],
"source": [
"import cubed\n",
"import cubed.array_api as xp\n",
"import numpy as np\n",
"import pandas as pd\n",
"import xarray as xr\n",
"\n",
"import flox.xarray"
]
},
{
"cell_type": "markdown",
"id": "2",
"metadata": {},
"source": [
"## Create data\n",
"\n",
"Note that we use fewer lat/long points so the computation can be run locally."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3",
"metadata": {},
"outputs": [],
"source": [
"spec = cubed.Spec(allowed_mem=\"2GB\")\n",
"ds = xr.Dataset(\n",
" {\n",
" \"tp\": (\n",
" (\"time\", \"latitude\", \"longitude\"),\n",
" xp.ones((8760, 72, 144), chunks=(744, 5, 144), dtype=np.float32, spec=spec),\n",
" )\n",
" },\n",
" coords={\"time\": pd.date_range(\"2021-01-01\", \"2021-12-31 23:59\", freq=\"h\")},\n",
")\n",
"ds"
]
},
{
"cell_type": "markdown",
"id": "4",
"metadata": {},
"source": [
"## Computation"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5",
"metadata": {},
"outputs": [],
"source": [
"hourly = flox.xarray.xarray_reduce(ds.tp, ds.time.dt.hour, func=\"mean\", reindex=True)\n",
"hourly"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6",
"metadata": {},
"outputs": [],
"source": [
"hourly.compute()"
]
}
],
"metadata": {
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

0 comments on commit efd88e1

Please sign in to comment.