Improve documentation 2

doc/customization_basic1.rst

@@ -0,0 +1,82 @@
+.. SPDX-FileCopyrightText:  PyPSA-Earth and PyPSA-Eur Authors
+..
+.. SPDX-License-Identifier: CC-BY-4.0
+
+.. _customization_basic1:
+
+#######################
+1. Basic customization
+#######################
+
+A good starting point to customize your model are settings of the default configuration file `config.default`. You may want to do a reserve copy of your current configuration file and then overwrite it by a default configuration:
+
+.. code:: bash
+
+    .../pypsa-earth (pypsa-earth) % cp config.default.yaml config.yaml
+
+The model can be adapted to include any country, multiple countries (e.g. Nigeria and Benin) or continents (currently `Africa` work as a whole continent) using `countries` argument:
+
+.. code:: yaml
+
+    countries: ["NG", "BJ"]
+
+.. note::
+
+    To build the model of regions outside of Africa, it is important to setup ``Copernicus Climate Data Store (CDS) API`` to build custom cutouts.
+    The same is true if the weather year other than 2013 is considered even for Africa. The detailed instructions for setting up the Copernicus API can be found in :ref:`customization_copernicus`.
+
+Likewise, the example's temporal scope can be restricted (e.g. to 7 days):
+
+.. code:: yaml
+
+    snapshots:
+        start: "2013-03-1"
+        end: "2013-03-7"
+        inclusive: "left" # end is not inclusive
+
+
+Year-related parameters are also being used  when specifying `load_options`:
+
+.. code:: yaml
+
+    load_options:
+      ssp: "ssp2-2.6"
+      weather_year: 2013
+      prediction_year: 2030
+      scale: 1
+
+The `weather_year` value corresponds to the weather data which was used to generate the electricity demand profiles for a selected area while `prediction_year` correspond to the point of a ssp trajectory. The available values for `weather_year` and `prediction_year` can be checked by looking into `pypsa-earth/data/ssp2-2.6` folder. Currently, there are pre-calculated demand data for 2011, 2013, 2018 weather years and for 2030, 2040, 2050, and 2100 scenario prediction years.
+
+To accurately model the temporal and spatial availability of renewables such as wind and solar energy, we process historical weather data using `atlite <https://atlite.readthedocs.io/en/latest/>`__ package.
+In order to simulate a specific country, it is crucial to specify the `cutout region` within the atlite configuration in the ``config.yaml`` file:
+
+.. code:: yaml
+
+    atlite:
+        nprocesses: 4
+        cutouts:
+            cutout-2013-era5-tutorial:
+                module: era5
+                dx: 0.3  # cutout resolution
+                dy: 0.3  # cutout resolution
+                # The cutout time is automatically set by the snapshot range.
+
+Replace ``cutout-2013-era5-tutorial`` with the region of interest. For example, when simulating Kazakhstan, it should be updated to ``asia-2013-era5``.
+Note please that a temporal dimension of the cutout should be consistent with the values set for `snapshots` parameter. A time range of the cutout is determined by the parameters set when building this cutout while the time resolution corresponds to those of the used climate archives. In case of ERA5 dataset used in PyPSA-Earth by default, hourly resolution is implied.
+It is also possible to decide which weather data source should be used to calculate potentials and capacity factor time-series for each carrier.
+For example, we may want to use the ERA-5 dataset for solar and not the default SARAH-2 dataset. Visit :ref:`config` page to get familiar with configuration details.
+
+Finally, for countries beyond Africa, it is imperative to initially set ``build_cutout: true`` for the first run. This facilitates the construction of cutouts from weather data. Subsequently, it can be switched to false to avoid reconstructing the cutout. The same applies for other parameters in ``enable`` section.
+
+.. code:: yaml
+
+    enable:
+        retrieve_databundle: true  #  Recommended 'true', for the first run. Otherwise data might be missing.
+        retrieve_cost_data: true  # true: retrieves cost data from technology data and saves in resources/costs.csv, false: uses cost data in data/costs.csv
+        download_osm_data: true  # If 'true', OpenStreetMap data will be downloaded for the above given countries
+        build_natura_raster: true # If True, than an exclusion raster will be build
+        build_cutout: false
+        # If "build_cutout" : true, then environmental data is extracted according to `snapshots` date range and `countries`
+
+To delve into the specifics of the provided configurations and explore additional settings, please refer to the :ref:`config` page.
+There are many more configuration options beyond what is adapted for the tutorial!

doc/customization_copernicus.rst

@@ -0,0 +1,29 @@
+.. SPDX-FileCopyrightText:  PyPSA-Earth and PyPSA-Eur Authors
+..
+.. SPDX-License-Identifier: CC-BY-4.0
+
+.. _customization_copernicus:
+
+####################
+Setup Copernicus API
+####################
+
+
+To build the model of regions outside of Africa, it is important to access to `Copernicus Climate Data Store <https://cds.climate.copernicus.eu>`_ to build custom cutouts.
+The same is true if the weather year other than 2013 is considered for the region of interest in Africa.
+
+.. note::
+
+    Skip this recommendation if the region of your interest is within Africa and you are fine with the 2013 weather year
+
+Steps to get access to Copernicus database:
+
+1. Register to  the `Copernicus Climate Data Store <https://cds.climate.copernicus.eu>`_;
+2. Install `cdsapi` package  (can be installed with `pip`);
+3. Setup your CDS API key as described `on their website <https://cds.climate.copernicus.eu/api-how-to>`_.
+
+These steps are required to use CDS API which allows an automatic file download while executing `build_cutouts` rule.
+
+The `build_cutout` flag should be set `true` to generate the cutout. After the cutout is ready, it's recommended to set `build_cutout` to `false` to avoid overwriting the existing cutout by accident. The `snapshots` values set when generating the cutout, will determine the temporal parameters of the cutout. Accessible years which can be used to build a cutout depend on ERA5 data availability. `ERA5 page <https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5>`_ explains that the data is available from 1950 and updated continuously with about 3 month delay while the data on 1950-1978 should be treated as preliminary as that is a rather recent development.
+
+After the first run, if you don't change country and don't need to increase a considered time span wider than the one you created the cutout with, you may set both `retrieve_databundle` and `build_cutout` to false.

doc/customization_run.rst

@@ -0,0 +1,56 @@
+.. SPDX-FileCopyrightText:  PyPSA-Earth and PyPSA-Eur Authors
+..
+.. SPDX-License-Identifier: CC-BY-4.0
+
+.. _customization_run:
+
+####################
+2. Running the model
+####################
+
+To solve full optimization problem, it is important to pick a solver in `config.yaml` file. For instance, this tutorial uses the open-source solver glpk and does not rely
+on the commercial solvers such as Gurobi or CPLEX (for which free academic licenses are available).
+
+.. code:: yaml
+
+    solver:
+        name: glpk
+
+.. note::
+
+    ``glpk`` can solve the network with low temporal and spacial resolution. To make a full model run, it is adviced to use ``CPLEX``, ``Gurobi``, or open-source `HIGHs <https://highs.dev/>`__.
+
+To execute a full model run, the following command needs to be applied:
+
+.. code:: bash
+
+    .../pypsa-earth (pypsa-earth) % snakemake --cores 1 solve_all_networks
+
+Here, ``snakemake`` is a workflow management tool inherited by PyPSA-Earth from PyPSA-Eur.
+Snakemake decomposes a large software process into a set of subtasks, or ’rules’, that are automatically chained to obtain the desired output.
+The flag ``--cores 1`` dictates the number of CPU cores allocated for the process. Notably, the ``solve_all_network`` rule within Snakemake orchestrates the process of solving the network.
+
+.. note::
+
+  ``Snakemake``, which is one of the major dependencies, will be automatically installed in ``pypsa-earth`` environment, thereby there is no need to install it manually.
+
+The snakemake included in the ``pypsa-earth`` conda environment pypsa-earth can be used to execute any custom rule with the following command:
+
+.. code:: bash
+
+    .../pypsa-earth (pypsa-earth) % snakemake < your custom rule >
+
+Starting with essential usability features, PyPSA-Earth `Snakemake procedure <https://github.com/pypsa-meets-earth/pypsa-earth/blob/main/Snakefile>`_ enables users to execute the entire workflow flexibly with diverse options, requiring no Python coding. For example, users can model the global energy system or specific subsets of countries using only necessary data. Wildcards, serving as special generic keys, adapt to multiple values based on configuration options, facilitating the execution of extensive workflows with parameter sweeps and diverse options.
+
+You can execute some parts of the workflow in case you are interested in some specific parts.
+E.g. power grid topology may be extracted and cleaned with the following command which refers to the script name:
+
+.. code:: bash
+
+    .../pypsa-earth (pypsa-earth) % snakemake -j 1 clean_osm_data
+
+Solar profile for the requested area may be calculated using the output name:
+
+.. code:: bash
+
+    .../pypsa-earth (pypsa-earth) % snakemake -j 1 resources/renewable_profiles/profile_solar.nc

doc/customization_steps.rst

@@ -0,0 +1,56 @@
+.. SPDX-FileCopyrightText:  PyPSA-Earth and PyPSA-Eur Authors
+..
+.. SPDX-License-Identifier: CC-BY-4.0
+
+.. _customization_steps:
+
+#######################################
+3. General Guidelines for Modeling
+#######################################
+
+
+1. Configure Model Parameters
+-----------------------------
+
+Begin by adjusting the model inputs in the ``config.yaml`` configuration file. This file serves as the cornerstone for tailoring PyPSA-Earth to specific regional requirements.
+
+Pay particular attention to regional-dependent parameters such as:
+
+* ``countries``: This parameter allows you to specify the countries to be included in your model. It's crucial for accurately representing the geographical scope of your analysis.
+
+* ``cutouts``: These parameters refer to the climate data archive name used for calculating renewable potential. By defining appropriate cutouts, you ensure that your model is equipped with the necessary weather data for accurate simulations.
+
+2. Generate Custom Cutout
+-------------------------
+
+The concept of a `cutout`` is central to climate data management in the PyPSA ecosystem, facilitated through the ``build_cutout`` rule.
+A cutout essentially represents a spatio-temporal subset of topology and weather datasets, allowing you to focus on specific regions and timeframes relevant to your analysis.
+It's important to understand that the spatial and temporal resolution of your cutout data is determined by the underlying dataset. Typically, a resolution of 30km x 30km grid and hourly data are recommended for optimal results.
+While a pre-built cutout for Africa is readily available for the 2013 year, users interested in other regions or updated data can generate their own cutouts by enabling the ``build_cutout`` rule. This process involves accessing the Copernicus Climate Data Store and installing the necessary packages for data retrieval as described in :ref:`customization_copernicus`.
+
+3. Assess `natura.tiff` Raster
+--------------------------------
+
+The ``natura.tiff`` raster file delineates areas designated as protected and reserved nature zones. These zones impose land use restrictions, influencing the calculation of renewable potential within a given geographical region.
+By activating the ``natura`` option and incorporating the ``natura.tiff`` file, users can account for protected and reserved nature areas, where renewable assets cannot be installed:
+
+.. code:: yaml
+
+    renewable:
+        onwind:
+            natura: true
+
+.. note::
+
+    By default, the option ``natura: true`` is set for all renewable sources.
+
+It's worth noting that a pre-built ``natura.tiff`` file is included in the dataset and dowloaded through ``retrieve_databundle_light`` rule, offering a global coverage of protected areas. This default file serves as a foundational resource for model execution.
+
+While the default file suffices for most analyses, users seeking the latest updates and revisions can opt to enable the ``build_natura_raster`` rule:
+
+.. code:: yaml
+
+    enable:
+        build_natura_raster: true
+
+However, it's worth noting that generating a customized ``natura.tiff`` file with the latest data may require a significant amount of time due to processing complexities.

doc/customization_validation.rst

@@ -0,0 +1,9 @@
+.. SPDX-FileCopyrightText:  PyPSA-Earth and PyPSA-Eur Authors
+..
+.. SPDX-License-Identifier: CC-BY-4.0
+
+.. _customization_validation:
+
+################
+Model Validation
+################

doc/tutorial.rst

@@ -6,9 +6,24 @@
 
 
 ##########################################
-Full model
+Model customization
 ##########################################
 
+* :doc:`customization_basic1`
+* :doc:`customization_run`
+* :doc:`customization_steps`
+* :doc:`customization_copernicus`
+
+.. toctree::
+   :hidden:
+   :maxdepth: 2
+   :caption: Model customization
+
+   customization_basic1
+   customization_run
+   customization_steps
+   customization_copernicus
+
 How to customise PyPSA-Earth?
 =============================