4_how_to_debug.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to debug?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Debuging is essential for users and code contributors. Without knowing its basic you loose plenty of your valuable time!\n",
    " \n",
    "This tutorial will cover:\n",
    "- Intro to VS Code debugging\n",
    "- Debugging the workflow\n",
    "- Reading errors\n",
    "- Understand how to fix dependecies of a workflow\n",
    "\n",
    "Requirements:\n",
    "- Installed PyPSA-Africa (i.e. in `~/home/pypsa-africa`)\n",
    "- Installed anaconda environment `(pypsa-africa)`\n",
    "- Tutorial on \"how to execute the workflow\" should be finished\n",
    "- **VS Code** (Visual Studio Code)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may need to install **ipython** while being in the `pypsa-africa` environment to choose the environment in VS code/Jupyter lab. Install the ipython kernel like this:\n",
    "\n",
    "```\n",
    "(pypsa-africa).../pypsa-africa % ipython kernel install --user --name=pypsa-africa\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally we need to import a package to display images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython import display"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction to VS Code debugging"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First let's open VS code. To see the screen below:\n",
    "1. open the terminal\n",
    "2. locate the path into the `pypsa-africa` folder i.e. `$ ~\\home\\max\\pypsa-africa`\n",
    "3. open VS code by typing `code .` into the terminal (the `.` means current location, and `code` will open VScode) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will test the debugging function by looking into `build_shapes.py` function which rule we just analysed previously in the *How to execute the workflow* tutorial. \n",
    "\n",
    "\n",
    "4. Let's open the `build_shapes.py`\n",
    "5. Set the `red dots`, also called `breakpoints`, as seen in the figure\n",
    "6. On the left side you see a `play button` with a small bug (see below). Click on it and try to `run and debug`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display.Image(\"./images/hackathon/vsc_bug.png\", width=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Maybe you will run into the first error. A common thing is that you missed to activate the `pypsa-africa` environment in the terminal. Try to activate the environment (`conda activate pypsa-africa`) and repeat step 6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display.Image(\"./images/hackathon/vscode_intro.png\", width=800)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Debug in Action\n",
    "\n",
    "You can see below how it would look like to debug the `build_shapes` rule. \n",
    "\n",
    "In the **DEBUG CONSOLE** you can now execute everything above the set breakpoint. \n",
    "\n",
    "Next to simply *see* variables, you can *modify*, *analyse* and even *plot* the data in this console."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display.Image(\"./images/hackathon/debug_in_action.png\", width=800)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running into errors and reading them\n",
    "\n",
    "With Snakemake, running into an error might be overwhelming. Let's check out a **closed** well reported issue from PyPSA-Eur.\n",
    "\n",
    "1. Click on the link and try understanding the issue: https://github.com/PyPSA/pypsa-eur/issues/221\n",
    "\n",
    "Workflow errors messages can be divided in the following parts:\n",
    "- **Rule description**, which shows you which rule the error occurs\n",
    "- **Traceback**, shows the most recent functions before the code breaks and helps locating the error line\n",
    "- **Break line**, the line which causes the error. It is the function/statement at the bottom of the traceback. \n",
    "\n",
    "```\n",
    "File \"/Users/<path>/scripts/tmpi7lseb_2.cluster_network.py\", line 190, in distribute_clusters\n",
    "    assert n_clusters >= len(N) and n_clusters <= N.sum(), \\\n",
    "AssertionError: Number of clusters must be 37 <= n_clusters <= 3476 for this selection of countries.\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes you run into an error but actually the issue is on your side. Most common false alarms are:\n",
    "- Environment is outdated\n",
    "- Environment is not activated\n",
    "\n",
    "A fresh setup can make wonders happening."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bug in one of the packages in the environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes you have a bug in a package for instance `atlite`, `pypsa`, `geopandas` or similar. Again, look **first of this bug is already tracked by someone**, if not share the bug with the community. You can try to fix it in your environment i.e. `pypsa-africa`. *Did you know that you always have local installation for the packages in each environment?*\n",
    "\n",
    " **Try to find** in your anaconda installation, the `pypsa-africa` environment and search for the `pypsa` package. \n",
    "```\n",
    "/home/<user>/anaconda3/envs/pypsa-africa/lib/python3.9/site-packages/pypsa\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Try it yourself."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Create** yourself errors and **understand** the error messages in the following scripts:\n",
    "- cluster_network.py\n",
    "- delete something in the Snakefile\n",
    "- remove something in the config.yaml\n",
    "- create an error in the `pypsa` package of your `pypsa-africa` environment\n",
    "\n",
    "Learning by doing is a great recipe!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Key points:\n",
    "- You understand how to debug in VS Code\n",
    "- You can use the Debug Console\n",
    "- You can read workflow error messages\n",
    "- You know how to fix bugs in dependecies of your workflow (i.e. atlite)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "b34881217c78f48c1cb2cd3434cb375f471e98a15a99a90b485a8349351a20f8"
  },
  "kernelspec": {
   "display_name": "pypsa-africa",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}