00 - Lazy Evaluation, Graphs and TensorFlow.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lazy evaluation and graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Just in Time calculation or Pure Python\n",
    "\n",
    "Doing a simple calculation and generating a value as it could be found in any function (like calculating the NLL)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "const_5 = 5\n",
    "const_3 = 3\n",
    "sum_5_3 = const_5 + const_3\n",
    "\n",
    "const_7 = 7\n",
    "const_2 = 2\n",
    "sum_7_2 = const_7 + const_2\n",
    "\n",
    "prod_sums = sum_5_3 * sum_7_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get an object out of the calculations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "72"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prod_sums"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unfortunately, this was not efficient and we cannot deduce any more information form the output about it (like where it came from, equivalent to doing a simple physics exercise algebraicly versus directly entering the numbers into every variable)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lazy evaluation\n",
    "\n",
    "So it would be better, to \"build\" the calculation in the first place. Therefore, we need so-called \"lazy evaluation\", an object that first gets composed and run afterwards.\n",
    "\n",
    "Let's do this in python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Two equivalent ways of writing a function in python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def func():\n",
    "    return 42\n",
    "func = lambda: 42"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "const_5 = lambda: 5\n",
    "const_3 = lambda: 3\n",
    "sum_5_3 = lambda: const_5() + const_3()\n",
    "\n",
    "const_7 = lambda: 7\n",
    "const_2 = lambda: 2\n",
    "sum_7_2 = lambda: const_7() + const_2()\n",
    "\n",
    "prod_sums = lambda: sum_5_3() * sum_7_2()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now we did not yet evaluate anything."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function __main__.<lambda>()>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prod_sums"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To evaluate this object, we simply call it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "72"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prod_sums()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The advantage over the previous approach: we _could_ use the information stored in prod_sums to improve the calculation _before_ we run it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "prod_sums = lambda: sum_5_3() * sum_7_2()\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import inspect\n",
    "lines = inspect.getsource(prod_sums)\n",
    "print(lines)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Graphs\n",
    "\n",
    "Of course, this was a simple example. Instead of setting up something like this, let's use an implementation: TensorFlow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from zfit import ztf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "const_5 = ztf.constant(5)\n",
    "const_3 = ztf.constant(3)\n",
    "sum_5_3 = const_5 + const_3  # equivalent to the below one\n",
    "# sum_5_3 = tf.add(const_5, const_3)\n",
    "\n",
    "const_7 = ztf.constant(7)\n",
    "const_2 = ztf.constant(2)\n",
    "sum_7_2 = const_7 + const_2\n",
    "\n",
    "prod_sums = sum_5_3 * sum_7_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'mul:0' shape=() dtype=float64>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prod_sums"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we hat a function, that was our lazy evaluatable object, now it's a Tensor. Names don't matter here.\n",
    "\n",
    "To run it, instead of putting parenthesis around, we use something else"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "import zfit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "72.0"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "zfit.run(prod_sums)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now let's look at the graph!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'mul:0' shape=() dtype=float64>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prod_sums  # output from the operation..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Operation 'mul' type=Mul>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prod_sums.op  # multiplies the inputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<tf.Tensor 'add:0' shape=() dtype=float64>,\n",
       " <tf.Tensor 'add_1:0' shape=() dtype=float64>]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prod_sums.op.inputs[:]  # with for example input 0 from the op.."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Operation 'add' type=Add>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prod_sums.op.inputs[0].op  # the first add with inputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<tf.Tensor 'Const_1:0' shape=() dtype=float64>,\n",
       " <tf.Tensor 'Const_2:0' shape=() dtype=float64>]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prod_sums.op.inputs[0].op.inputs[:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have the whole definition at hand! It is a simple matter of implementations to, for example, check if a value depends on another (by recursively searching its inputs if it is there).\n",
    "\n",
    "More useful: using the chain rule, we are able to compute the derivative of **any** value with respect to **any** other value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "derivate_sum_by_const5 = tf.gradients(prod_sums, const_5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<tf.Tensor 'gradients/mul_grad/Reshape:0' shape=() dtype=float64>]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "derivate_sum_by_const5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[9.0]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "zfit.run(derivate_sum_by_const5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What we didn't noticed: The whole code was also parallelized automatically on any available CPU/GPU."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}