Chapter21_Sensitivity_specificity_risks_rates_odds.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Setting up a fancy stylesheet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>\n",
       "\n",
       "<style>\n",
       "\n",
       "@font-face {\n",
       "    font-family: \"Computer Modern\";\n",
       "    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
       "}\n",
       "\n",
       "#notebook_panel { /* main background */\n",
       "    background: #ddd;\n",
       "    color: #000000;\n",
       "}\n",
       "\n",
       "\n",
       "div.text_cell_render{\n",
       "    font-family: 'Arvo' sans-serif;\n",
       "    line-height: 130%;\n",
       "    font-size: 115%;\n",
       "    width:1000px;\n",
       "    margin-left:auto;\n",
       "    margin-right:auto;\n",
       "}\n",
       "\n",
       "\n",
       "/* Formatting for header cells */\n",
       ".text_cell_render h1 {\n",
       "    font-family: 'Philosopher', sans-serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 2.2em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(0, 80, 120);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       ".text_cell_render h2 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 1.9em;\n",
       "    line-height: 100%;\n",
       "    color: rgb(200,100,0);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\t\n",
       "\n",
       ".text_cell_render h3 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "    margin-top:12px;\n",
       "    margin-bottom: 3px;\n",
       "    font-style: italic;\n",
       "    color: rgb(94,127,192);\n",
       "}\n",
       "\n",
       ".text_cell_render h4 {\n",
       "    font-family: 'Philosopher', serif;\n",
       "}\n",
       "\n",
       ".text_cell_render h5 {\n",
       "    font-family: 'Alegreya Sans', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 16pt;\n",
       "    color: grey;\n",
       "    font-style: italic;\n",
       "    margin-bottom: .1em;\n",
       "    margin-top: 0.1em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h6 {\n",
       "    font-family: 'PT Mono', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 10pt;\n",
       "    color: grey;\n",
       "    margin-bottom: 1px;\n",
       "    margin-top: 1px;\n",
       "}\n",
       "\n",
       ".CodeMirror{\n",
       "        font-family: \"PT Mono\";\n",
       "        font-size: 100%;\n",
       "}\n",
       "\n",
       "</style>\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.core.display import HTML\n",
    "css_file = 'style.css'\n",
    "HTML(open(css_file, 'r').read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Setting up the required python &#8482; environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#import numpy as np\n",
    "import pandas as pd\n",
    "#from scipy.stats import bayes_mvs\n",
    "#from math import factorial\n",
    "#import scikits.bootstrap as bs\n",
    "#import matplotlib.pyplot as plt\n",
    "#import seaborn as sns\n",
    "from warnings import filterwarnings\n",
    "\n",
    "#%matplotlib inline\n",
    "filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>When considering the use of special investigations or even clinical findings, it is important to understand how accurate these are and how this impacts on their use and the clinical decisions we make that depend on them.<p>\n",
    "<p>Whereas the sensitivity and specificity are independent of the prevalence of a condition, predictive values are and incidence profoundly affects the use of their reported values.  Fortunately, there are equations that can translate reported predictive values to a relevant prevalence.<p>\n",
    "<p>Predictive values make use of a form of Bayesian statistics.  Bayesian inference makes use of past knowledge (as gained from research) to predict, in essence, the future.<p>\n",
    "In this chapter I will look into the calculations and use of sensitivity, specificity, predictive values, incidence, cumulative risk, prevalence and some ratios."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sensitivity and specificity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Let's start by short explanations for sensitivity and specificity in the context of healthcare.  *Sensitivity* expresses the probability that a test will return a positive result given the existence of a (disease) condition.  In contrast to this, *specificity* expresses the probability that the same test will return a negative result given the absence of that disease condition.<p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculating sensitivity and specifcity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A simple example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Consider a research study in which, for instance, 1000 volunteers are divided into two groups.  In group one, a specific disease is absolutely shown to be present.  A good example of this would be tissue taken for histological examination and proved to be diseased.  This histological examination will be taken as the gold standard.  In group two, this disease is absolutely known to be absent.</p>\n",
    "<p>Study participants are then investigated with a new test.  Results for this test are either *positive* or *negative*.  Four results follow from this.\n",
    "* Group with disease\n",
    "    * A positive result\n",
    "    * A negative result\n",
    "* Group B\n",
    "    * A positive result\n",
    "    * A negative result\n",
    "    \n",
    "<p>We use specific words to desrcibe these results.  A positive result in a participant with the disease is termed a *true positive*.  A negative result in a participant with the disease is a *false negative*.</p>\n",
    "<p>A negative result in a participant without the disese is a *true negative* and a positive in this group would be a *false positive*.</p>\n",
    "<p>*Sensitivity* is then a simple ratio between the true positive incidence and the total number of participants with the disease.  *Specificity* is the ratio between the true negative rate and the number of participants without the disease.</p>\n",
    "<p>Sensitivity and specificity should be used and understood from the point of view of the decision to use a test in the work-up of a condition.  If a test has a high sensitivity it will diagnose a high percentage of patient who have the disease.  Likewise, if a test has a high specificity it will indicate a large percentage of patients who do not have that disease.</p>\n",
    "<p>Note also, how crucial it is to have a gold standard to divide the groups with.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>In this simple example of 1000 participants let's suppose that 10 individuals have the disease and an overwhelming 990 do not.  Of the 10 with the disease, nine tests positive (true positive) and tests negative (false negative).  Of the 990 without the disease, 90 test positive (false positive) and 900 test negative (true negative).</p>\n",
    "<p>Sensitivity (*S<sub>n</sub>*) is a simple fraction with true positive (*P<sub>T</sub>*) in the numerator and the sum of the true positives (*P<sub>T</sub>*) and the false negatives (*N<sub>F</sub>*) (in other words, all (10) with the disease) in the denominator.</p>\n",
    "<p>Specificity (*S<sub>p</sub>*) has true negative (*N<sub>T</sub>*) in the numerator and the sum of the true negative (*N<sub>T</sub>*) and false positive (*P<sub>F</sub>*) (sum of all without the disease) in the denominator.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ { S }_{ N }=\\frac { { P }_{ T } }{ { P }_{ T }+{ F }_{ N } }  $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ {{S}_{P} = \\frac {{N}_{T}}{{{N}_{T}} + {{P}_{F}}}} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Our code could look something like this:</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "true_positive = 9 # Create a computer variable called true_positive and give it the value 9\n",
    "false_negative = 1 # True and False are python code words, so don't use these as computer\n",
    "# variable names\n",
    "false_positive = 90\n",
    "true_negative = 900\n",
    "\n",
    "sensitivity = true_positive / (true_positive + false_negative) # A simple equation\n",
    "specificity = true_negative / (true_negative + false_positive) # Note to add denominator values\n",
    "# in brackets to make the algebra (order of arithmetic execution) work out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The sensitivity is:  90.0 %\n"
     ]
    }
   ],
   "source": [
    "print('The sensitivity is: ', (sensitivity * 100), '%')\n",
    "# Using the python 3.x print statement\n",
    "# Note the use of quotation marks for literal text\n",
    "# Multiplying the values with 100 to get percentage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The specificity is:  90.9090909090909 %\n"
     ]
    }
   ],
   "source": [
    "print('The specificity is: ', (specificity * 100), '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We could no say that this new test will pick up 90% of those with the disease with a positive result and 90% of those without, with a negative result.<p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example using our mock data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We'll do another example importing our mock data.</p>\n",
    "<p>I won't use the powerful * pandas .groupby()* function, instead doing a laborious construction of new *DataFrame* objects, to make it easy to follow along.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>File</th>\n",
       "      <th>Age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Delay</th>\n",
       "      <th>Stay</th>\n",
       "      <th>ICU</th>\n",
       "      <th>RVD</th>\n",
       "      <th>CD4</th>\n",
       "      <th>HR</th>\n",
       "      <th>Temp</th>\n",
       "      <th>CRP</th>\n",
       "      <th>WCC</th>\n",
       "      <th>HB</th>\n",
       "      <th>Rupture</th>\n",
       "      <th>Histo</th>\n",
       "      <th>Comp</th>\n",
       "      <th>MASS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>Female</td>\n",
       "      <td>3</td>\n",
       "      <td>6</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>NaN</td>\n",
       "      <td>97</td>\n",
       "      <td>35.2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10.49</td>\n",
       "      <td>10.4</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>32</td>\n",
       "      <td>Male</td>\n",
       "      <td>6</td>\n",
       "      <td>10</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>57</td>\n",
       "      <td>109</td>\n",
       "      <td>38.8</td>\n",
       "      <td>45.3</td>\n",
       "      <td>7.08</td>\n",
       "      <td>19.8</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>19</td>\n",
       "      <td>Female</td>\n",
       "      <td>1</td>\n",
       "      <td>16</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>NaN</td>\n",
       "      <td>120</td>\n",
       "      <td>36.3</td>\n",
       "      <td>10.7</td>\n",
       "      <td>13.00</td>\n",
       "      <td>8.7</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   File  Age  Gender  Delay  Stay ICU  RVD  CD4   HR  Temp   CRP    WCC    HB  \\\n",
       "0     1   38  Female      3     6  No   No  NaN   97  35.2   NaN  10.49  10.4   \n",
       "1     2   32    Male      6    10  No  Yes   57  109  38.8  45.3   7.08  19.8   \n",
       "2     3   19  Female      1    16  No   No  NaN  120  36.3  10.7  13.00   8.7   \n",
       "\n",
       "  Rupture Histo Comp  MASS  \n",
       "0      No   Yes  Yes     5  \n",
       "1      No    No  Yes     8  \n",
       "2      No    No   No     3  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv('MOOC_Mock.csv')\n",
    "# Imporitng the csv file and attaching it to the computer variable (object) arbitrarily  called data\n",
    "data.head(3) # Inspecting the first 3 rows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We will divide our data set into those with and without histologically proven appendicitis.  The histological evaluation is our gold standard.  We will now look at using a white cell count (WCC) of 12 or more as a positive result.<p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "appen_pos = data[data['Histo'] == 'Yes'] # A new DataFrame including only those with\n",
    "# histologically proven appendicitis\n",
    "appen_neg = data[data['Histo'] == 'No']\n",
    "\n",
    "tp_all = appen_pos[appen_pos['WCC'] >= 12] # A new DataFrame object from the appen_pos\n",
    "# DataFrame with a WCC of 12 or more as a positive result, hence true positive\n",
    "fn_all = appen_pos[appen_pos['WCC'] < 12]\n",
    "\n",
    "tn_all = appen_neg[appen_neg['WCC'] < 12]\n",
    "fp_all = appen_neg[appen_neg['WCC'] >= 12]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    87.000000\n",
       "mean     16.507471\n",
       "std       3.729035\n",
       "min      12.000000\n",
       "25%      13.605000\n",
       "50%      15.650000\n",
       "75%      17.720000\n",
       "max      26.400000\n",
       "Name: WCC, dtype: float64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tp_all['WCC'].dropna().describe()  # Just interested in the counts here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    31.000000\n",
       "mean      9.361935\n",
       "std       2.010896\n",
       "min       2.950000\n",
       "25%       8.195000\n",
       "50%       9.390000\n",
       "75%      10.650000\n",
       "max      11.990000\n",
       "Name: WCC, dtype: float64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fn_all['WCC'].dropna().describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    18.000000\n",
       "mean      8.375000\n",
       "std       2.481184\n",
       "min       4.220000\n",
       "25%       6.795000\n",
       "50%       8.155000\n",
       "75%      10.542500\n",
       "max      11.700000\n",
       "Name: WCC, dtype: float64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tn_all['WCC'].dropna().describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    11.000000\n",
       "mean     16.730909\n",
       "std       3.835021\n",
       "min      13.000000\n",
       "25%      14.155000\n",
       "50%      15.970000\n",
       "75%      17.780000\n",
       "max      26.190000\n",
       "Name: WCC, dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fp_all['WCC'].dropna().describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "tp = 87\n",
    "fn = 31\n",
    "tn = 18\n",
    "fp = 11\n",
    "\n",
    "sens = tp / (tp + fn)\n",
    "spec = tn / (tn + fp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The sensitivity of a raised WCC in appendicitis is:  73.72881355932203 %\n"
     ]
    }
   ],
   "source": [
    "print('The sensitivity of a raised WCC in appendicitis is: ', (sens * 100), '%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The specificity of a raised WCC in appendicitis is:  62.06896551724138 %\n"
     ]
    }
   ],
   "source": [
    "print('The specificity of a raised WCC in appendicitis is: ', (spec * 100), '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>There we have it.  A sensitivity and specificity value for a raised WCC in our data set.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicitive values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Predictive values make use of a form of Bayesian statistics.  Bayesian inference makes use of past knowledge (as gained from research) to predict, in essence, the future.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>In contrast to sensitivity and specificity, predictive values should be viewed from the point of view that the test was already performed.  The values can either be expressed as a *positive* predictive value, which expresses the probability that a patient has a disease once the test result was positive, or as a *negative* predictive value, which expresses the probability that the patient does not have the disease, once the test result is negative.</p>\n",
    "<p>Predictive values should be viewed only once a clear indication of the prevalence of the disease in the particular study is given, as this greatly influences the result.  In order for predictive values to be used properly, they have to be converted to reflect the prevalence of a disease in the patients seen by a practicing clinician, in his or her setting.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculating some values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Considering the first research study example above, in which (very importantly and for our purposes here), participants were recruited randomly and thus accurately reflect the true prevalence of a disease (in which case the predictive values will be accurate), calculating the predictive values is quite simple.</p>\n",
    "<p>The positive predictive (*P<sub>P</sub>*) value is the ratio between the true positive results and the sum of the true positive and false negative rates.  It expresses the probability that a patient will have the disease in the case of a a positive test result.</p>\n",
    "<p>The negative predictive (*P<sub>N</sub>*) value is the ratio between the true negative rate and the sum of the true negative and false negative rates.  It expresses the probability that a patient does not have the disease, in the case of a negative test result.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ {P}_{P} = \\frac {{P}_{T}}{{{P}_{T}} + {{F}_{P}}} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ {P}_{N} = \\frac {{N}_{T}}{{{N}_{T}} + {{F}_{N}}} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>So, for our first example above we have:</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "ppv = true_positive / (true_positive + false_positive) # Positive predicitive value\n",
    "npv = true_negative / (true_negative + false_negative)  # Negative predicitive value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The positive predicitive value is  9.090909090909092 %\n",
      "The negative predicitive value is  99.88901220865705 %\n"
     ]
    }
   ],
   "source": [
    "print('The positive predicitive value is ', (ppv * 100), '%')\n",
    "print('The negative predicitive value is ', (npv * 100), '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Just look how low the positive predicitive value is.  The chance of actually having the disease when the test results comes back positive is only 9%.  It should be clear that this is highly dependent on the incidince of the disease in our study population.  Here we only had 10 out of 1000 patients with the disease (a 1% incidence).</p>\n",
    "<p>If your research sample does not accurately refelct the population the predictive values can be highly skewed and not representative of the population at large.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>For instance, in our appendicitis study, with a WCC of less than 12 as a negative result, the negative predicitive value would only be:</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The negative predicitive value of a normal WCC (the chance of not having appendicitis given a negative result is 36.734693877551024 %\n"
     ]
    }
   ],
   "source": [
    "print('The negative predicitive value of a normal WCC (the chance of not having appendicitis given a negative result is', (tn * 100 / (tn + fn)), '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>That makes no sense!</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The positive predicitive value is : 88.77551020408163 %\n"
     ]
    }
   ],
   "source": [
    "print('The positive predicitive value is :', (tp * 100 / (tp + fp)), '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Correcting a positive predictive value (<i>P<sub>P</sub></i>)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>In order to convert positive predictive values to a population group with a different prevalence, first consider the pre-test odds (*O<sub>P</sub>*).  That is the ratio between having the disease and not having the disease in the group under consideration.  This is simply done by dividing the prevalence (*P<sub>v</sub>*) by one minus the prevalence, with the prevalence being a simple fraction of those with the disease (*d*) over the total number in the study (*n*).</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ {{P}_{v} = \\frac {d}{n}} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ {{O}_{P} = \\frac {{P}_{v}}{1 - {{P}_{v}}}} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>This result has to be multiplied by the likelihood ratio (*R<sub>L</sub>*).  This is the sensitivity divided by 1 minus the specificity.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ {{R}_{L} = \\frac {{S}_{N}}{1 - {{S}_{N}}}} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Now we get the post-test odds *O<sub>T</sub>* (multiplying the pre-test odds and the likelihood ratio):</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ {{O}_{O} = {{{O}_{P}} \\times {{R}_{L}}}} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Now we calculutate the new positive predictive value (*P<sub>N</sub>*), which is the post-test odds divided by one minus the post-test odds:</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ {{P}_{N} = {\\frac {{O}_{O}}{1 - {O}_{O}}}} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Remember that in our test example above the positive predictive value was 9.1%.  Let's do that again but consider a population in which the prevalence of appendicitis is only 0.5% (0.005).</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The positive predicitive value in a population with a prevalence of only 0.5% is:  3.5726018396846264 %\n"
     ]
    }
   ],
   "source": [
    "pre_test_odds = 0.005 / (1 - 0.005)\n",
    "likelihood_ratio = sens / (1 - sensitivity) # Remember sensitivity was our computer variable for sensitivity\n",
    "# in the test example above\n",
    "post_test_odds = pre_test_odds * likelihood_ratio\n",
    "new_PPV = post_test_odds / (1 + post_test_odds)\n",
    "\n",
    "print('The positive predicitive value in a population with a prevalence of only 0.5% is: ', new_PPV * 100, '%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Incidence, cumulative risk, and prevalence<br> Difference in rates and absolute difference in risk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>While major parts of these lectures are devoted to statistical analysis and comparisons between groups, it is important to consider and measure the absolute and relative frequencies with which parameters in a study present.  In this section I will commence with a description of the various methods of expressing the frequency of a parameter before moving on to comparing frequencies to each other in the form of differences.</p>\n",
    "<p>There are three main measures of frequency: incidence (incidence rate), cumulative risk (cumulative incidence) and finally prevalence.  There are two main differences, namely a difference in rates and a cumulative difference in risk.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The incidence rate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>The word <i>rate</i> suggests time involvement and that is indeed the case.  It measures the magnitude of occurrence over time and as such can only be used in studies that follow patients over time, such as cohort studies and randomized trials.</p>\n",
    "<p>The incidence (*R<sub>I</sub>*) is usually expressed in terms of events per person per year.  This would indicate a simple equation, with division of events by person-years, with events as *e<sub>n</sub>*, sample size as *n* and average follow-up as *t* with a bar (line) over it.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ {{R}_{I} = \\frac {{e}_{n}}{n \\times \\overline{t}}} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>So if five events occur in 100 patients, followed-up for on average one year would have an incidence rate of 5 per 100 patients per year.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cumulative risk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Cumulative risk is also known as cumulative incidence (*I<sub>C</sub>*).  It expresses the number of events as a percentage of a larger total.  This equation makes use of the number of events per person involved in the study.  It is less powerful than the incidence rate and due to the fact that time is not included in the calculation, it can change dramatically with shorting or extending of the study period.  Following a given sample size over a longer period, for instance, might results in more events occurring.  Likewise a shorter follow-up may falsely show a smaller cumulative risk, as not enough time might have passed for events to occur.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ {{I}_{C} = \\frac {{e}_{n}}{n}} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prevalence"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Prevalence gives the percentage of occurrence of an event at any point in time.  The calculation is a simple ratio of events and number involved (which, bar the context, is calculated in the same way as cumulative risk).  Multiplying the ratio by 100%, expresses the result in percentage.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Difference in rates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Calculating the difference in the rates between two events is the most accurate method of expressing outcome.  It is simply a subtraction of one rate from another.   If the incidence rate of a complication using drug A is 2.1 events per 100 people per year and the incidence rate of the same complication while taking drug B is 4.5 events per 100 people per year, then the difference is simply a subtraction of one value from the other, here resulting in a difference rate of 2.4 events per 100 people per year.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>The difference rate can also be used to calculate the very useful and often-quoted value, **number needed to treat**.  The number needed to treat is the reciprocal of the difference rate.  Depending on specific circumstances this value would represent the number of patients required to suffer an event.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Difference in cumulative risk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>As with the difference rate, the difference in cumulative risk is merely a subtraction of one cumulative risk from another, resulting in a value expressed in the same units as the original cumulative risks.  As an example, if the cumulative risk of a particular adverse event using drug A is 3.8% and the cumulative risk for the same adverse event suing drug B is 1%, then drug B a 2.8% decrease in the cumulative risk for that adverse event.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Rate, risk and odds ratios"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Rate ratios"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>In the prior sections that dealt with differences, I performed simple subtraction.  Ratios imply *division* and the rate ratio is the ratio of incidence rates, i.e. dividing one incidence rate by another.  These divisions result in dimensionless values that require interpretation.\n",
    "The *hazards ratio* is a special kind of rate ratio and looks at instantaneous incidence rates, calculated using Cox regression.</p>\n",
    "<p>The rate ratio can take one of three sets of values, the first being exactly *1.0*.  This would require the incidence rates of two events to be equal.  It is called the null value and is interpreted as no difference.  A value of less than *1.0* would imply a protective effect as the causative agent in the numerator lowers the rate and a value of more than *1.0* is seen as harmful and indicates an increase in risk.</p>\n",
    "<p>The rate ratio is usually expressed in a very particular manner in the literature and requires some explanation.  Suppose that drug A causes 2.1 events per 100 patients per year and drug B causes 4.5 events per 100 patients per year.  Dividing 2.1 by 4.5 results in 0.46.  Drug A can therefore be seen as having a protective effect, in as much as the value is less than *1.0*.  By subtracting 0.46 from *1.0* the resultant 0.54 can be expressed as a percentage, 54%.  This is the value that is usually expressed in the literature and would be used by stating that drug A reduces the rate of the particular adverse event by 54%.</p>\n",
    "<p>If the rate ratio was more than *1.0*, *1.0* is subtracted from the rate ratio value.  As an example, the rate ratio might be 2.4.  Subtracting 2.4 from *1.0* leaves 1.4.  This will be expressed as an increase in the incidence rate of an event by 140%.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Risk ratio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>The risk ratio can be expressed as the division of two cumulative risk values or two prevalence values.  As with the rate ratio the resultant value can be subtracted from *1.0* (or *1.0* subtracted from it) and multiplied by 100%, so as to express the value as a decrease or an increase in risk or prevalence.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Odds ratio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Odds is indeed a very peculiar term and must always be interpreted with caution and in view of how common or rare an event is.  Whereas rates and risk result in the probability of likelihood of occurrence (change), the odds are just that, the odds of an occurrence.</p>\n",
    "<p>It is useful for two reasons.  Firstly, when dealing with case-control studies, it is the only valid measure of relative risk.  Remember that in a case-control study the number of cases and controls are chosen beforehand.  This choice commands an absolute effect on rates and risks and is set by the study design.  Secondly, it is usually calculated by logistic regression (and then expressed as adjusted odds ratio as opposed to unadjusted odds ratio).  Using logistic regression can compensate for confounding.</p>\n",
    "<p>Odds are the ratio of the probability of an event divided by the probability of the event not occurring.  If the probability fo an event occuring is a half (0.5), then the probability of it not occuring is 0.5.  This gives you odds of 1.  Let's do another example.  If the probability of an event occuring is 0.75, then the probability of it not occuring is 0.25 (1.0 - 0.75), giving odds of 3 to 1 or 3.</p>\n",
    "<p>For rare events, arbitrarily chosen as equal to or less than 10%, risks and odds are close in value, but when events are more common, the odds are much higher than the risk.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### An example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>The concept of odds is best explained by an example.  Consider two parameters.  The first has three values (it is categorical): low, moderate, and high, and the second is expressed as the prevalence of that parameter with respect to patients in the specific group of parameter 1.</p>\n",
    "<p>Let's say that for the low group the prevalence of parameter 2 is 0.9%, for the moderate group it is 2.3% and for the high group it is 3.0%.</p>\n",
    "<p>the risk ratio in the low group is reference (0.9% / 0.9%) = *1.0*.  For the moderate group it is (2.3% / 0.9%) 2.56 and for the high group it is 3.33.</p>\n",
    "<p>Now, let's do the odds ratio.  remember it is the ratio of the probability of something occuring and it not occuring.  So for the low group it will be 0.9% divided by 99.1% (note I am expressing values in percent here), which is 0.0091.  This is our reference.  The odds of parameter 2 in the moderate group is 2.3% and therefore 97.7% against.  The odds ratio for parameter 2 in this groups is then 2.3% / 97.7%, which is 0.0235.  Dividing this by our reference 0.0091 gives an odds ratio of 2.58.  For the high group we have odds for parameter 2 of 3%, thus odds (probability) against of 97%, with an odds ratio of (3 % / 97%) 0.031.  This divided by our refernce (0.0091) gives an odds ratio of 3.41.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Again, we intepret this being above or below 1.00 and convert to percentage.  So, 2.58 becomes (2.58 - 1.00) 1.58, which is 158%.  The odds ratio of 3.41 becomes 241%.  That is an increase in odds of parameter 2 in the high group over the low group of 241%.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>The 10% rule applies here too.  Be very carful when reading the literature.  Note when parameters are expressed as risk and odds ratios.  One more example will make it crystal clear.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "data_odds_ratio = pd.read_csv('Odds_ratio.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Param_1</th>\n",
       "      <th>preval_param_2</th>\n",
       "      <th>risk_ratio</th>\n",
       "      <th>meaning_risk_ratio</th>\n",
       "      <th>odds_param_2</th>\n",
       "      <th>odds_ratio</th>\n",
       "      <th>meaning_odds_ratio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>low</td>\n",
       "      <td>0.2</td>\n",
       "      <td>ref</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.25</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>moderate</td>\n",
       "      <td>0.6</td>\n",
       "      <td>3</td>\n",
       "      <td>200</td>\n",
       "      <td>1.50</td>\n",
       "      <td>6</td>\n",
       "      <td>500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>high</td>\n",
       "      <td>0.9</td>\n",
       "      <td>4.5</td>\n",
       "      <td>350</td>\n",
       "      <td>9.00</td>\n",
       "      <td>36</td>\n",
       "      <td>3500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Param_1  preval_param_2 risk_ratio  meaning_risk_ratio  odds_param_2  \\\n",
       "0       low             0.2        ref                 NaN          0.25   \n",
       "1  moderate             0.6          3                 200          1.50   \n",
       "2      high             0.9        4.5                 350          9.00   \n",
       "\n",
       "   odds_ratio  meaning_odds_ratio  \n",
       "0         NaN                 NaN  \n",
       "1           6                 500  \n",
       "2          36                3500  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_odds_ratio.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>I've imported another *csv* spreadsheet file called *Odds_ratio.csv* and show the only three rows.  Here we have a much higher prevalence of an event (parameter 2) in the moderate and high groups.  Note that for the high group there is a 350% increase ratio risk, but a 3500% odds ratio increase.  There are examples in the literature where risk and odds values are used intermittently and without reading carefully, your impression of results can be represented in a verty skew manner.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>For a prevalence of about 10% risk and odds ratios are very similar.  With higher and higher prevalence, these become very different!</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}