GEO4310-Lecture06.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chapter6\n",
    "\n",
    "### Confidence interval and hypothesis testing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The confidence intervals and hypothesis testing are used for parameter estimation, forecasting and\n",
    "hypothesis testing, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1. Definition (Confidence interval)\n",
    "\n",
    "   A confidence interval is a range of values that has a specified probability of containing\n",
    "    the parameter being estimated. This statement may be written as \n",
    "    \n",
    "$$\n",
    "\\begin{align}\n",
    "P(L < \\theta < U) = 1-\\alpha \\ \\ \\ \\ \\ \\ \\ (1)\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "Where:\n",
    "- $L$ and $U$ are lower and upper confidence limits; $L,U$ are random variables.\n",
    "- [$L, U$] is the confidence interval (C.I)\n",
    "- $\\alpha$ is the significance level; this is a probability and the values usually used are:\n",
    "  $1\\%$ ($\\alpha=0.01$), $5\\%$ ($\\alpha=0.05$), or $10\\%$ ($\\alpha=0.1$).\n",
    "- $1- \\alpha$ is the confidence level, a probability, this is a probability and the values usually\n",
    "    used are: $99\\% \\, (1- \\alpha=0.99)$, $95\\% \\, (1 - \\alpha=0.95)$, or $90\\% \\, (1 - \\alpha=0.90)$.\n",
    "- $\\theta$ is the parameter; this is a constant.\n",
    "\n",
    "Equation (1) reads as: the probability that the interval $L$ to $U$ contains $\\theta$ is $1-\\alpha$.\n",
    "It is NOT correct to read: the probability that $\\theta$ is between $L$ and $U$ is $1-\\alpha$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "#### 2. Calculation of confidence intervals\n",
    "\n",
    "Let the probability that the interval $(v_1,v_2)$ includes the variable $V$ be $1-\\alpha$:\n",
    "\n",
    "$$P(v_1 < V < v_2) = 1-\\alpha \\ \\ \\ \\ \\ \\ \\ (2) $$\n",
    "\n",
    "#### C.I. for the mean of a Normal distribution with unknown variance (n small)\n",
    "\n",
    "Assume that we select randomly a small sample ($n$=small) from a population, and calculate the\n",
    "mean of the sample ($\\bar{x}$). We want to know if the calculated mean is representative of the population mean ($\\mu$). Therefore we set the probability of the interval $(\\bar{x} - \\Delta,\\bar{x} + \\Delta)$ including $\\mu$, and then calculate this interval. $\\Delta$ is a number (name randomly chosen here) depending on the assumed distribution, siginificance level and sample's standard deviation.\n",
    "\n",
    "$$ \\text{If} \\ \\ x_i \\sim N(\\mu,s^2) \\ \\ \\text{then} \\ \\ \\bar{x} \\sim N(\\mu,s_{\\bar{x}}^2) = N(\\mu,s^2/n)$$\n",
    "\n",
    "From (2), let \n",
    "\n",
    "$$V = \\frac{(\\bar{x}-\\mu)}{s_{\\bar{x}}}$$\n",
    "\n",
    "Then $V$ has a **t distribution** with $n-1$ degrees of freedom.\n",
    "\n",
    "$$\\begin{align}\n",
    "    &P(v_1 < \\frac{\\bar{x}-\\mu}{s_{\\bar{x}}} < v_2) = 1-\\alpha \\ \\ \\ \\ (3) \\\\\n",
    "    \\Rightarrow &P(t_{\\frac{\\alpha}{2},n-1} < \\frac{\\bar{x}-\\mu}{s_{\\bar{x}}} < t_{1-\\frac{\\alpha}{2},n-1}) = 1-\\alpha \\\\\n",
    "    \\Rightarrow &P(-t_{1-\\frac{\\alpha}{2},n-1} < \\frac{\\bar{x}-\\mu}{s_{\\bar{x}}} < t_{1-\\frac{\\alpha}{2},n-1}) = 1-\\alpha \\\\\n",
    "    \\Rightarrow &P(\\bar{x}-t_{1-\\frac{\\alpha}{2},n-1}s_{\\bar{x}} < \\mu < \\bar{x}+t_{1-\\frac{\\alpha}{2},n-1}s_{\\bar{x}}) = 1-\\alpha\n",
    "  \\end{align}\n",
    "$$\n",
    "\n",
    "This latter equation is on the form of (1), so the confidence limits are:\n",
    "\n",
    "$$\\begin{align}\n",
    "    l &= \\bar{x}-t_{1-\\frac{\\alpha}{2},n-1}s_{\\bar{x}} \\\\\n",
    "    u &= \\bar{x}+t_{1-\\frac{\\alpha}{2},n-1}s_{\\bar{x}} \n",
    "  \\end{align}\n",
    "$$\n",
    "\n",
    "where $t_{1-\\frac{\\alpha}{2},n-1}$ can be read from t distribution table (note that this is the $\\Delta$ mentioned above)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### C.I. for the mean of a Normal distribution with known variance (n large)\n",
    "\n",
    "If the population variance is known or the sample size is large, i.e. $x_i \\sim N(\\mu,\\sigma^2)$, then\n",
    "\n",
    "$$\\bar{x} \\sim N(\\mu,\\sigma_{\\bar{x}}^2) = N(\\mu,\\frac{\\sigma^2}{n})$$\n",
    "\n",
    "The pivotal quantity in (3) becomes $$\\frac{(\\bar{x}-\\mu)}{\\sigma_{\\bar{x}}},$$ which  has a **standard normal** distribution. The confidence limits become:\n",
    "\n",
    "$$\\begin{align}\n",
    "    l &= \\bar{x}-z_{1-\\frac{\\alpha}{2}}\\sigma_{\\bar{x}} \\\\\n",
    "    u &= \\bar{x}+z_{1-\\frac{\\alpha}{2}}\\sigma_{\\bar{x}} \n",
    "  \\end{align}\n",
    "$$\n",
    "\n",
    "where $z_{1-\\frac{\\alpha}{2}}$ can be read from a standard normal distribution table."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "###### Example 6.1\n",
    "\n",
    "From $99$ years annual discharge data it was calculated that the mean is 66.54 and the standard\n",
    "deviation is $22.32$. (i.e. $x = 66.54$, $s_x = 22.32$)\n",
    "\n",
    "Calculate the $95\\%$ confidence interval for the mean assuming that:\n",
    "\n",
    "(a) the true variance is unknown\n",
    "\n",
    "(b) the true variance is known as $(22.32)^2$\n",
    "\n",
    "###### Solution\n",
    "\n",
    "(a) $s_{\\bar{x}} = \\frac{s_x}{\\sqrt{n}} = \\frac{22.32}{\\sqrt{99}} = 2.243$\n",
    "\n",
    "From the t table we get $t_{1 − \\alpha/2 , n−1} = t_{0.975,98} = 1.99$\n",
    "\n",
    "From equation 5:\n",
    "$$\\begin{align}\n",
    "    l &= \\bar{x} − t_{1 − \\alpha/2,n−1}s_x = 66.54 − 1.99(2.243) = 62.076 \\\\\n",
    "    u &= \\bar{x} + t_{1 − \\alpha/2,n−1}s_x = 66.54 + 1.99(2.243) = 71.004\n",
    " \\end{align}\n",
    "$$\n",
    "\n",
    "Thus, we say we are $95\\%$ confident that interval $62.076$ to $71.004$ contains the true population\n",
    "mean.\n",
    "\n",
    "(b) $\\sigma_{\\bar{x}} = \\frac{s_x}{\\sqrt{n}} = \\frac{22.32}{\\sqrt{99}} = 2.243$\n",
    "\n",
    "From the Z table we $z_{1-\\alpha/2} = z_{0.975} = 1.96$\n",
    "\n",
    "From equation 6:\n",
    "$$\\begin{align}\n",
    "    l &= \\bar{x} − z_{1-\\alpha/2}\\sigma_{\\bar{x}} = 66.54 − 1.96(2.243) = 62.144 \\\\\n",
    "    u &= \\bar{x} + z_{1-\\alpha/2}\\sigma_{\\bar{x}} = 66.54 + 1.96(2.243) = 70.911\n",
    " \\end{align}\n",
    "$$\n",
    "\n",
    "Thus, we say we are $95\\%$ confident that interval $62.144$ to $70.911$ contains the true population\n",
    "mean.\n",
    "\n",
    "Question: why using Z distribution is confidence interval is smaller than using t distribution?\n",
    "\n",
    "INSERT FIGURE 6.1!!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### C.I. for the variance of a Normal distribution\n",
    "\n",
    "We know that if $x_i \\sim N(\\mu,\\sigma^2)$ then the quantity $\\sum_{i=1}^n \\frac{(x_i - \\bar{x})^2}{\\sigma^2}$ has a Chi-square distribution (Lecture 4.9). Then:\n",
    "\n",
    "$$\\sum_{i=1}^n \\frac{(x_i - \\bar{x})^2}{\\sigma_x^2} = \\frac{n-1}{\\sigma_x^2} \\sum_{i=1}^n\\frac{(x_i - \\bar{x})^2}{n-1} = (n-1) \\frac{s_x^2}{\\sigma_x^2}$$\n",
    "\n",
    "Letting this quantity equal V in (3) results in:\n",
    "\n",
    "$$P(v_1 < \\frac{(n-1)s_x^2}{\\sigma_x^2} < v_2) = 1-\\alpha $$\n",
    "\n",
    "Since we have Chi-square distribution, choose $v_1$ equal $\\chi_{\\alpha/2,n−1}^2$ and $v_2$ equal $\\chi_{1−\\alpha/2,n-1}^2$. Then:\n",
    "\n",
    "$$P(\\chi_{\\alpha/2,n−1}^2 < \\frac{(n-1)s_x^2}{\\sigma_x^2} < \\chi_{1−\\alpha/2,n-1}^2) = 1-\\alpha  $$\n",
    "\n",
    "$$\\Rightarrow P(\\frac{(n-1)s_x^2}{\\chi_{1−\\alpha/2,n-1}^2} < \\sigma_x^2 < \\frac{(n-1)s_x^2}{\\chi_{\\alpha/2,n−1}^2}) = 1-\\alpha $$\n",
    "\n",
    "which is in form of (1) and the confidence limits on $\\sigma^2$ are:\n",
    "$$\\begin{align}\n",
    "    l &= \\frac{(n-1)s_x^2}{\\chi_{1−\\alpha/2,n-1}^2}  \\\\\n",
    "    u &= \\frac{(n-1)s_x^2}{\\chi_{\\alpha/2,n−1}^2}\n",
    " \\end{align}\n",
    "$$\n",
    "\n",
    "INSERT FIGURE 6.2!!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 6.2\n",
    "\n",
    "Determine the $90\\%$ confidence interval on the variance for the situation described in example\n",
    "$6.1$.\n",
    "\n",
    "#### Solution\n",
    "\n",
    "$\\chi_{\\alpha/2,n-1}^2 = \\chi_{0.05,98}^2 = 76.1$ (Chi-square distribution table, appendix)\n",
    "\n",
    "$\\chi_{1-\\alpha/2,n-1}^2 = \\chi_{0.95,98}^2 = 122.1$ ( Chi-square distribution table, appendix )\n",
    "\n",
    "$l = \\frac{(n-1)s_x^2}{\\chi_{1−\\alpha/2,n-1}^2} = \\frac{98(22.32)^2}{122.1} = 400$\n",
    "\n",
    "$u = \\frac{(n-1)s_x^2}{\\chi_{\\alpha/2,n−1}^2} = \\frac{98(22.32)^2}{76.1} = 642$\n",
    "\n",
    "The $90\\%$ confidence intervals for the variance are found to be 400 to 642. \n",
    "\n",
    "Or: The $90\\%$ confidence intervals for the standard deviation are found (by taking the square roots of the above limits) to be $20$ to $25$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### One-sided confidence intervals\n",
    "\n",
    "Sometimes one is only interested in an interval estimate on one side of a parameter. Then\n",
    "equation (1) becomes:\n",
    "\n",
    "$$P(L<\\theta) = 1-\\alpha $$\n",
    "\n",
    "Following the same procedure we get,\n",
    "\n",
    "$$l = \\bar{x}-t_{1-\\alpha,n-1}s_{\\bar{x}}$$\n",
    "\n",
    "The analogous results would hold for any one-sided, lower or upper confidence limit."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Confidence intervals for proportions/probabilities\n",
    "\n",
    "Sometimes we are interested to know what fraction of the population has a certain characteristic.\n",
    "Since we cannot examine the whole population, we choose a random sample (sample size$=n$), and\n",
    "examine what proportion ($\\hat{p}$) of this sample bares this characteristic. The confidence interval\n",
    "population proportion is calculated as:\n",
    "\n",
    "$$P = \\hat{p} \\pm z_{1-\\frac{\\alpha}{2}}\\sqrt{\\frac{\\hat{p}(1-\\hat{p})}{n}} $$\n",
    "\n",
    "Note that when the sample size increases as $P$ approaches $\\hat{p}$, which means that the sample estimate\n",
    "becomes more reliable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 6.3\n",
    "\n",
    "$500$ volunteers participated in a drug test, $8$ of which reported a particular side effect.\n",
    "\n",
    "a) Find the $95\\%$ confidence interval for the population of all people that are going to use this drug\n",
    "to have the same side effect.\n",
    "\n",
    "b) Find the number of volunteers needed, so that we are $95\\%$ confident that the **estimate for the\n",
    "population proportion** will not exceed the **estimate for the sample proportion** with more than $0.5\\%$\n",
    "\n",
    "#### Solution\n",
    "\n",
    "$\\hat{p} = \\frac{8}{500} = 0.016$\n",
    "\n",
    "$z_{1-\\frac{\\alpha}{2}} = z_{1-\\frac{1-0.95}{2}} = z_{0.975} = 1.96$ (from normal distribution table)\n",
    "\n",
    "a) $n = 500$\n",
    "\n",
    "$$P = \\hat{p} \\pm z_{1-\\frac{\\alpha}{2}}\\sqrt{\\frac{\\hat{p}(1-\\hat{p})}{n}} = 0.016 \\pm 1.96\\sqrt{\\frac{0.016(1−0.016)}{500}}$$\n",
    "\n",
    "$$\\Rightarrow P = (0.005, 0.027)$$\n",
    "\n",
    "We are $95\\%$ confident that the drug will have side effects on $0.5$ to $2.7$ percent of the whole\n",
    "population that will use it.\n",
    "\n",
    "b) \n",
    "\n",
    "$$z_{1-\\frac{\\alpha}{2}}\\sqrt{\\frac{\\hat{p}(1-\\hat{p})}{n}} \\leq 0.005 $$\n",
    "\n",
    "$$n \\geq \\left(\\frac{z_{0.975}}{0.005}\\right)^2 \\cdot 0.016 \\cdot (1-0.016) = 2420 $$\n",
    "\n",
    "$$\\Rightarrow P = 0.016 \\pm 0.005 = (0.011,0.021) $$\n",
    "\n",
    "After testing the drug on at least $2420$ people, we will be $95\\%$ confident that the drug will have side\n",
    "effects on not more than $2.1\\%$ of the whole population that will use it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3. Summary of confidence interval\n",
    "\n",
    "- **HOW CONFIDENCE INTERVALS BEHAVE**\n",
    "    - Ideal situation – high confidence and small margin of error\n",
    "    - Margin of error$(E) = \\frac{z_{\\alpha/2}\\sigma}{\\sqrt{n}}$ \n",
    "    - The smaller the margin of error, the more precise our estimation of $\\mu$\n",
    "    - The margin error equation can also be used to estimate sample size in order to achieve a given margin of error:\n",
    "    \n",
    "    $$n = \\left(\\frac{z_{\\alpha/2}\\sigma}{E}\\right)^2$$\n",
    "\n",
    "- **Properties of error**\n",
    "    - **Error increases with smaller sample size:** \n",
    "        For any confidence level, large samples reduce the margin of error (sample size)\n",
    "    - **Error increases with larger standard Deviation:** \n",
    "        As variation among the individuals in the population increases, so does the error of our\n",
    "        estimate (data behavior)\n",
    "    - **Error increases with larger z values:**\n",
    "        Tradeoff between confidence level and margin of error (significance level)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Insert figures and Tables from page 72-75!!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hypothesis testing\n",
    "\n",
    "(1) **Definition:** Hypothesis tests are procedures for making rational decisions about the reality of\n",
    "effects.\n",
    "\n",
    "- Statistical decision – decision made about the population on the basis of sample information\n",
    "- The *null hypothesis*, $H_0$ represents a theory that has been put forward, either because it is believed to be true or because it is to be used as a basis for argument, but has not been proved.\n",
    "- The *alternative hypothesis*, $H_a$ , is a statement of what a statistical hypothesis test is set up to establish.\n",
    "- The *significance level* (fixed level measured by $\\alpha$) of a statistical hypothesis test is a fixed probability of wrongly rejecting the *null hypothesis* $H_0$ , if it is in fact true.\n",
    "- A *test statistic* is a quantity calculated from our sample of data. Its value is used to decide whether or not the *null hypothesis* should be rejected in our hypothesis test. The choice of a *test statistic* will depend on the assumed probability model and the hypotheses under question.\n",
    "- The *critical region* CR, or *rejection region* RR, is a set of values of the *test statistic* for which the *null hypothesis* is rejected in a hypothesis test.\n",
    "- The probability value (p-value) is the probability of wrongly rejecting the *null hypothesis* if it is in fact true. It is similar as $\\alpha$ but smaller, $\\alpha= 0.05$, this would be reported as '$p < 0.05$'. Reject the *null hypothesis* if the p-value is less than the *level of significance*. You will fail to reject the *null hypothesis* if the p-value is greater than or equal to the *level of significance*. The P value is the **observed significance level** (or **P value**)--**the smallest fixed level at which the null hypothesis can be rejected**. If your personal fixed level is greater than or equal to the P value, you would reject the *null hypothesis*. If your personal fixed level is less than to the P value, you would fail to reject the *null hypothesis*. For example, if a P value is $0.027$, the results are significant for all fixed levels greater than $0.027$ (such as $0.05$) and not significant for all fixed levels less than $0.027$ (such as $0.01$). A person who uses the $0.05$ level would reject the *null hypothesis* while a person who uses the $0.01$ level would fail to reject it.\n",
    "\n",
    "(2) Procedure - steps\n",
    "\n",
    "1. formulate the null hypothesis, $H_0$, to be tested\n",
    "2. formulate the alternative hypothesis, $H_a$\n",
    "3. determine an test statistic\n",
    "4. determine the distribution of the test statistic\n",
    "5. define the rejection region or critical region of the test statistic\n",
    "6. calculate the test statistic\n",
    "7. determine if the calculated value of the test statistic falls in the rejection region of the distribution of the test statistic\n",
    "\n",
    "(3) Examples in test\n",
    "\n",
    "- **one sample two tailed**\n",
    "\n",
    "(a) Test if the mean of a normal distribution is $\\mu_0$, knowing that the variance is $\\sigma^2$ (Z-test).\n",
    "\n",
    "#### Solution\n",
    "\n",
    "Hypothesis: $H_0\\text{:} \\ \\ \\mu = \\mu_0 \\ \\ \\text{against} \\ \\ H_a\\text{:} \\ \\  \\mu \\neq \\mu_0 $\n",
    "\n",
    "Get a sample and calculate the mean and standard deviation $(\\bar{x},\\sigma_x^2)$\n",
    "\n",
    "Test statistic: $$Z = \\frac{(\\bar{x}-\\mu_0)}{\\sigma_{\\bar{x}}}$$\n",
    "\n",
    "$H_0$ is rejected if $$|Z| = \\left| \\frac{(\\bar{x}-\\mu_0)}{\\sigma_{\\bar{x}}} \\right| > z_{1-\\alpha/2}  $$\n",
    "\n",
    "Equivalent to say if $\\bar{x}$ is outside of the confidence interval: \n",
    "\n",
    "$$ \\bar{x} < \\mu_0 - z_{1-\\alpha/2}\\sigma_{\\bar{x}}$$\n",
    "\n",
    "or\n",
    "\n",
    "$$\\bar{x} > \\mu_0 + z_{1-\\alpha/2}\\sigma_{\\bar{x}} $$\n",
    "\n",
    "(b) Test if the mean of a normal distribution is $\\mu_0$, with unknown variance (**This is the famous T-test in the lecture**)\n",
    "\n",
    "$$H_0\\text{:} \\ \\ \\mu = \\mu_0, \\ \\  H_a\\text{:} \\ \\ \\mu \\neq \\mu_0 $$\n",
    "\n",
    "$$t = \\frac{(\\bar{x}-\\mu_0)}{s_{\\bar{x}}} $$\n",
    "\n",
    "$H_0$ is rejected if: \n",
    "\n",
    "$$|t| = \\left| \\frac{(\\bar{x}-\\mu_0)}{s_{\\bar{x}}} \\right| > t_{1-\\alpha/2} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 6.4\n",
    "\n",
    "The annual runoff for a station for the period $1953$-$1970$ has been calculated as $14.65$cm and the\n",
    "standard deviation $4.75$cm. Test the hypothesis that the true mean annual discharge is $16.5$cm.\n",
    "\n",
    "#### Solution \n",
    "\n",
    "Assume normal distribution and unknown variance since $n$ is small.\n",
    "\n",
    "$$H_0\\text{:} \\ \\ \\mu = 16.5; \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ H_a\\text{:} \\ \\ \\mu \\neq 16.5 $$\n",
    "\n",
    "$$t = \\frac{(\\bar{x}-\\mu)}{s_{\\bar{x}}} = \\frac{(14.65 − 16.5)}{4.75/\\sqrt{18}} = -1.65 $$\n",
    "\n",
    "Using $\\alpha = 5\\%$ and $t_{1 − \\alpha/2,n-1} = t_{0.975,17} = 2.11$ (from appendix, table A13). Since $|t| < t_{1 − \\alpha/2,n-1} $ we do not reject the hypothesis that the true mean might be $16.5$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(4) **Errors and power in hypothesis testing**\n",
    "\n",
    "| Decision     | Hypothesis true |Hypothesis false|\n",
    "| ------------- |:-------------:| -----:|\n",
    "| Reject      |Type I error | No error |\n",
    "| Don't reject      | No error      |   Type II error |\n",
    "\n",
    "- Type I error = significance level, $\\alpha$\n",
    "- Type II error is denoted by $\\beta$. The exact probability of a type II error is generally unknown. A type II error is frequently due to sample sizes being too small.\n",
    "- Power of hypothesis test - measures the test's ability to reject the null hypothesis when it is actually false - that is, to make a correct decision. In other words, the power of a hypothesis test is the probability of not committing a __type II error__. It is calculated by subtracting the probability of a type II error from $1$, usually expressed as:\n",
    "\n",
    "$$\\text{Power} = 1-P(\\text{type II error}) = 1-\\beta $$\n",
    "\n",
    "**The maximum power a test can have is $1$, the minimum is $0$**.\n",
    "\n",
    "**INSERT FIGURE 6.3!!!**, **INSERT FIGURE 6.4!!!**, **Insert something in the style of page 79**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Factors Affecting Type II Error\n",
    "\n",
    "- True value of population parameter\n",
    "    - $\\beta$ Increases when the difference between hypothesized parameter and its true value decrease\n",
    "- Significance level\n",
    "    - $\\beta$ increases when $\\alpha$ decreases\n",
    "- Population standard deviation\n",
    "    - $\\beta$ increases when $\\sigma$ decreases\n",
    "- Sample size\n",
    "    - $\\beta$ increases when $n$ decreases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 6.5\n",
    "\n",
    "Assume a single observation is selected from a normal distribution with mean $\\mu=7$ and variance $\\sigma_0^2 =9$. It is hypothesized that $\\mu=\\mu_0=5$. If the test is conducted at the $10\\%$ significance level, what is $\\beta$?\n",
    "\n",
    "#### Solution\n",
    "\n",
    "$\\alpha/2 = 0.05$ , which corresponds to $z_{1-\\frac{\\alpha}{2}} = 1.645$\n",
    "\n",
    "$(X_u - \\mu_0)/s = z_{1-\\alpha/2}$, where  $X_u$ is the boundary of the upper critical region\n",
    "\n",
    "$(X_u-5)/3 = 1.645$\n",
    "\n",
    "$X_u = 9.935$\n",
    "\n",
    "$A_u=$ the area of a normal distribution with mean of $7$ and variance $9$ to the left of $9.935$.\n",
    "\n",
    "The standardized variate corresponding to $X_u = 9.935$ is:\n",
    "\n",
    "$$z_\\mu = (9.035 – 7)/3 = 0.978 $$\n",
    "\n",
    "The area to the left of $z_u = 0.978$ from a standard normal distribution is $0.8365$.\n",
    "\n",
    "Similarly, if $X_l$ is the boundary of the lower critical region, we have:\n",
    "\n",
    "$$(x_l-5)/3=-1.645=0.0645 $$\n",
    "\n",
    "$A_l$ is the area of a normal distribution with mean $7$ and variance $9$ to the left of $0.0645$.\n",
    "\n",
    "$$z_l =(0.0645 -7)/3= -2.31$$\n",
    "\n",
    "$$A_l =0.0104$$\n",
    "\n",
    "Now, $\\beta = A_u – A_l = 0.8365 – 0.0104 = 0.8261$.\n",
    "\n",
    "Thus, the probability of accepting the hypothesis that $\\mu=5$ when in fact $\\mu=7$ is $0.8261$ when $\\alpha$ is $0.1$. The probability of Type II error is $0.8261$.\n",
    "\n",
    "**INSERT FIGURE 6.5**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(5) **More examples of hypothesis testing**\n",
    "\n",
    "- **One sample, one tail. Test on mean, normal distribution,**\n",
    "\n",
    "(a) Known variance:\n",
    "\n",
    "$$H_0\\text{:} \\ \\ \\mu = \\mu_1, \\ \\ H_a\\text{:} \\ \\ \\mu = \\mu_2 $$\n",
    "\n",
    "or\n",
    "\n",
    "$$H_0\\text{:} \\ \\ \\mu = \\mu_0, \\ \\ H_a\\text{:} \\ \\ \\mu > \\mu_0 \\ \\ \\text{or} \\ \\ \\mu < \\mu_0 $$\n",
    "\n",
    "$$Z = \\frac{\\bar{x}-\\mu_1}{\\sigma_{\\bar{x}}} $$\n",
    "\n",
    "If $\\mu_1 > \\mu_2$ , then $H_0$ is rejected if:\n",
    "\n",
    "$$\\bar{x} \\geq \\mu_1 + z_{1 − \\alpha}\\sigma_\\bar{x} $$\n",
    "\n",
    "(b) Unknown variance:\n",
    "\n",
    "$$H_0\\text{:} \\ \\ \\mu = \\mu_1, \\ \\ H_a\\text{:} \\ \\ \\mu = \\mu_2 $$\n",
    "\n",
    "or\n",
    "\n",
    "$$H_0\\text{:} \\ \\ \\mu = \\mu_0, \\ \\ H_a\\text{:} \\ \\ \\mu > \\mu_0 \\ \\ \\text{or} \\ \\ \\mu < \\mu_0 $$\n",
    "\n",
    "Test statistic:\n",
    "\n",
    "$$t = \\frac{(\\bar{x}-\\mu_1)}{s_{\\bar{x}}} $$\n",
    "\n",
    "If $\\mu_1 > \\mu_2$ , then $H_0$ is rejected if:\n",
    "\n",
    "$$\\bar{x} \\leq μ_1 − t_{1 − \\alpha, n−1}s_\\bar{x}$$\n",
    "\n",
    "If $\\mu_1 < \\mu_2$ , then $H_0$ is rejected if:\n",
    "\n",
    "$$\\bar{x} \\geq μ_1 + t_{1 − \\alpha, n−1}s_\\bar{x}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 6.6\n",
    "\n",
    "The annual runoff for a station for the period $1953$-$1970$ has been calculated as $14.65$cm and the\n",
    "standard deviation $4.75$cm. Can the mean annual discharge be $16.5$ in reality? (i.e. can the\n",
    "population mean be $16.5$ instead of $14.65$ as sample shows).\n",
    "\n",
    "#### Solution\n",
    "\n",
    "$\\alpha = 5\\%$, $n=18$\n",
    "\n",
    "Assume normal distribution and unknown variance since $n$ is small.\n",
    "\n",
    "$$H_0\\text{:} \\ \\ \\mu = \\mu_1 = 16.5; \\ \\ \\ \\ \\ \\ \\ \\ \\ H_\\alpha\\text{:} \\ \\ \\mu = \\mu_2 = 14.65; \\ \\ \\mu_1>\\mu_2$$\n",
    "\n",
    "$$t = \\frac{(\\bar{x}-\\mu)}{s_{\\bar{x}}} = \\frac{(14.65 − 16.5)}{4.75/\\sqrt{18}} = -1.65$$\n",
    "\n",
    "$t_{1−\\alpha,n−1} = t_{0.95,17} = −1.74$ (from appendix, table A13):\n",
    "\n",
    "$$\\mu_1 - t_{1−\\alpha,n−1}\\cdot s_\\bar{x} = 16.5 - 1.74\\cdot \\frac{4.75}{\\sqrt{18}} = 14.55$$\n",
    "\n",
    "$$\\Rightarrow \\bar{x} > \\mu_1 - t_{1−\\alpha,n−1}\\cdot s_\\bar{x} $$\n",
    "\n",
    "Thus, we do not reject the hypothesis that the population mean may be $16.5$\n",
    "\n",
    "Compare with the previous example, here we test if the mean $16.5$ instead of $14.65$, in the\n",
    "previous example, we test if the mean is $16.5$ nothing else.."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: level of significance and the rejection region\n",
    "\n",
    "**INSERT FANCY FIGURE AT PAGE 83**\n",
    "\n",
    "- Two-sample, two-tail: if the means of two normal distributions are significantly different.\n",
    "\n",
    "(a) Known variances:\n",
    "\n",
    "$$x_1 \\sim N(\\mu_1,\\sigma_1^2) \\ \\ \\ \\ x_2 \\sim N(\\mu_2,\\sigma_2^2)$$\n",
    "\n",
    "$$H_0\\text{:} \\ \\ \\mu_1-\\mu_2 = \\delta, \\ \\ H_a\\text{:} \\ \\ \\mu_1-\\mu_2 \\neq \\delta; \\ \\ \\delta = 0$$\n",
    "\n",
    "Test statistic:\n",
    "\n",
    "$$ z=\\frac{(\\bar{x}_1-\\bar{x}_2-\\delta)}{\\sqrt{(\\sigma_1^2/n_1+\\sigma_2^2/n_2)}}$$\n",
    "\n",
    "$H_0$ rejected if: \n",
    "\n",
    "$$ |z| > z_{1-\\alpha/2} $$\n",
    "\n",
    "(b) Unknown variances:\n",
    "\n",
    "Test statistic:\n",
    "\n",
    "$$ z=\\frac{(\\bar{x}_1-\\bar{x}_2-\\delta)}{\\sqrt{(s_1^2/n_1+s_2^2/n_2)}}$$\n",
    "\n",
    "$H_0$ rejected if: \n",
    "\n",
    "$$|t| > \\frac{w_1t_1+w_2t_2}{w_1+w_2}$$\n",
    "\n",
    "where:\n",
    "\n",
    "$$w_1 = \\frac{s_1^2}{n} $$\n",
    "\n",
    "$$w_2 = \\frac{s_2^2}{n} $$\n",
    "\n",
    "$$t_1 = t_{1-\\alpha/2,n_1-1}$$\n",
    "\n",
    "$$t_2 = t_{1-\\alpha/2,n_2-1}$$\n",
    "\n",
    "- One sample, two tail test for the variance of normal distribution\n",
    "\n",
    "$$H_0\\text{:} \\ \\ \\sigma^2 = \\sigma_0^2; \\ \\ \\ \\ \\ \\ H_a\\text{:} \\ \\ \\sigma^2 \\neq \\sigma_0^2$$\n",
    "\n",
    "Test statistic:\n",
    "\n",
    "$$\\chi_c^2 = \\sum_{i=1}^n \\frac{(x_i-\\bar{x})^2}{\\sigma_0^2} = \\frac{(n-1)S^2}{\\sigma_0^2}$$\n",
    "\n",
    "$H_0$ rejected if: \n",
    "\n",
    "$$\\chi_c^2 > \\chi_{1-\\alpha/2,n-1}^2$$\n",
    "\n",
    "or\n",
    "\n",
    "$$\\chi_c^2 < \\chi_{\\alpha/2,n-1}^2$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 6.7\n",
    "\n",
    "For the example 6.6 test the hypothesis that the variance is 36.\n",
    "\n",
    "#### Solution\n",
    "\n",
    "$$H_0\\text{:}\\ \\ \\sigma^2 = \\sigma_0^2 = 36; \\ \\ \\ \\ \\ \\ H_a\\text{:}\\ \\ \\sigma^2 \\neq \\sigma_0^2$$\n",
    "\n",
    "Then\n",
    "\n",
    "$$\\chi_c^2 = \\sum_{i=1}^n \\frac{(x_i-\\bar{x})^2}{\\sigma_0^2} = \\frac{(n-1)S^2}{\\sigma_0^2} = \\frac{17(4.75)^2}{36} = 10.65$$\n",
    "\n",
    "From a Chi-square table:\n",
    "\n",
    "$$\\chi_{\\alpha/2,n-1}^2 = \\chi_{0.025,17}^2 = 7.6$$\n",
    "\n",
    "$$\\chi_{1-\\alpha/2,n-1} = \\chi_{0.975,17}^2 = 30.2$$\n",
    "\n",
    "Since $10.65$ is neither smaller than $7.6$ nor bigger than $30.2$, $H_0$ is not rejected."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The F-test\n",
    "\n",
    "Testing whether two samples come from the populations with the same variance\n",
    "Used to test a null hypothesis that the variance of one population is equal to that of a second population $\\sigma_{\\text{pop}1}^2 = \\sigma_{\\text{pop}2}^2$\n",
    "\n",
    "$$H_0\\text{:} \\ \\ \\sigma_1^2 = \\sigma_2^2; \\ \\ \\ \\ \\ \\ H_a\\text{:} \\ \\ \\sigma_1^2 \\neq \\sigma_2^2$$\n",
    "\n",
    "The F statistic is a simple ratio of the two unbiased estimator of population variance:\n",
    "\n",
    "$$F_c = \\frac{s_1^2}{s_2^2}$$\n",
    "\n",
    "where F is distributed as F distribution with $n_1-1$ and $n_2-1$ degrees of freedom, and $s_1^2 > s_2^2$.\n",
    "\n",
    "The F value will be close to one if the variances of two populations are equal or close.\n",
    "\n",
    "$H_0$ is rejected if:\n",
    "\n",
    "$$F_c > F_{1-\\alpha,n_1-1,n_2-1} \\ \\ \\text{(table A15)}$$\n",
    "\n",
    "- Test for equality of variances from several Normal distributions\n",
    "\n",
    "To test the $H_0\\text{:} \\ \\ \\sigma_1^2 = \\sigma_2^2 = \\cdots = \\sigma_k^2$ for $k$ independent samples each from a normal population with mean $\\mu_i$ and variance $\\sigma_i^2$, it is first necessary to calculate the $k$ sample variances $S_i^2$. The quantity $Q/h$ is approximately distributed as a chi-square distribution with $k-1$ degrees of freedom, where: \n",
    "\n",
    "$$Q = \\sum_{i=1}^k (n_i-1)\\cdot ln\\left[\\sum_{i=1}^k(n_i-1)\\cdot\\frac{S_i^2}{N-k}\\right] - \\sum_{i=1}^k (n_i-1)\\cdot ln(S_i^2)$$\n",
    "\n",
    "$$h = 1+\\frac{\\sum_{i=1}^k\\left[\\frac{1}{n_i-1}\\right]-\\frac{1}{N-k}}{3(k-1)}$$\n",
    "\n",
    "$$N = \\sum_{i=1}^k n_i$$\n",
    "\n",
    "$H_0$ is rejected if:\n",
    "\n",
    "$$\\frac{Q}{h} > \\chi_{1-\\alpha,k-1}^2$$\n",
    "\n",
    "In this test $H_a$ is that at least one $\\sigma_i^2$ is different from the other $\\sigma_i^2$ . The test is known as Bartlett's test for homogenity of variance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary and Review: Hypothesis Testing Facts\n",
    "\n",
    "- **Hypotheses:**\n",
    "    - *Null Hypothesis* $H_0$: The accepted explanation, status quo. This is what we're trying to disprove.\n",
    "    - *Alternate Hypothesis* $H_a$ : What the researcher or scientist thinks might really be going on, a (possibly) better explanation than the null hypothesis.\n",
    "- **Test:**\n",
    "    - The goal of the test is to reject $H_0$ in favor of $H_a$ . We do this by calculating a *test statistic* and comparing its value with a value from a table in the book, the *critical value*.\n",
    "    - If our test statistic is more extreme than our critical value, then it falls within the *rejection region* of our test and we reject $H_0$ . We can set up the rejection region before computing our test statistic.\n",
    "- **Decisions:**\n",
    "    - Reject $H_0$.\n",
    "    - Fail to reject $H_0$.\n",
    "- **Errors:**\n",
    "    - Type I: Reject $H_0$ when $H_0$ is really true.\n",
    "    - Type II: Fail to reject $H_0$ when $H_0$ is really false.\n",
    "- **Z-test or T-test:**\n",
    "    - When we test the mean, both one sample test and two sample test, we used either Z-test (when we assume the variance is known) or T-test (when the variance is unknown). These are called parametric test.\n",
    "- **The basic assumptions of T test are:**\n",
    "    - Normality – the population from which the samples are drawn was assumed to be normally distributed.\n",
    "    - Homogeneity of variance – the variance of the population from which the samples are drawn was assumed to be constant.\n",
    "    - Independence – each individual score is assumed not to be influenced by any other score.\n",
    "    \n",
    "**FANCY FIGURE AT PAGE 86**\n",
    "\n",
    "**LARGE TABLE AT PAGE 87**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}