Lab 22 - Multi-linear regression, R-squared, and prediction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 22 - Multi-linear regression, R-squared, and prediction\n",
    "\n",
    "We will use the dataset with information about the labor market for recent college graduates from Lab 20 and 21.  You can download the CSV file on Blackboard.\n",
    "\n",
    "### 22.1 Loading and cleaning the data\n",
    "\n",
    "Import Seaborn and the other libraries so we can use them in our code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import statsmodels.formula.api as smf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As in Lab 20 and 21, we need to read our CSV file into a dataframe and clean it.  *This section is the same as in Lab 21.*\n",
    "\n",
    "Load the data in the dataframe `labor`, remembering to skip the non-data rows at the start and end of the file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display your `labor` dataframe below to check it was created properly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make things easier, we will rename the columns to shorter names without spaces."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "labor.columns = [\"major\",\"unemployment\",\"underemployment\",\"early\",\"mid\",\"graduate\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we need to remove the commas from the `early` and `mid` columns, and change the column type to float."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details> <summary>Answer:</summary>\n",
    "<code>labor[\"early\"] = labor[\"early\"].str.replace(\",\",\"\").astype(float)\n",
    "labor[\"mid\"] = labor[\"mid\"].str.replace(\",\",\"\").astype(float)</code>\n",
    "</details>\n",
    "\n",
    "Check that this code worked correctly by displaying `labor` again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 22.2 Review:  simple linear regression\n",
    "\n",
    "In Lab 21, we modeled the median mid-career wage using the median early career wage.  Let's generate that linear model again so that we can compare it with the new model.\n",
    "\n",
    "First, create the linear model, as in Lab 21:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details> <summary>Pattern:</summary>\n",
    "<code>lm_var_name = smf.ols(formula = 'dependent_var ~ independent_var', data = df).fit()</code>\n",
    "</details>\n",
    "\n",
    "Next display the summary information about the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details> <summary>Pattern:</summary>\n",
    "<code>linear_model_variable_name.summary()</code>\n",
    "</details>\n",
    "\n",
    "\n",
    "## 22.3 Multi-linear regression\n",
    "\n",
    "We have more information in our dataset than just the median wage early career.  We can use these other variables (unemployment rate, underemployment rate, and share with graduate degree) to improve our model of the median wage mid-career.\n",
    "\n",
    "The (statistical) variables that we are using to make the prediction are called the *independent variables* and the variable we are trying to predict with our model is called the *dependent variable*.\n",
    "\n",
    "The following code creates a multi-linear model using the `unemployment`, `underemployment`, `early`, and `graduate` columns as the independent variables and the `mid` column as the dependent variable.  We will call the model `lm_multi` to distinguish it from your simple linear regression model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "lm_multi = smf.ols(formula = 'mid ~ early + unemployment + underemployment + graduate', data = labor).fit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The information about the linear model is stored in the (coding) variable `lm_multi`.  Display the summary information for `lm_multi` (same command as in Lab 21): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details> <summary>Pattern:</summary>\n",
    "<code>linear_model_variable_name.summary()</code>\n",
    "</details>\n",
    "\n",
    "Compare the two summaries.  Do they contain the same information, or is there anything new?\n",
    "\n",
    "Our new model has a coefficient for each of the independent variables.  We can use them to write the equation for our multi-linear model. If:<br>\n",
    "$x_1$ = `early`<br>\n",
    "$x_2$ = `unemployment`<br>\n",
    "$x_3$ = `underemployment`<br>\n",
    "$x_4$ = `graduate`<br>\n",
    "\n",
    "then what is the linear model?\n",
    "\n",
    "What does the histogram of the residuals look like?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "<details> <summary>Hint:</summary>\n",
    "<code>linear_model_variable_name.resid</code> is a Pandas Series containing the residuals.\n",
    "</details>\n",
    "\n",
    "Does the histogram look normal?\n",
    "\n",
    "### 22.4 P-values\n",
    "\n",
    "Look at the column labeled `P>|t|` in the middle section of the summary.  This column gives the probabilities that for each row, we would still get the coefficient show in the `coef` column or one larger in magnitude if the independent variable at the start of the row did NOT predict the dependent variable. This probability is called a *p-value*.  We want small p-values.  Look at the probabilities - are they small?\n",
    "\n",
    "If the p-value for an independent variable is small, then it is likely that variable helps predict the dependent variable.  However, if the p-value for an independent variable is large, then that variable probably does not help predict the dependent variable.\n",
    "\n",
    "### 22.5 R-squared\n",
    "\n",
    "To know how good our prediction is, we can look at the `R-squared` value in the second column at the top.  `R-squared` is the proportion of variance in the dependent variable (median mid-career wage) that is explained by the independent variables.  `R-squared` is between 0 and 1, with closer to 1 being better.\n",
    "\n",
    "Compare the `R-squared` values between the single linear regression and the multi-linear regression.  Did adding more independent variables improve the model?\n",
    "\n",
    "### 22.6 Predictions\n",
    "\n",
    "Finally, let's make some predictions with our model.  For example, suppose your major is not on the list, but you know the unemployment rate is 3.5%, the underemployment rate is 35%, the median early career wage is $42,000, and the share with graduate degrees is 37%.  We want to predict the median mid-career wage.\n",
    "\n",
    "First, we make a dataframe with this information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "new_info = pd.DataFrame({\"unemployment\":[3.5], \"underemployment\":[35], \"early\":[42000],\"graduate\":[37]})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display your dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make the prediction, we use the function `lm_multi.predict()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "lm_multi.predict(new_info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the predicted median mid-career wage?\n",
    "\n",
    "What is the predicted median mid-career wage for a major with an unemployment rate of 2.8%, an underemployment rate of 15%, a median early career wage of $52,000, and the share with graduate degrees is 27%?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 22.7 Starbucks drinks\n",
    "\n",
    "Let's try making a multi-linear model for the number of calories in a Starbucks drink based on the amount of carbs, fat, protein, and sodium the drink contains.\n",
    "\n",
    "First, load in the Starbucks drinks dataset from Lab 18 into a dataframe called `drinks`.  To save time, the code to do this and clean the data is below.  Make sure you change the file name if needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "drinks = pd.read_csv(\"starbucks-menu-nutrition-drinks.csv\",index_col = 0, na_values = \"-\")\n",
    "drinks = drinks.dropna(axis = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display `drinks`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a linear model where the number of calories is the dependent variable and the amount of fat, carbs, fiber, protein, and sodium are the independent variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details> <summary>Pattern:</summary>\n",
    "<code>linear_model_var_name = smf.ols(formula = 'dependent_var ~ independent_var_1 + independent_var_2 + ... + last_independent_var', data = df).fit()</code>\n",
    "</details>\n",
    "\n",
    "Display the summary for your linear model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What's the equation for the linear model?\n",
    "\n",
    "What's the R-squared value?  Does it suggest the model is good?\n",
    "\n",
    "We should also check that the residuals look normal:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Do the residuals have a normal distribution?\n",
    "\n",
    "Look at the p-values for the coefficients.  Do any of them suggest that an independent variable does not help predict the number of calories?\n",
    "\n",
    "Can you predict how many calories would be in a drink with 0g of fat, 15g of carbs, 1g of fiber, 0g of protein, and 5g of sodium?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}