end-to-end-bluebook-bulldozer-price-regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 🚜 Predicting the Sale Price of Bulldozers using Machine Learning \n",
    "\n",
    "In this notebook, we're going to go through an example machine learning project with the goal of predicting the sale price of bulldozers.\n",
    "\n",
    "Since we're trying to predict a number, this kind of problem is known as a **regression problem**.\n",
    "\n",
    "The data and evaluation metric we'll be using (root mean square log error or RMSLE) is from the [Kaggle Bluebook for Bulldozers competition](https://www.kaggle.com/c/bluebook-for-bulldozers/overview).\n",
    "\n",
    "The techniques used in here have been inspired and adapted from [the fast.ai machine learning course](https://course18.fast.ai/ml).\n",
    "\n",
    "## What we'll end up with\n",
    "\n",
    "Since we already have a dataset, we'll approach the problem with the following machine learning modelling framework.\n",
    "\n",
    "| <img src=\"../images/ml101-6-step-ml-framework.png\" width=500/> | \n",
    "|:--:| \n",
    "| 6 Step Machine Learning Modelling Framework ([read more](https://whimsical.com/9g65jgoRYTxMXxDosndYTB)) |\n",
    "\n",
    "To work through these topics, we'll use pandas, Matplotlib and NumPy for data anaylsis, as well as, Scikit-Learn for machine learning and modelling tasks.\n",
    "\n",
    "| <img src=\"../images/supervised-projects-6-step-ml-framework-tools-highlight.png\" width=500/> | \n",
    "|:--:| \n",
    "| Tools which can be used for each step of the machine learning modelling process. |\n",
    "\n",
    "We'll work through each step and by the end of the notebook, we'll have a trained machine learning model which predicts the sale price of a bulldozer given different characteristics about it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Problem Definition\n",
    "\n",
    "For this dataset, the problem we're trying to solve, or better, the question we're trying to answer is,\n",
    "\n",
    "> How well can we predict the future sale price of a bulldozer, given its characteristics previous examples of how much similar bulldozers have been sold for?\n",
    "\n",
    "## 2. Data\n",
    "\n",
    "Looking at the [dataset from Kaggle](https://www.kaggle.com/c/bluebook-for-bulldozers/data), you can you it's a time series problem. This means there's a time attribute to dataset.\n",
    "\n",
    "In this case, it's historical sales data of bulldozers. Including things like, model type, size, sale date and more.\n",
    "\n",
    "There are 3 datasets:\n",
    "1. **Train.csv** - Historical bulldozer sales examples up to 2011 (close to 400,000 examples with 50+ different attributes, including `SalePrice` which is the **target variable**).\n",
    "2. **Valid.csv** - Historical bulldozer sales examples from January 1 2012 to April 30 2012 (close to 12,000 examples with the same attributes as **Train.csv**).\n",
    "3. **Test.csv** - Historical bulldozer sales examples from May 1 2012 to November 2012 (close to 12,000 examples but missing the `SalePrice` attribute, as this is what we'll be trying to predict).\n",
    "\n",
    "## 3. Evaluation\n",
    "\n",
    "For this problem, [Kaggle has set the evaluation metric to being root mean squared log error (RMSLE)](https://www.kaggle.com/c/bluebook-for-bulldozers/overview/evaluation). As with many regression evaluations, the goal will be to get this value as low as possible.\n",
    "\n",
    "To see how well our model is doing, we'll calculate the RMSLE and then compare our results to others on the [Kaggle leaderboard](https://www.kaggle.com/c/bluebook-for-bulldozers/leaderboard).\n",
    "\n",
    "## 4. Features\n",
    "\n",
    "Features are different parts of the data. During this step, you'll want to start finding out what you can about the data.\n",
    "\n",
    "One of the most common ways to do this, is to create a **data dictionary**.\n",
    "\n",
    "For this dataset, Kaggle provide a data dictionary which contains information about what each attribute of the dataset means. You can [download this file directly from the Kaggle competition page](https://www.kaggle.com/c/bluebook-for-bulldozers/download/Bnl6RAHA0enbg0UfAvGA%2Fversions%2FwBG4f35Q8mAbfkzwCeZn%2Ffiles%2FData%20Dictionary.xlsx) (account required) or view it on Google Sheets.\n",
    "\n",
    "With all of this being known, let's get started! \n",
    "\n",
    "First, we'll import the dataset and start exploring. Since we know the evaluation metric we're trying to minimise, our first goal will be building a baseline model and seeing how it stacks up against the competition.\n",
    "\n",
    "### Importing the data and preparing it for modelling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import data analysis tools \n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we've got our tools for data analysis ready, we can import the data and start to explore it.\n",
    "\n",
    "For this project, we've [downloaded the data from Kaggle](https://www.kaggle.com/c/bluebook-for-bulldozers/data) and stored it under the file path `\"../data/\"`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/daniel/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.7/site-packages/IPython/core/interactiveshell.py:3063: DtypeWarning: Columns (13,39,40,41) have mixed types.Specify dtype option on import or set low_memory=False.\n",
      "  interactivity=interactivity, compiler=compiler, result=result)\n"
     ]
    }
   ],
   "source": [
    "# Import the training and validation set\n",
    "df = pd.read_csv(\"../data/bluebook-for-bulldozers/TrainAndValid.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 412698 entries, 0 to 412697\n",
      "Data columns (total 53 columns):\n",
      " #   Column                    Non-Null Count   Dtype  \n",
      "---  ------                    --------------   -----  \n",
      " 0   SalesID                   412698 non-null  int64  \n",
      " 1   SalePrice                 412698 non-null  float64\n",
      " 2   MachineID                 412698 non-null  int64  \n",
      " 3   ModelID                   412698 non-null  int64  \n",
      " 4   datasource                412698 non-null  int64  \n",
      " 5   auctioneerID              392562 non-null  float64\n",
      " 6   YearMade                  412698 non-null  int64  \n",
      " 7   MachineHoursCurrentMeter  147504 non-null  float64\n",
      " 8   UsageBand                 73670 non-null   object \n",
      " 9   saledate                  412698 non-null  object \n",
      " 10  fiModelDesc               412698 non-null  object \n",
      " 11  fiBaseModel               412698 non-null  object \n",
      " 12  fiSecondaryDesc           271971 non-null  object \n",
      " 13  fiModelSeries             58667 non-null   object \n",
      " 14  fiModelDescriptor         74816 non-null   object \n",
      " 15  ProductSize               196093 non-null  object \n",
      " 16  fiProductClassDesc        412698 non-null  object \n",
      " 17  state                     412698 non-null  object \n",
      " 18  ProductGroup              412698 non-null  object \n",
      " 19  ProductGroupDesc          412698 non-null  object \n",
      " 20  Drive_System              107087 non-null  object \n",
      " 21  Enclosure                 412364 non-null  object \n",
      " 22  Forks                     197715 non-null  object \n",
      " 23  Pad_Type                  81096 non-null   object \n",
      " 24  Ride_Control              152728 non-null  object \n",
      " 25  Stick                     81096 non-null   object \n",
      " 26  Transmission              188007 non-null  object \n",
      " 27  Turbocharged              81096 non-null   object \n",
      " 28  Blade_Extension           25983 non-null   object \n",
      " 29  Blade_Width               25983 non-null   object \n",
      " 30  Enclosure_Type            25983 non-null   object \n",
      " 31  Engine_Horsepower         25983 non-null   object \n",
      " 32  Hydraulics                330133 non-null  object \n",
      " 33  Pushblock                 25983 non-null   object \n",
      " 34  Ripper                    106945 non-null  object \n",
      " 35  Scarifier                 25994 non-null   object \n",
      " 36  Tip_Control               25983 non-null   object \n",
      " 37  Tire_Size                 97638 non-null   object \n",
      " 38  Coupler                   220679 non-null  object \n",
      " 39  Coupler_System            44974 non-null   object \n",
      " 40  Grouser_Tracks            44875 non-null   object \n",
      " 41  Hydraulics_Flow           44875 non-null   object \n",
      " 42  Track_Type                102193 non-null  object \n",
      " 43  Undercarriage_Pad_Width   102916 non-null  object \n",
      " 44  Stick_Length              102261 non-null  object \n",
      " 45  Thumb                     102332 non-null  object \n",
      " 46  Pattern_Changer           102261 non-null  object \n",
      " 47  Grouser_Type              102193 non-null  object \n",
      " 48  Backhoe_Mounting          80712 non-null   object \n",
      " 49  Blade_Type                81875 non-null   object \n",
      " 50  Travel_Controls           81877 non-null   object \n",
      " 51  Differential_Type         71564 non-null   object \n",
      " 52  Steering_Controls         71522 non-null   object \n",
      "dtypes: float64(3), int64(5), object(45)\n",
      "memory usage: 166.9+ MB\n"
     ]
    }
   ],
   "source": [
    "# No parse_dates... check dtype of \"saledate\"\n",
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7fdef9472b50>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAD4CAYAAACqnDJ3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO29f5Rc1XXn+91VXZKqha2WjOxBDUIyJtJDlkFGA0o0awI4RhgMdAAbMNgkj/eYydjjgRCNxUQrErYclKXnh+03GSfEJsaBBwJBGhFwZGLJKyvYwrRoCVkOCgKMUIuYtqVWQF2SqrvP/HHPKd26dc655/6qulW9P2u11H3q/jj31r1nn73P/kFCCDAMwzBMsym0ugMMwzDM5IQFEMMwDNMSWAAxDMMwLYEFEMMwDNMSWAAxDMMwLaGr1R1Im1NPPVXMmzev1d1gGIZpK3bs2PErIcTsZp6z4wTQvHnzMDAw0OpuMAzDtBVE9Eazz8kmOIZhGKYlsABiGIZhWgILIIZhGKYlsABiGIZhWgILIIZhGKYldJwXHDP56B8cwoYte3FwpII5PWWsXLEAfUt6W90thmFCYAHEtDX9g0O464ndqFTHAQBDIxXc9cRuAGAhxDA5J9QER0T3E9HbRPQzzWd/RESCiE6VfxMRfZOI9hHRS0T0Ud+2txDRK/LnFl/7+US0W+7zTSIi2T6LiJ6V2z9LRDPTuWSmk9iwZW9N+Cgq1XFs2LK3RT1iGMYVlzWg7wK4LNhIRGcA+DiA/b7mTwA4W/7cBuBbcttZANYAuBDABQDW+ATKt+S2aj91rlUAfiiEOBvAD+XfDFPHwZFKpHaGYfJDqAASQvwjgEOaj+4F8N8B+CvaXQ3ge8JjO4AeIjoNwAoAzwohDgkhDgN4FsBl8rP3CiF+IrzKeN8D0Oc71gPy9wd87QxTY05POVI7wzD5IZYXHBFdBWBICLEr8FEvgDd9fx+Qbbb2A5p2APiAEOItAJD/v9/Sn9uIaICIBoaHh2NcEdOurFyxAOVSsa6tXCpi5YoFLeoRwzCuRHZCIKJuAH8M4FLdx5o2EaM9EkKI+wDcBwBLly7lGuOTCOVowF5wDNN+xPGCOwvAfAC7pL/A6QBeJKIL4GkwZ/i2PR3AQdl+UaD9R7L9dM32APBLIjpNCPGWNNW9HaOvzCSgb0kvCxyGaUMim+CEELuFEO8XQswTQsyDJ0Q+KoT4VwCbAXxOesMtA3BEms+2ALiUiGZK54NLAWyRn71DRMuk99vnADwpT7UZgPKWu8XXzjAMw3QALm7YDwP4CYAFRHSAiG61bP4MgNcA7APwVwD+CwAIIQ4B+AqAF+TPl2UbAPwBgG/LfV4F8H3Zvh7Ax4noFXjeduujXRrDMAyTZ8hzPuscli5dKrgeEMMwTDSIaIcQYmkzz8m54BiGYZiWwAKIYRiGaQksgBiGYZiWwAKIYRiGaQksgBiGYZiWwAKIYRiGaQksgBiGYZiWwAKIYRiGaQlcEZWZVHD5bobJDyyAmEkDl+9mmHzBAojJJVloKrby3SyAGKb5sABicoMSOkMjFRBOFoZKS1PJqnw3m/UYJh7shMDkAmUeG5LCIJgiV2kqSciifLe/3wInhWX/4FDsYzLMZIEFEJMLdOaxIEk1lSzKd9vMegzD2GETHJMLXIRLEk0FyKZ8d1ZmPYaZDLAAYnLBnJ5yzfymI6mmoki7fLep30mFJcNMBtgEx+QCnXmM5P+9PWXcc83iXC7sZ2HWY5jJAmtATC7IwjzWDNq13wyTB7gkd5Nhl12GYfJIK0pyswbURDgSn2EY5iSha0BEdD8RvU1EP/O1bSCil4noJSL6WyLq8X12FxHtI6K9RLTC136ZbNtHRKt87fOJ6HkieoWINhLRFNk+Vf69T34+L62LbhXsssswDHMSFyeE7wK4LND2LIAPCyE+AuBfANwFAER0DoAbACyS+/wvIioSURHAnwP4BIBzANwotwWAPwNwrxDibACHAdwq228FcFgI8SEA98rt2hp22WXyRP/gEJav34r5q57G8vVbOXiWaTqhAkgI8Y8ADgXafiCEGJN/bgdwuvz9agCPCCGOCyFeB7APwAXyZ58Q4jUhxAkAjwC4mogIwCUANsn9HwDQ5zvWA/L3TQA+JrdvW7KIxGeYOHAGByYPpOGG/X8C+L78vRfAm77PDsg2U/v7AIz4hJlqrzuW/PyI3L4BIrqNiAaIaGB4eDjxBcUlbEbJLrtMXmBzMJMHEgkgIvpjAGMAHlJNms1EjHbbsRobhbhPCLFUCLF09uzZ9k5nhMuMsm9JL+65ZjF6e8og5Du+hels2BzM5IHYXnBEdAuATwL4mDjpy30AwBm+zU4HcFD+rmv/FYAeIuqSWo5/e3WsA0TUBWAGAqbAPOGa6j/tSHyGiQNncGDyQCwNiIguA/AlAFcJIUZ9H20GcIP0YJsP4GwAPwXwAoCzpcfbFHiOCpul4NoG4Dq5/y0AnvQd6xb5+3UAtoocBy3xjJJpJ9gczOSBUA2IiB4GcBGAU4noAIA18LzepgJ4VvoFbBdC/GchxB4iehTAz+GZ5j4vhBiXx/kCgC0AigDuF0Lskaf4EoBHiGgdgEEA35Ht3wHwN0S0D57mc0MK15sZPKNk2gnO4MDkAc6EkBLBIFPAm1HyGg/DMO0AZ0JoY3hGyTAMEw0WQCnCDgatg3PsMVnCz1c2sABi2h7OscdkCT9f2cH1gJi2h4MqmSzh5ys7WAAxbQ+7wDNZws9XdrAJjmk7gvb4nu4SDo9WG7ZjF3gmDTjEIjtYA2LaCl3Ko3ePjaFUrM/cxEGVTFpw0G52sAbUBNiDJj109vjqhEBPuYTpU7sS32P+rpggHGKRHSyAMoY9aNLFZHc/Uqli55pLEx2bvyvGBIdYZAOb4DKGPWjSJcuaSvxdMUxzYQGUMexBky5Z2uP5u2KY5sImuIzJ2oNmsq1ZZGmPZ28nhmkuLIAyZuWKBdokpWnM2CfrmkVW9vgsvyuGYRphE1zGZFkFldcs0oUr1jJMc2ENqAlkNWPnNYv0YW8nhmkerAG1MVl6hDEMw2QNC6A2hiO0GYZpZ9gE18Zk6REWx7tusnnk6eB7EA++b5MTLskdg05/WVzKiwfvwcULZ+PxHUOTuiQ5l2WPB9+3fNCKktxsgouILhnmXU/sRv/gUKu7lhph3nW6e/DQ9v2T3iOPvRLjwfdt8hIqgIjofiJ6m4h+5mubRUTPEtEr8v+Zsp2I6JtEtI+IXiKij/r2uUVu/woR3eJrP5+Idst9vklEZDtHq5kML0uYd53uHpj06MnkkcdeifHg+2anf3AIy9dvxfxVT2P5+q0dNdl10YC+C+CyQNsqAD8UQpwN4IfybwD4BICz5c9tAL4FeMIEwBoAFwK4AMAan0D5ltxW7XdZyDlaymR4WcK866Jca1KPvHZ6+bL2SmynexEF9uY00+kWl1ABJIT4RwCHAs1XA3hA/v4AgD5f+/eEx3YAPUR0GoAVAJ4VQhwSQhwG8CyAy+Rn7xVC/ER4i1HfCxxLd46WMhleljDvOtO1UuDvpB557fbyZemV2G73IgrszWmm0y0ucdeAPiCEeAsA5P/vl+29AN70bXdAttnaD2jabedogIhuI6IBIhoYHh6OeUluTIaXJSwjgOke3LRsbqpZBJr18qWlWeQp60U7aUucgcJMp1tc0nbDDk6CAW95IGp7JIQQ9wG4D/C84KLuH4XJUpzKlhGgWfegGS/f6v7deGj7/tpDlzSfXh6yXuhyBK7ctAtrN+/BkUo1l88sZ6DQ0+kJcuMKoF8S0WlCiLekGe1t2X4AwBm+7U4HcFC2XxRo/5FsP12zve0cLYdflubcg2ZkEvcLH4XSLPL0HUe5F9qqseMCI5UqgMmTtLYT6PQEuXFNcJsBKE+2WwA86Wv/nPSGWwbgiDSfbQFwKRHNlM4HlwLYIj97h4iWSe+3zwWOpTvHpKWdzCppkLW5c8OWvW3jvRflXrj0PQ/rCJPteQaiX3OnmydDNSAiehie9nIqER2A5822HsCjRHQrgP0APiU3fwbA5QD2ARgF8PsAIIQ4RERfAfCC3O7LQgjl2PAH8DztygC+L39gOcekpNmlF/IQbJuGqc92HbaBOm8mjij3wqQtBXERVFk9B5OxlEjca+5kiwtnQnAgD4Px8vVbtYNKb08Zz626JNVz9Q8OYeWmXaiOn3w2SkXChuvObasXISzC3nRPCcC915/XVtfqR3fdOsKenSwzFDTzeW42pvEi79fMmRBySF7cX5vpDXP3U3vqhA/grSHc/dSe1M+VJWGeYzqzFgG4adncthU+QKPZZmZ3CaVCvb+PiykzSy/ETvXuso0XnXrNSeBkpAGCs5ejx8eML2EzB6lmesMcHq1Gas8rYS98J3s0Bs02cbT4LAfMTvXusgntTr3mJLAA8qGz0Zpo9qyl071hssDlhe9k+7qfONeZ5YDZqc+zTWjfe/15HXnNSWATnA/d7MVEs2ctzfSG6SmXIrXnlckQNJwlWd6/TvXusmVK6dRrTgJrQD5ctZpWDWLNmq2vvWoRVj62C9UJnxNCgbD2qkWZnztNOtnE1gyyvn+dqH2GaXadeM1JYC84HyYvlZndJXRP6ZpUg1gePP/y1A+GcaVdn9lWeMGxAPLBhbHyBX8fDNM8WiGA2ATnIwuTQ9hsKM3ZUrvOvEzYPIqy/E4YhmkOLIACpGmjDYt8TjMavBMjy7NwA87bfWJhyExm2AsuQ8IC+dIM9OvEuiFZ1F7K033KS5BzOzMZ88l1EqwBpUhwNmuKI1Iz+DRn+KZ9hkYqmL/q6bacXWcRK9LKaPS8Bjm3K3nTZpnosABKCd3LQNAXN+rp9uJp0gz0swk8/+waaJ+XM4s1uZ7ukjajQ5olHnT9zXOQc7uS1Roh0zxYAKWE7mUw+Re+e2zMS/iZ4gxfd6wg/pezXdYeXNbkXK+lf3AI7x4ba2gvFSnVstm6GXmeg5zbFc6t1v7wGlBKRHnoqxOiJgjSiowOHsvWz05ae4hyLRu27K0LrlVMn9KVednsvAc5tyNZrBEyzYU1oJRwrcGi8CfEDLplL1+/NZbbtv9YpqDaOT3ljjJdRLkWkxA4UkknyaptRm56PiZjkHNadGo+uckEa0ApocubZUM3SwubzUeZ7dvyeLWD6cLVu8n1WvoHh1AgvW6Y1ozZNiM3fR9rrlyE51ZdgtfXX4HnVl3CwicCOgvCtef3YsOWvewV1yawAEqJ4MtQNAx2gHmWlqbbts28l3fTRRRB63It6njjmqwfac6YbUKfE1FmQ9+S3poAX7liAR7fMdQRpuXJApvgUsRvApu/6mnjdqaBJ2w2H1VzMS3g5910EcWs5nItJgeAIlEqQsBvFp1RLmFaqYCR0arVRJqUdnEiaSadZFqeLLAAygiTzb9XpmWPso+azafltp1VluO0BsUogtblWkzHmxAilWv2C8CRShXlUjHTkt4c/6KnHUzLTD0sgDIijpYRtk+amkvaaeHTHBSjCtqwa8mysForZt0809fj8j2z5pgvEq0BEdEdRLSHiH5GRA8T0TQimk9EzxPRK0S0kYimyG2nyr/3yc/n+Y5zl2zfS0QrfO2XybZ9RLQqSV+bTRybf9g+rVxHCHMKSDPFTdqF0LIsrNaKWTfP9PWEfc+dFH7QKcTWgIioF8AXAZwjhKgQ0aMAbgBwOYB7hRCPENFfALgVwLfk/4eFEB8iohsA/BmA64noHLnfIgBzAPwDEf2GPM2fA/g4gAMAXiCizUKIn8ftc7OJo2WE7dOKglYu2k2ag2LaJsIsC6tlqV1lfc5O0wbCvmfWHPNHUhNcF4AyEVUBdAN4C8AlAD4jP38AwFp4Auhq+TsAbALwP4mIZPsjQojjAF4non0ALpDb7RNCvAYARPSI3LZtBFCn4PLiziiXMKKJp5kRs4x32oI2K8HdCoeONM7ZqetItu+ZNcf8EdsEJ4QYAvD/ANgPT/AcAbADwIgQQuU7OQBAPQ29AN6U+47J7d/nbw/sY2pvgIhuI6IBIhoYHh6Oe0mMAZcX1+R1/m/Hqh0dkxE0i/ZIL7g7Nu7M7JrTMMXmKSt4s8h7+MFkJIkJbiY8jWQ+gBEAjwH4hGZTFXyhG6KEpV0nHLXp1YQQ9wG4D/Aqolo7HgIXiGvEZPLp6S7VsjaYbrrKfNOsGXYr7rmadTdTq0iq0aWhDTT7Xic9X97DDyYjSZwQfgfA60KIYSFEFcATAH4LQA8RKcF2OoCD8vcDAM4AAPn5DACH/O2BfUztmZHmImUnLXjqFndLRcK7x8Zq1+dC1jPsVt/zdtIqkmoDzb7XaZyPg4HzRxIBtB/AMiLqlms5H4O3PrMNwHVym1sAPCl/3yz/hvx8qxBCyPYbpJfcfABnA/gpgBcAnC296qbAc1TYnKC/oeShQFweC2zpXtzpU7q0iT3DyNLe3moB0E5rDEk9A5t9r9M6nz9zAqc+aj2xTXBCiOeJaBOAFwGMARiEZwZ7GsAjRLROtn1H7vIdAH8jnQwOwRMoEELskR50P5fH+bwQYhwAiOgLALYAKAK4XwixJ25/XbAVdUvrWLbBKKkJJ2iiuHjhbGx7eTg1TzL/vrZMDzaytLe3WgC0wiMuLkk9A5t9r1v93TLZkMgLTgixBsCaQPNrOOnF5t/2GIBPGY7zVQBf1bQ/A+CZJH2MgmkAIXiDe5TBO85glMRNVCe8Hty+v/Z5nPWI1f278fDzb2JcCBSJcOOFZ2Bd32Lr9dnI2t6ehgBIss7QbmsMSdaRmh30maVw75S12naEk5H6WLligdEjIqqqH8fEkWSW51LwLIrJYnX/bjy4fX8tgee4EHhw+36s7veEWJTs382ytyc1KyVdZ5hMawzNDvrMKpi41euGkx1OxeOjb0kvbt+4U/tZVFVfZ+K4eOFsbNiyF3ds3KmdaYXN8mwzNdf+uW738PNvGtvX9S1uuL4CkTbbdG9PGc+tusTpnElJalZKI1AxiVbRTjPxZgd9ZhVMzMGprYUFUIDeFFV9/2Dksr5jM+GY9h944xC2vTzs7Inmeh06YeJv1603Pb5jqOXmp6AACCvw56eV6wwuz0feBFSzgz6zCCbmtaXWwia4ABcvnB2p3RUXLx6bCce0/0Pb9zuvxUQRCKZ6RkUirdni8R1DuPb83lyZn6KaV1oZqBj2fLSbqahdgj7bpZ+dCgugANte1mdSMLXb8LtUm4REcKZlchM1zchMmk9vTxk3L5sbWyDceOEZxnbTYLnt5WE8t+oS3Hv9eQCQKBtAXHd0/353ProrkutulklLwwibibfaxTwqrbyXUWiXfnYqbIILkJZKHjSpmChIjSJMMETxOiMg0bpL/+BQg8D1e8GZXLAPjlRSyQYQ9xjB/UxmRFsBPyCbpKVhhK3/tZupqJX3Mgrt0s9OhQVQgJ7uEg6PNibV7OmOllTTxSsN8AZJl8F15YoFWPnYLqfgzyTmA53gLJeKddqT7R6lsagb9xiu99x2f1qRbRwId+FupxgjRavuZVTapZ+dCJvgAhgmzcZ2E1Fmpi6mlL4lvThlWvh8Ian5wMXUY7tHaczU4wbxumiIeTWvhLlws6mI6URYAwpwRFNSwNZuImqgpssAPaLROhQkz+k3H8TxmnIZ/G33KI2ZetRjKK3NRJEIE0Jkbl5J6qVmm4nHNRXlzXOOYfywAAqQlqnDZFKZVipozVcuxzf1TRdrE3cdxeX6bdukkQ0g6jFspreg+TArmpEJO6qpqFNr/jCdA5vgAqRl6jCZVNZcuSj28aP0La7X1MoVC1As1LtgFwtUdw5dPwjeALdhy97E7thRa+zYtMdmuYLn0Ustj31iGD+sAaHRTHHt+b2pJPG0zVjjmEWimGHirsUMvHEI4wFHh/EJgYE3DtXO4+/H0EgFhJPu4ComKOnAH6XGjk0zNPUhjmkqTiaKsPvdPziEtZv31KrJzuwuYc2Vi1IRms3ynGMzHxOXSS+AdANcGgOojSReN677xvXmC0vBE+zH8vVbGwb/NFOZuHjERTXZxTFNhe0Tx3TbPzjU4Nl4eLSKlZt2WfviSjM859jMxyRh0pvg4pgpXIMkW1nbx+Spdqw6bu1TWAqeIFnPsl2O7zfZAZ7TgfoOdfc8zndu26d/cAijJ8Ya9gkzrW7YslfrVl8dF6mYyZrhOTcZzHx5rNHVKUx6DSjqAOo648tqZuhq7jB5qlWqE7VZsa5PRUNSUVNqnrgzf1eTjevx1f4u9zyO0LTVilq5aReq4/X3rKdcwtqr7Ka0OOeLQjOCLOO6zLeLyW51/248tH1/nYmZNbz0mPQaUNRcUGs373Ga8WUxM4ySD8zVzBLsky0FT7AvyvwWFE0u5i/XnGZZOF7Eyf9l+owIDcJHtbtkt4jzWZ6w3Uud5tBOOe36B4fqhI+i0zS8VjLpBVCUAa5/cKi2WBwkOOPLwjQVRahFqdfj79O6vsW4edncmsZTJMLNy+bWrf/4BxHAc0BQQijM6y2qYI5SY8f1nscxTZn2MZk6detvumOWCo2aZalIqZjJXAb7pOYl0325eOFs7bldJ3B5YMOWvcZci3lNgdRuTHoTXBQzhe0lCVaCNNXHSTKzjSLUdNc1emLMKQZpXd9iLD1zVm3fbS8P1+Wr0wkRAbfaPy7XoDPRuOS2i2qui2IGMu1jqh/lgjpmVl5wYQ4caZiJTffFdG5TvFYeB3Rbn9pFQ807k14AAeGeZWpAtGU2CFaC1AmfpAvAUddbdLVxXLzFwgamJNqdS9G9uIOizhsOAEZPjDUkfI3jieh3DVeFBW2cddczdWXM1fUFB+uday6N1A/TcYLXkyTDdtIMDmH3JkgeB3TTs0pAqo4ck5lJb4ILI2hu0jGzu2TVDgDPlJXUtTupV5OrOSvMTJakhkrYNSRZO1PX11OudzU/PFpNbZ0haNayESxjntb6h+txwr6nLD0YTeee2V1qm5x2poDrm5bNZQeElEgkgIioh4g2EdHLRPTPRPSbRDSLiJ4lolfk/zPltkRE3ySifUT0EhF91HecW+T2rxDRLb7284lot9znm0QGV6wMCcuwXC4VsebKRbW/TS/vhBCJH9oo6yG2Y+jqDfkJG5iSCMKwa4jrVaXWMTZs2QvdU1KpjuPOR3clFkKuGbf9qNiqtBxTXI8T9j1lWYzNdO41Vy5K/Aw3C92zeu/159VptEwykprgvgHg74UQ1xHRFADdAP4HgB8KIdYT0SoAqwB8CcAnAJwtfy4E8C0AFxLRLABrACyFt5Swg4g2CyEOy21uA7AdwDMALgPw/YR9joRt4OvVmD5Martr3Z8wmpE6PsxM5rKGYjMRma4hztqZzmRnwrX0hY042oG6nrQ0DtfjhH1PaeTtMxF27jwKHB1cqiFbYgsgInovgP8I4PcAQAhxAsAJIroawEVyswcA/AieALoawPeEEALAdqk9nSa3fVYIcUge91kAlxHRjwC8VwjxE9n+PQB9aLIAipIAFPBKdz+4fX9DexqDX7NwGZhsL2aSTANR186iaiRJszREzXIOnIyhSiszQZTjZJFh2xUevJkwkpjgPghgGMBfE9EgEX2biKYD+IAQ4i0AkP+/X27fC8Cf5+WAbLO1H9C0N0BEtxHRABENDA9HL51tI6q5yVa6O404oGZEZCc19aWVaQAIXzuLo5EkWePQPQ+lAmGmJcWRiqFKKzNBmhkOXEyyDJMVSUxwXQA+CuC/CiGeJ6JvwDO3mdCt34gY7Y2NQtwH4D4AWLp0acTScSexmY1cZokuRdHiDn4uWkWaEeZJZq9xTE2m+zYeWDsLXuOMckkbm0VkTkeUxBwa9jys7t+Nh59/E+NC1JUxd9k3rT74ySLrQDtlMmDyTRIBdADAASHE8/LvTfAE0C+J6DQhxFvSxPa2b3t/OP3pAA7K9osC7T+S7adrts+EsAHeJVOyrSiaIu4CbzNiOtIiTiJUlxRAumssFQmlAjXkVLNVsE1qDrU9D+v6FlsXqdMyS0V5JtN8JvL0nDHtT2wTnBDiXwG8SURK7/8YgJ8D2AxAebLdAuBJ+ftmAJ+T3nDLAByRJrotAC4lopnSY+5SAFvkZ+8Q0TLp/fY537FSJ6mHkstaRJII9yQxHc3GlgjVhEsSVN01VscFTpnWVTMXmnLWBUkr4WwYrUpk2T84hDsf3ZX6M5GX54wThHYGSb3g/iuAh6QH3GsAfh+eUHuUiG4FsB/Ap+S2zwC4HMA+AKNyWwghDhHRVwC8ILf7snJIAPAHAL4LoAzP+SAzB4SkHkpO28U2DoYvPNuSZc5f9XRTTCXKNGNKV1SpThhNX70WZw+F6RpHRqsY/BMvmHP+qqed+5s04WwYrdIWVJmHqBnMXWhWjSEbrIV1DonigIQQO4UQS4UQHxFC9AkhDgshfi2E+JgQ4mz5/yG5rRBCfF4IcZYQYrEQYsB3nPuFEB+SP3/tax8QQnxY7vMF6UGXCUljIly2q07ET7MfN6YDQFOSProE7ALmdEYuC+su31EUE6dp2ySzfP/MPAsNxIW1m/doyzwoksT5ZBk75EpetDAmOZwJQZLUs8g1+WfcmWKYZ5rL+bN8SV3doU3X7+J55/Id6bYJlhhXXLxwdqQ+ulQ39WcoyEIDccGkgSpM1+1CM2oMhZEHLYxJB84FJ0nqoRTc3xRQ6ffASttrzX9+lyy+No+tIGF9dX35bTPlsIV103cEAMvXbzWWVD96fEw7KJtc5uPG67gKYVtQbTO8y2yhAmFkHTvkQjMqvTLNgQWQj6QeSv79V/fvtgakDrxxCI/vGHK2Y5vs3gNvHKobbNVgsOTLP7B6ogX7p/KWAWgQQi42d5cAzTRmymEJVnUl1U3rQiahGTdDgIsQtpX6SMvNfqbBCzFKP220OsA0ywwOcWC39PiwCS4jwgJSH37+zUh2bJPd+6Ht+7VJKU2rZapd5ScLomt3sblrAzSLhJ5yKdOcXy59i7puETcQ13S8IlHoccKuI0oi0zVXLkKpaPYGnFE2u8O3A2nkREyLdiqwl0dYA8qIsFmmaX1gaKSCs+56BuNCoLenjIsXzsa2l4eN2oWpWqOpJLdqd3F7VrjY3NMwzVjhJ5EAACAASURBVMSZSbr0Lc6MOc4s33Qel8ExzdIJ6u8/fHQndL4IzU/pmz6t1sIUaZW0mKywAMqIODnDFEoIDI1UtGa8MNQAbrOTuwR+KkzZBoIzaTUo+OvlbNiy10mQRHGt9Qsql+SlzVq3iHselySsURfe+5b0GmvyjFjMc3mhXcxaWTpEtMs9SAILoIwwJSVNE4I+tEg9rLZZ/40XnqHtn8pbVncew4xZ1x43RsN1Jhk8vmvy0mbNmKOexzUJa5yF93ZdrG+nOJ+s7nE73YMk8BpQRiTxNHKht6eMm5bNNbrEhtnJ1/Utxs3L5tY0niIRbl42t6F65/L1W40L2rqZdNwYDdeZpC1paRrrAUki7OPs65qENY77cx5cpl0I3re7n9rTNnE+Wd3jqO9Ru2aGYA0oIq5qcRQVvFwqoFKdcN7eXwpi6ZmzItfdUdjylunKdwfRzfLimiRcZ5Km46g1s4MjldpLGlUIxZ119g8O4e6n9tQJatd9XQsYxjHv5cFlOowo9ZzyGOeT1T2O8h61s7bEAigCUb7oKGtAx8cmUC4VnWJIgvXoszItuVSC1c3y4pokXB0FTMcnnBy84r6AcRaUbYLaZTE6rdo+JvKyWG8iSj2nvJoOs7jHUZ6LdnaEYBNcBKKoxStXLEDJEIEfZEKgZi4D7Ak1hezHvFVP46y7nsE8n8odRQ0P29YmPG1mrrgmCVfXWt3xdWthcUw2cbS3sAE0uG/wvl+8cLY2g8XoibG67yQNE0sezTSuWk0eTYdZEuU9aufMEKwBRcCW8NMfia8eElejGqFRlTc5GKjzAfXecisf2wWQlx1atdm8yGyaXP/gkPH8pkqwCv91DI1UUCRCpTqOu5/ag7ueeKlmapzZXcKaKxfFmtGr49u8/YDoL2Ac7c3FtKgwBc1ee34v/m7XW3WehodHq3XlPZKaWNI006TpnWW65z3lEqZP7cqt6TBNktYha1dnE4AFUCRsg13Q/EMQGLckhPRD5GUm8GdGiIou+aRJDQ9T2Tds2asVPkHznwl1Pv+AF3RkODxaxcpNu2rbRxkggyaP5eu3pvICmjwXbbnTbM9EcMZquu/bXh7G9KldDa7ufi0uqYklLTNN2usNJtPr2quiT07akaR1yID8ZYaIApvgNJhMFStXLNCWaQ1SqY5jNIJTwYSANjNCGqhyDP7rCFPZTZ8LuA8yLrb96vjJ7OCmAfLOR3eFmhd15s5SIXrtJZPn4sPPv2k0V9mSwE7tqn+9bPc97meupGWmSTsTdZ6yGrSCNO5nO99D1oAChM1IbjcE9yXFlJkgDVSKEKVxhKnspoqmvRE0iqh1lGzebbdv3IkCTpo0dbPuoLh3F/+NfdH1waaNASdNjn7T5UilipWP7cLdT+3ByGg1NNjU9p0k1fDSMtOEWQDikHdHiSwJmxi4mjvb9R6yBhQgbEbS45hHK2q6E9dKnkmojgvc/dQeo0np4oWz0T84hHePjTV8FrWaa9Q6SmHbBwWK/zu5+6k9DebO8QlPcEVZbLf1QTcrVVqZyjjQUy41mC6rEwKHR6vG8gzKVGJbdLZ9X66sXLFAmx8uamkG03PajOe3E7HlKZwMeeZYAAUIm5G4vGelIuGmC+c6e8GVil4pBJd6Qi7Yznp4tGo0NW17eRgbtuzVrieNjQvcEWFAd6lP5BdqrvWU/KjvxJb5OcpLG9YH/7OhGxzC6vAodEGzNjOK7fuKwvh44/e68QWzeVF7jAg5BJlwbBOPyVB4j01wAcJMFS55tKZP6cK6vsV4ffhdPPfqodDtN1x3LvqW9NYFlc4ol0DknW9OTxmHjx53WlcqEmEiZDCIsx6gjui66Bz04unpLuFYddzqBTe1qxBpHUzVVgrDdbFdfX7no/py1uoZ6B8cMm7jwoQQeH39FdrzRwlqtpm9gqabo8fHtGZJtQ7nar5xKZ3OuGPzdjPl8msH92pXWAAFCPMocQkwPVKpYnX/bifhM7O7pM1coAYQJfCWzO1xOl7YoFguFTBr+tTI6w1+lFt1mG3a1S7tknVBx7gQtXWtMFxf2lrqm8d21WmCyqnBlrvNlajrLrbgW1Xc0E+U7AJAo2an+15Ve3CdC2gfj6u8YnpP2tm92hU2wQUI8yhxMRXN6Skb6+0E0Y1jOvPOjw3CR5lzXG3w0+RgYVL7XU1hh0erqdmmo0TDB6mOCyfPxMgvbfCg8u8ofe0plxrWXeIM1ibvSxWUHCTq/fRrdro1h9X9u2vt6ryqP+3kcdVutEsuvyQk1oCIqAhgAMCQEOKTRDQfwCMAZgF4EcBnhRAniGgqgO8BOB/ArwFcL4T4hTzGXQBuBTAO4ItCiC2y/TIA3wBQBPBtIcT6pP11IWzmbjMVqXUNV2+5kUq1IYhVN4CY5tvKnGOq+tlwvtFq7dr8+cuUy3CYGcpEktQfSU0KYb2M+tJu2LK3FtCrUKYq174SgJ1rLtVqFAAavvMwc6bpedL1J+r9PHp8rNZP3ZqDKtvuR8Cb9BwcqWDt5j01T79mBY1OhlIF7ZDLLylpmOD+G4B/BvBe+fefAbhXCPEIEf0FPMHyLfn/YSHEh4joBrnd9UR0DoAbACwCMAfAPxDRb8hj/TmAjwM4AOAFItoshPh5Cn2uw/VhdjIVyffUlsnAjy6HWZzZq2vuOb8mcMy3pjRSqdat7ZjszzZ0GSFcXhZT31XWhdX9u7WDoI0iAUqGBONxwr5v2xpZ1PscnMzY3PwB82BjWnvRaXamPnaXCqiMTTRo3eq7Nz13YY4HfueLZiTCbOfkm1FpV/dqVxKZ4IjodABXAPi2/JsAXAJgk9zkAQB98ver5d+Qn39Mbn81gEeEEMeFEK8D2AfgAvmzTwjxmhDiBDyt6uok/dWhMzus3LQL5939g4agR6fgyglvpjylK/zWmnKYRXFpjeJF5tcE1m62p7yPY2dWwjSqWS7M1LCubzFevedyZxd44KTwAU4OsCqgNcy11eYaG/U+BzFpGWs377H2K4o5RlsevUCoTghjqXbbcxfVxTprT63J4B02WUi6BvR1AP8dJ8M03gdgRAihAkkOAFDiuxfAmwAgPz8it6+1B/YxtTdARLcR0QARDQwPR3NN1T3M1XGBkUq1YSCIElx5fCzcY800nx8Xwtkl2e/AEFy7unnZXO1aVv/gkNFlWM2co7pFJ0kI6hrJvfaqRc6u7UFUX1wGL9tgH+U+6zA9QyOVqrVfUaLdddueMq2rwawYRPfclUvFWCECScyqYUlT2zn5JlNPbBMcEX0SwNtCiB1EdJFq1mwqQj4zteuEo/YNEkLcB+A+AFi6dGkk9ySXh1YNBK7mlzQiIqaVCpjaVcARKQhNzF/1dJ25xkVdtwkFNdsN2p9tfTCZh4Bo3meubtIuiUij9sX/WZjtXWdW08Xk6Ex9Ufs8FOiXzuNN18/gti5rhL2+Ncjg8fwhAqaMDn7iemq5mNcmg3fYZCHJGtByAFcR0eUApsFbA/o6gB4i6pJazukADsrtDwA4A8ABIuoCMAPAIV+7wr+PqT01XAeEoZEKvn79ebHcheNweLSKUoFw7/XnYe3mPUaNxa+lAW42cNv1+gcW/yB23t0/0Pahp1zCc6suSS0haBjBgdXULx1R0trEdSFX38XAG4fqksuq9mvP721IOutaC8r13Kr/weuzfe9+DS8s1UvYWmgSTy2XpKntnHyTqSe2CU4IcZcQ4nQhxDx4TgRbhRA3AdgG4Dq52S0AnpS/b5Z/Q36+VQghZPsNRDRVetCdDeCnAF4AcDYRzSeiKfIcm+P210QUU9OGLXtx7fm9NdNG1ulHqhMCazfvwdqrFoVuGzQj9Q8O4by7f4B5q56u/Zx39w/QPzhk7bcpoNC0i2q/eOFsreeyckzQrQWlUZ/mk+ee5rSd38086BodNc2QQgWkmjzHTJmvdaa0KOdU98x0bl0KIlMdJQT64PJ9BE18PeUSZnaXUkmE6WJea+fkm0w9WQSifgnAI0S0DsAggO/I9u8A+Bsi2gdP87kBAIQQe4joUQA/BzAG4PNCiHEAIKIvANgCzw37fiHEnrQ7G8XUNDRSwUPb90PAbnZKk5FK1TkJqj+BYTCQUh1r5WN292p/bjC/ece0x8hoFf2DQ3h8x1DDNrbsCa6zd5vHmjpvGEUiXHt+by2QsoGINtP+wSGrVgqYPccOjlS0WsYdG3caS2D4z+u/Z7bvcWikgjs27sTtG3fWTGv3XLPYei+jeJZl5Z3lal7rdO+wyQKJDsvhtHTpUjEwMBB7f5MpqZX8Yv0VTv3qKZewc82lia7B7/qshG3Y9oBbNmR/MTtTH/3b6Ew95VKxNtt1vc7lZ83Ci/uPWM1cYYX2FCbhHqRoWCcxnWfRn/w9jp5o7N/0KUXs+fJlAJI9m/77psN07HKpgBNjAuNCoEhezsJ1fY0aW1pxOWHfOZMdRLRDCLG0mefkTAgBomYHzpqZ3Z7rsYupUJnDkngDHRypoH9wyEn4KLNW1NILtj7628M81lzPu/21w6FrLK7HWrt5T6jwMXmO2dYpRjXCJ9ie5HsN80g0HbtSnagJ0nEh8OD2/Vjdv7tumzSzNrN5bXLBueACRM0wnBY95RKOnhirc5UtFQlrrvTWf4J1Z3SorAZRPa38zOkpGyui+ikQcO35nhnE1qfgsf2/h5lawoSU63W6BLC6OkuEOTwUiWoDpt9zLEwrMJam7i7VgntN3mcmbSvI0EgFZ931jFabifLMPPz8m3VaUJrVVv33697rz4steNohU0LcPrbDtbnCGlCAVsUSjFSq2HDduXUzP5UlW9G3pBcrVyywOhHc9Fc/ia3FqYSbLvdgQgCP7/ACO021ZoL4Z/+mPqq0MIA9IFQdz8WBxMVZZN77knvrlUtFfO3TJ7+zviW9eG7VJXh9/RV4btUl1kFCGzxaJLx7bKymWZjqCX3t0+fWNOUwTNpMFGeccSHqtJs04nLS1KJ0x7pj407MS+DskjZxr7fTagSxAArQylgCJWDm9JRxcKSCDVv21j1Y/YNDWLnJ7kTw3KuH8He73nI6n39c7imXsOFT3uDpeg/qZrkhM/CecqluwXvjT/XJWkcqVazctKsm2GxmrKC5pmR4mk89JXxw/vGrh5xeYtNAXyAkMhXpTE/Tp3RpzX26ekIuZUJ0qKS5uvPbYn79g17YRMGFNLMb2HIppjFgp+G9Gfd6Oy0LBJvgAphiDK49vxcPbt8f+7gF8gYO2/pBmCfS3U/tCY1mB8LNRDWE5+AQRHcPTKhZrq1UUblUxCfPPS3UlKRQlVsH/+RSAPUBoRcvnI21m/fUvAJVXSEARk/BX75zIvQ6VGbpMAGy5spFWLlpV4Op1K+txjWRuAaP6uoJ2UxoNo9NU9wXAKzu32185v2TjzTictLMbhC2T5LEuWnloYt7vZ2WBYIFUABbFPxjAwecUuzomBDAZy48wyrETLObOx/1at7YKn/GYUa5hCVf/kHtuD3lEtZe5RWJG3jjkJMjgsssNxh86bJeofoUDIAMeqAdHq3iDx/diWLMFD1+XF7i4FpckaiWKVuRVqJMk1AR8ISDfx3m4oWzG74vAnDTsrlY17e4tvYTxGaeVMc3PbPqfgXfGVVM8Y6NO7Fhy14nAWy61gIR5q96Gj3dJQjh1dqKu56m63tUoqx32SYicbM5dFoWCBZAGnQxBqv7d8cWPorHdwwZs2QXyPxSjAtRlzE5DUoFwr8dq8KvkKk4IcBzxnBx0FdrOQUCdMpdgbxjJcke4S+GpmNCABMOmmEYri+xejZ0gkZXqiPujNumiSqhsK5vsTUO6/EdQ1h65izcaJj83HjhGQ1tftb1Lca2l4dDBz31ztgyQ2x7edioFZquVQlN/+QrTKi7aPBxB2xXDSRMU4qrNXZaFgheAwpgsu+6FpizUamOY5phoeI3PzgLBctsNM30P709ZUzpKmgFhsq+4OoR9eD2/Vi+fit+84OztJ9/5sK5sWabKvO1f9E1S0pFwtHjY852fdNM2GT+jHMP1LqMCfVM2rK0q8wI214exvKzZtU0niIRbpbakQ7/e3D0+JhzYT3TfXlw+37rwnlwDSrMccS/7hF8ZwHUjgU0JptMMmC7rneFrdXEdTf37wd490kdtx0dEVgD8mGbtSQpweznWHUCNy+bW6tvUyTCsg/OxIv7j6R2DhsE4LlVl2CeJTml8xqSZGikgkNHT2D5WbOw/bXDDW6+phm0SRssFaiWfihJtdQweqWzR093Ce8eG6tdt4vZLKpAiTvjtmXBUM+LS1/Ud+T30jMRfA9GKlUUyHNaEeJkZgndcVzvi04r9FseXJKnqpi14Du78rFdOGVaF0ZGq+iV64Y27SsKrhqIa0qhOP2waeD+z9sBFkA+muFhUiDCQ9v3170Iy9dvjTXIFonwtU+fC8C8AB8kK1txpTqOPQffwav3XN7wmeml9ZeHMNnKXQa0AgHFAjk5aCiCWRmC62thZjPbOkNQsCY1kZjifJSW4BrD42oK1L0Hfm15XIiaaS9q0lM/tgKGLsdRMWsN5VQmRO37HBqp4PEdQ6kFs4ZlSvf3Lcu1mrRir1oNp+LxMX/V06mUUnCFAPzWWbPw3KuHEh0DFOoFDeCkN59JI0mDr19/HoDGFxSoLwGuZtO9hhc4bN1HUSDPzLf0zFm449GdTvdBCQh1blsetqC3meqb/1p0KKHRIxfkTeWqg3nllFefizfazcu86w7LS+dyTX5c3wNdWiGnqsEGgpOSsIzb91yz2PjdufQ1S7JOKWT6jly+XxOtSMXDAshHlnngXKPVszjvhBA1F+ZgKYC0mT6liAmBhhdPV4ZAUSoSpk/pqnk4Re1nFDd5nXYytatgHMCDAjLKAFsqEiBQ57UXHGR1eeWCbt0A6sqSK/Pm0jNnafticggBPAHXPaXLOnt3fQ9sAvrOR+3xaiaCuQDVRMbkBZe0r1mSZcYCl1yKUWEBlAJJBFD/4JDzjCoKaq2hFXfa/+K1MtFq1gLYtJ7kwszuEo5VJ0Jn21EGPBtqkLAdyz9xMA1ctkFIZ/YME4gKVyEbHOz8A+60UgEVW3CYgaiCIm5f250sNKxWCCBeA/LhWvYgCqomTqs0IBVH4VKQzPYSz5Qz0KgOCoqsrz3J0UdGq7j3+vOMJj+/bT2NgD91DNux1P2yuTHbFrrVIOQ3FY6NC2PZ9GAQbaU6XntmZ0onDb/g8td6UiZW/4AYR/gA9c+ri8agPreZIdvZTdmE61pU3mEBFCDtOj/qlY0yAPeUSzg+Zp6RR8E/kJm0BH8pZtO1j4xW62amNi+6dkO5vz+36hKjbX1opOJ5ZiVRtSRqIXpGueQk0CvV8bogUyWUerpL2nUo/0L3MZ8gMHXbX0cqWG+oXCrWMk2o58N/C4ZGKrh94840bkvtnOq4Uby6gjF6wXW+PA3MaZnmOqEmEgugAFHS0GRBuVSsc0F2Nd2VCoRSkTAqBxzdOoBA4/ipZrIqYt0khKJ478QtMR12zGmlQurZIID6QF+bpihq/8RHzcb7B4fwzvEx5/10msvUrkLDvfbP9l1d2AtEdZpP8DwbtuytJVM1mf2y0G9dMwyMnhjT5n7Lo9nNFuoBtL9GExUWQAFcyh5kAQEND536358ux4RKJKowaSj+aq7BmexdT+zWOgsETRi2gLfuUgF/es3iSJ5ZOsqlAmZNn1r3Mj42sD+Rx6ANNdhlMQHxz8YvXjg7tWfrSOWk6TCuCztwUgCbrtl/nCT9jqMluWQYcN03D5iE/NrNe+qsHu0a1xMVFkAalGrbzEV728JrmPVu+VmN8Ri22JHnVl2C8+7+QYOAqFTHse3lYWvpZgDWuKhKdaJWIyiuACoVCPdc85GGa7oj5fW5IOq79l9/GjN7vykojnAzDdxzespWM0yUmBz/mo/uOIq4a5lxTXQuGQZc980DJqGoe1eUYOpkAcSpeCy41rnJmiOWgXz5WbPw0P/9mw3tpkFC1XKxpYwJq2Njm1mqlz7u7NNfFkKhUq00w4VDzTrV9aeFKq8RR7P6rbNmRaquqtDWGLIkbVVrPrbzRBE+M7tLtTQzYXspE7Lt3ID7c6XT2pOWUEiDqEJxpFJtyxQ7rrAGhEabsj91R9xBT5cC5PDR47U1Gj9TNELO3ydT+YLenrJW+KjPTC66Ng1mRjm8do5tZq3yqYWVXAiiC8AEkgU2xiG47jDTsNAfFVXjKQ6/+HUlVCvVoctSffSEed3J74yiO0//4JCzBhRcf7FZE9R5/X2NmmGgp1zC9Kn6+Ka0SiikgSkriG19s92yG0QhtgAiojMAfA/AvwMwAeA+IcQ3iGgWgI0A5gH4BYBPCyEOExEB+AaAywGMAvg9IcSL8li3AFgtD71OCPGAbD8fwHcBlAE8A+C/iZQDl3QPZ5K6P4B58dMUZ3RiXNSl19d5IwUJmwHbclbZTFkOxUONpqQCnTQlRDXTHDO47WaZC86EX1Do6v9ERd33uGs/fieRqAOR30S3fP3WUFdlk0lPPZMu36vu2QxLx+Tvrw3TcVQZER15Sltjcp8GzOm08riWlRZJNKAxAHcKIV4kovcA2EFEzwL4PQA/FEKsJ6JVAFYB+BKATwA4W/5cCOBbAC6UAmsNgKXwzMQ7iGizEOKw3OY2ANvhCaDLAHw/QZ8byGKA85ebDmpXptf3we37se3l4dpApetTMDgRQC2X1oxyCdXxCRw94e3XUy7ho3Nn1CUHVQkkw9ytdQSvQ6X08XsiJdEUVNbm4EAb9vIVC4QbLzij1pceTcxKVIJmkq6IeeYAfSCpa40lHVFn7TpXX9u9DAtgtD2Tyz44U/uc+UkatxLMijC1q+BUGwiwJwb1p3xS2p2r67ZLGiUdJiFvSu+Ux7WstEgtEwIRPQngf8qfi4QQbxHRaQB+JIRYQER/KX9/WG6/F8BF6kcI8Z9k+18C+JH82SaEWCjbb/RvZyJqJoQs4lmUBhTHfFQq2gc7lVVBmVOiDIxqxgnAmPHBNb9XcPaaZh4918wDuoFidf9u/P/P76+5oJcK9mqttnObUuW4QECdh1oaghGwZx/wT0x031eBUJug6I5rG3Rt36/OFTytnGdA8qh/0zNki7cLO36UNEquZJ0/Loy2zYRARPMALAHwPIAPCCHeAgAphN4vN+sF4C+qc0C22doPaNp1578NnqaEuXPnRup7FhkK1IwrjnYVJlDUixTHw8wf06FzaY5a48VvwjBXtDTnJQvrZ9+SXm2lz1KRUCpQLQhy7eY9WHvVIgy8cajBfOoifHTaSpJ8ZgAAqjeppBW/5M8+ENSowori2ayrQQ0rKNhMQbOqHk3wXGmat5Ka0C5eOFtrVre9Q/5KxLpzbNiyVzuZUNVx/etPNi0pzLLQ6bFAiQUQEZ0C4HEAtwsh/o3Miwi6D0SM9sZGIe4DcB/gaUBhffbjOsD0OEatAydV5lblXbOhzA4//cXhhs+i1njxt+sEBRBd+AT7qav0OTYu6gS1quQaV1hMCFHn8RZlvcNElpmHhkYqxnWpSnXcOOkJ65K/9Ih/dj80UkGBPE+14KBrukdpPvuuVUhNbHt5ONZ5/QHKUTxB/ZkldCXkV27aVfs7uP6cZumIdiCRGzYRleAJn4eEEE/I5l9K0xvk/2/L9gMA/PV/TwdwMKT9dE17qvQ4eH3N7C5h55pLnY7n1yLCqjq2ghnlEv7w0Z3awevpl97S7hNWBdIkKJIgANz1xEvawVR3nuqEiC3sksSatIo4ThEuz+PBkQrWbt7TIGgmhKd5urwvrudyxbUKqYkki/imemC2c6vPwrSkZtQfyzuxBZD0avsOgH8WQvy/vo82A7hF/n4LgCd97Z8jj2UAjkhT3RYAlxLRTCKaCeBSAFvkZ+8Q0TJ5rs/5jpUaYe8JAbjiI6c5+eKrsrqAZ3duRfLRMI6eGDMO1CYz0coVCxriR0oFwsULZ2P5+q24fePOTAbsuAkto3L0+Fjd9xs2YFlCaXLBzO6SNp7nxgvPaGgPMqenbNT0R6sTeOeYW/qg4LOfJA5HF88U5gXqP5+t1L0LuudB904AnpBW/QrTkuJodnmJZ0qLJCa45QA+C2A3ESlj9/8AsB7Ao0R0K4D9AD4lP3sGngv2Pnhu2L8PAEKIQ0T0FQAvyO2+LIRQixN/gJNu2N9Hyh5wgNnrSyEAPL5jCH/7ov2LVqWumx23EpXY7sSBd606IRK7q+eFkUq1ztQSlkGAVDW9GKjbGGXvKAlyCaglD/WvPUwrFbD0zFlYeuYsbVJRtW/YeVwnVb0+DSFpHE5UDzqXMIYo6LQdXRbu4PqO7TmymelN2lWe4pnSIrYAEkL8E/TrNADwMc32AsDnDce6H8D9mvYBAB+O20cXXNKVuAiTOT3lWAvXZ79/OvYNH810zcCVcslTiHXJHpPEwbQD/kXtsJQ54wk92V5ff4U2FZKOIpFzbR8CcNOyuTUnAn/QqVp72HDduQ0F32yZ0uMQ1E6SOhFEzR4dFsag80g0FU20aVph2ahXrlhg9JSzeSuazpeneKa0mPSZENJKPnnxwtmxFq73vX20JYXqdEwrFSMle+w0lOkjy4S0ana79qpFTi7e40JYgxdNA/PdT+1pmDRUxwX++G93N+yT9DpLBcIp07qMZceTOBHEmfWbjut3NjEJNaUhpuGF5qIlAe6aXVJnjDzCFVFRPxOMg0oDEnf/NGefQLKEkVESWHYauhioNOObAO9ZUVH7QRddHUUivHrP5c7Hj/oshz176pk4enzM6Ib9tU/b415M2dxdyiXEKT2dRbnqPJD1dbUiDoiTkQK15Ju9jl41fkpFwtqrFiWahaQ9BYhr855WKrT1bCoJ/gqf/oXdtKPQ1XpT/+AQ+pb0YueaS/H1688zbh/2Xa7u342z7noG81Y9jXmrnsbtG3dGmkDYju73ZPvkuadpHQHChE//4BDe1Tgu5/WOFgAADihJREFU+M1QNuLM+l2cFvK8mG/qWxxnjLzDAsiHP4WOC9OnFLHhunO9P3LuGeVCpTrhlIy0EwkGc6qXPuoz4ULQ1bZvSa/RvVmty+lY3b8bD27fn5m35bjwSnir+JRrz+9Fb0+5luHaJV7F5Io8fUqXU6obkwebbWLQt6QX91yz2NjXm/7qJzVBra7P/523EmVy1PUt7LrakUm/BuQnasDahAAG3jjkxcB0iCUzh6FLTce/sPt3u/SxUUkJzuBN971SnagNPkEefv5NzR7J0ZlwK9VxPP3SWxj8E7d4OIVJU7GVGAHswcAus36Tg8Dq/t3aoobBxXzTGlFa5bRNhDkadEIZbj8sgHxENT9VquN4+Pk3cxnvE5cwt/QgUTJEtIq4lThtdZOSIoBaSh3AnqrHNDBGuaaecgmL5rwHP371kHU/m8v34dGTtWlcB2HTmqJNgwnzJp1m0QrDsAltfwYDneODmmxm6QbdiY4GNlgA+eiJUfulk4TP9ClF9HRPibSGMH1qF4jSy3WWNqouU9C9Now5IXWT0mBopIKVj+0KNd8OSWGYpMz5SKUaWs5caRa2wT9q6WhbWRAdLmmQDo9WYw/8tuP6MxjotBDdZDNtd3KbwM5a+2oFvAbko4NkSWSKBcJXf3cxVq5YEGk56+BIJdf37blVl2Bd32Lcc81izOx2W99SA2QzZp3VCREaY0Xw4kWy0MaKRA3rCbZBeqRSjZQ+Juq6hWsaJNeUNcEFfVsWi9ETXjFF0wTMdF/8z4nJgcC2tuPH5GigwjzyuG6VBNaAfITZpaOQIFi+Jdx4gZeOb8OWvZFMO+3itq3qIIVpav6yBFnEAcVBwC0YOg66+jdRMi8obMI6yrpFFKEftq3OlGaacRcctHhTeIM/J6IpZsk1iNQU89WJQagAa0B1JHW5VTO8r19/Xvq+1Rmz8advYuWmXXUDj5os9pRLmD6lMYeYqyttWhSJ0B3B/h/0LAsbVH+x/go8t+qS2gutm43mkdozF5PgbHrligVepoUApQIZtci03NWjHCdsW92gPQHPs1C5mBeJMH1KMTSRbalI2lx6fnPi3U/tMQqJKGs7Kizkdd/z2KlrQyyAfCQZTHt7ynUPTLtVMdSZggS869q55lJ89XcXNyRfrI4LYxnhLJgQAn96zUe0SSCDlApefJYfW4ZmXQyYznwURQC6ohvsXfGX0tZNElzxm7T6lvRiw3Xn1gmbcqmAU6Z14fBotcFEm2Ysik7olwrUcI9czmkanI9VJ/DqPZfjF+uvwKv3XI5RQ5E+P9OndNVMuTpzYv/gkFGDGpKajA7XccK03YxyKbfxTC6wAPIRpsqabpZOE8jD7LlIJwfduEOcv7he0mqeSZnTU/YGx0+daw0aLhJhw6caAyRtaxu2fF/+2eg155+u3S4uM7tL2HCd/XqmTylqn6WZ3aW69ZSv/u7iRMIsOGB3T/Es9ATPHVwNsP5iXWnHouiE/oZPnVu7R1HiX1wHfRchoMzzOu0EgHU9SuXzSxJEahLMR0+MtfW6EK8BBTC5FfeUS0ZvL11Qnc6WO3pirKneYu+ZdjLtC+AF4IV5QgVRL2ccVT9ONVQbqmyCWlOIWsLYtLYxs7vUsL2p1PXjOxpf7mKBYiUoLZeKtbxgtRLggUJzpaLnHAKEuz4Hn7me7hKOVKp130GpSJg+pUv7jJvWMnRXprTjLFLbmNaMogo5Vw88l3yQYULK9n6MC1EzB6p1pLAS6EFcx5N2WxfiXHABdFUMSwVvRn3Hxp3al5GAuoqatmM302QFnIyB6fUNojp33lKBAKov1+AfzE15qHQUybOX+xM7ziiXcDSFrNpBARPFNdX23fr3MQm2qV2FyJMTE/6ccME+pulqaxKkNsHt+l27PvetxHT9pjZdZnDbpEZhu2dxjueCKU9h3O+lFbngWAMKYKs9YvKKcrXj9i3pxWMD+7VayPKzZuEXv/aKVBViJhPVEUwxc881i7FzzaXOL6Z/QT5slqibEQcH9qSeZcEZXuTI8KCFSmOxMnkcma79SKWK19df0XBPbdc5fao+FU3ake6245m+a1dttx3WOYPXb/JUu+eaxQ1lKqJMAkylF4BGDTItLSVOkG/eYA0oAlFNPiZW9++uBbUpbWFd32LreUpFSqUmTxKzia1+TJT7kLRoX9wZnms24agZsE33NGxWnFftwUUDSmsW32yyzCgdDBaeaQlsT+P7T2s8qvWJNaB8Y9OOorCub3GdwHE9TxpxKUncNv2zySSmomC9HV2qHOVCbVuriIqrK6tpZjmzu4Rj1YnQNQXFyhULjGbbPM9Sw7RdXU2bdiFLd2adtmkSeGl8/2mNR62EBVBEmpUM0HSeMM0hLO9ZWgNf0vvgIsxMM7y4Lr+uJgvT4rUqde36wvct6cXAG4fw0Pb9DdpinlPoBycIcRfO80izzVZRUxFFpd2Tk7IJrs0IM4Nde34vtr08nNhMlhfSXJSPYrJI+7ztPEvtJNI2W7mesx2+/1aY4FgAtTFhD3a7PPjNhO8Jw8+AHhZAGojoMgDfAFAE8G0hxHrb9pNJADEMw6QFl+QOQERFAH8O4BMAzgFwIxGd09peMQzDMGmQawEE4AIA+4QQrwkhTgB4BMDVLe4TwzAMkwJ5F0C9APwlDA/ItjqI6DYiGiCigeHhaGW1GYZhmNaQdwGky6zYsGglhLhPCLFUCLF09uzZTegWwzAMk5S8C6ADAM7w/X06gIMt6gvDMAyTIrn2giOiLgD/AuBjAIYAvADgM0KIPZZ9hgG8EfOUpwL4lfxf0cy/W3lu7kv++8L3IX99yeN9+BXicaYQoqkmpFxnQhBCjBHRFwBsgeeGfb9N+Mh9Yt9AIhoQQiwlopofdzP/buW5uS/57wvfh/z1JY/3odmu1EnItQACACHEMwCeaXU/GIZhmHTJ+xoQwzAM06HkXgNqMvcF/g+2N+PvVp6b+5L/vvB9yF9f8nwfck2unRAYhmGYzoVNcAzDMExLYAHEMAzDtISmrgER0f0APgngbSHEh4loGoDtABYAmBbYfAye6/V4s/vJMAzDaBE4maFG/S4AvANvvJ4C4ASAIwDeBfBFIcQW08GarQF9F8Blvr+PA7gWwE3wgqdG4Qmc2+Bd1HEAO+XvPwTwS9++xwD8WH6mFrIqvv0Q+AyB3xmGYSYjtjFR95kaRyfgCZwKvDFW/X4IwHvgKQo3yf/HAVwO4H/JqgZamiqAhBD/CK+z6m8hhHhVCPEEPOEDAFV42lAB3gWrKN/vAvi3wCFnwbsJSiKX5O9H5d/BXHK2itVJhRMLN4Zh8kDYWESB3ycCfwvNzxi8MVnAG6Or8ITMVAD75L57AdwK4EfwFIT3y88uMHWk5WtARFQkop3wcr6V4Amf98G74GkAzoQnnP4CwIfgXTTkZwtRf7OVqW6W/xTBU5q6Iv+PK0hMx2UYhmkmUceiCU0bwZMP6v+gkOqSbePwTG7K8jQHwOvwxu9eGCoYKFougIQQ40KI8wAslf05BOAz8CTrRni2RAB4Gd7FKHXuKXhS2H+zx2HmhGOXbF8eazkMw+Qd3TjlYmpTTKB++UIJGj/+cdJ0PmH5HEAOBBAAEFEJwD3wFrIETgqdOfAExzCAD8OzMyqm4KSTgkIJmeAF/7PcPnFXUzgGwzBMlpg0GtPfwTUaJRf8QqTk+1yZ4Qpy36nyeFPhVSuYL7c/iJAKBi0VQEQ0m4h6ADwAT20rwlPdFshNzpFtc+X/R327/0d4dka1D+CZ5fxeGop5hi7YNCaGYZh2xLjoLwlO0AuBz0jzo0xuBE+4KAXgBICz5b4LANwP4CIAZQBvy89+aupIUzMhENHD8Dp3KjyPtr+C5zVxtmU31UHWPhiGYVqLboIP6N2w3wFwuxDi+6aD5TIVDxF9G5469zsA7oDnfLBP/v1teGtBPwHwX+TPEICPA/iyEOI7RPQsgBkAlgB4DJ4XxivwpPi5ALYC+C0Az8v/S/KztwEshufNcQo8Z4ZbAPwegOkAPghgEMBzsj/rZH/uhGceXCfb74E3AzgF3trV2fA8+N4EsBnA7bL/OwBcAk9DmwevjtFfA1gB4Ddkn4cAXAHg7+A5XfyrvIZX5fFmy2s9BG+xTwD4me92Xg1PXf6MvFe74bm5P+O7L/Nl/94rj3dY3vf74Jk/z5H34esAHofnJFKV+78f3gP3kuzHDHhq90XwJhibAXwC3uzpNAAPyXu8G8D1qI/xeo/8Lr4N4N/Jc/w2gH+C9/2ekPf0LgBfk/f2T+E9H38kj/VH8NYTx+TPBQBelPtPk9f2Crzv8n3yGIvhvTzTZf/Ok/fkSQD/IoRYT0TvAOiW9+BKeM/BOfCepRcAfFYe97PyXv69/J6+AuCP4c0yXwewTF7nDHimiZ8D+PcAHoH3Yl8EbzH3/5DX+iUAN8q2AoBN8nucBi9EYba8lh/DeyZ+B96LPw/A9+HV0vpbef9/V97fX8F7Xs6EZ0U4BV6Yw6i8Z78t7+mg/B7GAVwMYABefa6PAvi17Ntfwov3eAfAP8j9b4D3jnwcwP8nPzsE4MvyXP8C77laCG8t9xp4z/4cAD8TQqyQ7/BcAB+A954ugvfOfBTeIPe6vMYDAL4Jz/vqNHjPzGH53c6H9x6+JH//rLwHYwD64L0rr8N7Rr8I7zk5XV7nb8NzjHpL3rtReR1vwuPfy3uzWQhxgxyz3gXwBQD/AcCD8vv4oLy3/yTP/9fy3i6W9+CA/Pmi/Pm/AHxbCLEdk4BcCiCGYRim88mFEwLDMAwz+WABxDAMw7QEFkAMwzBMS2ABxDAMw7QEFkAMwzBMS2ABxDAMw7SE/w2XxNmQGP4MmAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "ax.scatter(df[\"saledate\"][:1000], df[\"SalePrice\"][:1000])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7fde780ac990>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAD4CAYAAAAgs6s2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAapElEQVR4nO3df7RddXnn8ffHRH5ZIYkJmCZpb9As2sDSIdxCrJ2OYwQSsARnSSeMa0gxbWYwtNpOlyTiMo5KV6iOIFNFIqQkDBIw/iDDj8nESOt0LQhcQH4EiLmGFC5QcjHhR0XB6DN/7OfCzs259557ss85914+r7XOOns/+7v39zn75Nwne+/v2UcRgZmZWZXe1O4EzMxs7HFxMTOzyrm4mJlZ5VxczMysci4uZmZWufHtTmCkmDx5cnR0dLQ7DTOzUeXee+99LiKm9I+7uKSOjg66urranYaZ2agi6Z9rxX1azMzMKufiYmZmlXNxMTOzyrm4mJlZ5VxczMysci4uZmZWORcXMzOrnIuLmZlVzsXFzMwq52/oV6Bj+a1t63vXqjPb1reZ2UB85GJmZpVzcTEzs8q5uJiZWeVcXMzMrHIuLmZmVjkXFzMzq5yLi5mZVc7FxczMKufiYmZmlXNxMTOzyrm4mJlZ5ZpWXCStkbRb0sM1lv21pJA0Oecl6QpJ3ZIelDSn1HaxpB35WFyKnyTpoVznCknK+CRJm7P9ZkkTm/UazcystmYeuVwLzO8flDQDOBV4ohReAMzKx1Lgymw7CVgJnAKcDKwsFYsrs23fen19LQe2RMQsYEvOm5lZCzWtuETED4E9NRZdBnwSiFJsIbAuCncBEyRNBU4HNkfEnojYC2wG5ueyIyPizogIYB1wdmlba3N6bSluZmYt0tJrLpLOAp6KiAf6LZoGPFma78nYYPGeGnGAYyLiGYB8PnqQfJZK6pLU1dvb28ArMjOzWlpWXCQdAVwMfKbW4hqxaCA+LBGxOiI6I6JzypQpw13dzMwG0Mojl3cAM4EHJO0CpgP3SXo7xZHHjFLb6cDTQ8Sn14gDPJunzcjn3ZW/EjMzG1TLiktEPBQRR0dER0R0UBSIORHxL8BG4LwcNTYXeCFPaW0CTpM0MS/knwZsymUvSZqbo8TOA27OrjYCfaPKFpfiZmbWIs0cinwDcCdwnKQeSUsGaX4bsBPoBr4BfAwgIvYAnwfuycfnMgZwAXB1rvMT4PaMrwJOlbSDYlTaqipfl5mZDW18szYcEecOsbyjNB3AsgHarQHW1Ih3ASfUiP8UmDfMdM3MrEL+hr6ZmVXOxcXMzCrn4mJmZpVzcTEzs8q5uJiZWeVcXMzMrHIuLmZmVjkXFzMzq5yLi5mZVc7FxczMKufiYmZmlXNxMTOzyrm4mJlZ5VxczMysci4uZmZWORcXMzOrnIuLmZlVzsXFzMwq17TiImmNpN2SHi7FvijpMUkPSvqupAmlZSskdUvaLun0Unx+xrolLS/FZ0raKmmHpBslHZLxQ3O+O5d3NOs1mplZbc08crkWmN8vthk4ISLeBfwYWAEgaTawCDg+1/mapHGSxgFfBRYAs4Fzsy3ApcBlETEL2AssyfgSYG9EvBO4LNuZmVkLNa24RMQPgT39Yv83Ivbl7F3A9JxeCKyPiFci4nGgGzg5H90RsTMiXgXWAwslCXg/sCHXXwucXdrW2pzeAMzL9mZm1iLtvObyUeD2nJ4GPFla1pOxgeJvA54vFaq++H7byuUvZPsDSFoqqUtSV29v70G/IDMzK7SluEi6GNgHXN8XqtEsGogPtq0DgxGrI6IzIjqnTJkyeNJmZla38a3uUNJi4IPAvIjo+6PfA8woNZsOPJ3TteLPARMkjc+jk3L7vm31SBoPHEW/03NjScfyW9vS765VZ7alXzMbHVp65CJpPnARcFZEvFxatBFYlCO9ZgKzgLuBe4BZOTLsEIqL/huzKN0BfDjXXwzcXNrW4pz+MPCDUhEzM7MWaNqRi6QbgPcBkyX1ACspRocdCmzOa+x3RcR/jYhtkm4CHqE4XbYsIn6V27kQ2ASMA9ZExLbs4iJgvaQvAPcD12T8GuA6Sd0URyyLmvUazcystqYVl4g4t0b4mhqxvvaXAJfUiN8G3FYjvpNiNFn/+C+Ac4aVrJmZVcrf0Dczs8q5uJiZWeVcXMzMrHIuLmZmVjkXFzMzq5yLi5mZVc7FxczMKufiYmZmlXNxMTOzyrm4mJlZ5VxczMysci4uZmZWORcXMzOrnIuLmZlVzsXFzMwq5+JiZmaVc3ExM7PKubiYmVnlXFzMzKxyTSsuktZI2i3p4VJskqTNknbk88SMS9IVkrolPShpTmmdxdl+h6TFpfhJkh7Kda6QpMH6MDOz1mnmkcu1wPx+seXAloiYBWzJeYAFwKx8LAWuhKJQACuBU4CTgZWlYnFltu1bb/4QfZiZWYs0rbhExA+BPf3CC4G1Ob0WOLsUXxeFu4AJkqYCpwObI2JPROwFNgPzc9mREXFnRASwrt+2avVhZmYt0uprLsdExDMA+Xx0xqcBT5ba9WRssHhPjfhgfRxA0lJJXZK6ent7G35RZma2v5FyQV81YtFAfFgiYnVEdEZE55QpU4a7upmZDaDVxeXZPKVFPu/OeA8wo9RuOvD0EPHpNeKD9WFmZi3S6uKyEegb8bUYuLkUPy9Hjc0FXshTWpuA0yRNzAv5pwGbctlLkubmKLHz+m2rVh9mZtYi4+tpJOmEiHh46Jb7rXMD8D5gsqQeilFfq4CbJC0BngDOyea3AWcA3cDLwPkAEbFH0ueBe7Ld5yKib5DABRQj0g4Hbs8Hg/RhZmYtUldxAb4u6RCKP+bfjIjnh1ohIs4dYNG8Gm0DWDbAdtYAa2rEu4ATasR/WqsPMzNrnbpOi0XEHwAfobj+0SXpm5JObWpmZmY2atV9zSUidgCfBi4C/h1whaTHJP2HZiVnZmajU13FRdK7JF0GPAq8H/ijiPjdnL6sifmZmdkoVO81l78DvgF8KiJ+3heMiKclfbopmZmZ2ahVb3E5A/h5RPwKQNKbgMMi4uWIuK5p2ZmZ2ahU7zWX71MM+e1zRMbMzMwOUG9xOSwi/rVvJqePaE5KZmY22tVbXH7W7zdWTgJ+Pkh7MzN7A6v3mssngG9J6rt/11TgPzYnJTMzG+3qKi4RcY+k3wGOo7gj8WMR8cumZmZmZqNWvUcuAL8HdOQ6J0oiItY1JSszMxvV6r1x5XXAO4AfAb/KcN8vQJqZme2n3iOXTmB23mDSzMxsUPWOFnsYeHszEzEzs7Gj3iOXycAjku4GXukLRsRZTcnKzMxGtXqLy2ebmYSZmY0t9Q5F/kdJvw3MiojvSzoCGNfc1MzMbLSq95b7fwZsAK7K0DTge81KyszMRrd6L+gvA94LvAiv/XDY0Y12KukvJW2T9LCkGyQdJmmmpK2Sdki6MX9WGUmH5nx3Lu8obWdFxrdLOr0Un5+xbknLG83TzMwaU29xeSUiXu2bkTSe4nsuwyZpGvAXQGdEnEBxem0RcClwWUTMAvYCS3KVJcDeiHgnxQ+TXZrbmZ3rHQ/MB74maZykccBXgQXAbODcbGtmZi1Sb3H5R0mfAg6XdCrwLeB/H0S/43Nb4ynurvwMxa9absjla4Gzc3phzpPL50lSxtdHxCsR8TjQDZycj+6I2JkFcX22NTOzFqm3uCwHeoGHgP8C3AY09AuUEfEU8CXgCYqi8gJwL/B8ROzLZj0U13XI5ydz3X3Z/m3leL91BoofQNJSSV2Sunp7ext5OWZmVkO9o8V+TfEzx9842A4lTaQ4kpgJPE9xFLSgVrd9qwywbKB4rYJZ8xReRKwGVgN0dnb67gNmZhWp995ij1PjD3REHNtAnx8AHo+I3tz2d4DfByZIGp9HJ9OBvtv79wAzgJ48jXYUsKcU71NeZ6C4mZm1wHDuLdbnMOAcYFKDfT4BzM3vyvwcmAd0AXcAH6a4RrIYuDnbb8z5O3P5DyIiJG0Evinpy8BvArOAuymOaGZJmgk8RXHR/z81mKuZmTWg3tNiP+0XulzSPwGfGW6HEbFV0gbgPmAfcD/FqalbgfWSvpCxa3KVa4DrJHVTHLEsyu1sk3QT8EhuZ1lE/ApA0oXAJoqRaGsiYttw8zQzs8bVe1psTmn2TRRHMm9ttNOIWAms7BfeSTHSq3/bX1AcKdXaziXAJTXit1EMOjAzszao97TY/yhN7wN2AX9ceTZmZjYm1Hta7N83OxEzMxs76j0t9leDLY+IL1eTjpmZjQXDGS32exQjtwD+CPgh+39Z0czMDBjej4XNiYiXACR9FvhWRPxpsxIzM7PRq97bv/wW8Gpp/lWgo/JszMxsTKj3yOU64G5J36X4pv6HgHVNy8rMzEa1ekeLXSLpduDfZuj8iLi/eWmZmdloVu9pMShujf9iRHyF4j5fM5uUk5mZjXL1/szxSuAiYEWG3gz8r2YlZWZmo1u9Ry4fAs4CfgYQEU9zELd/MTOzsa3e4vJqRAR5231Jb2leSmZmNtrVW1xuknQVxW+u/BnwfSr44TAzMxub6h0t9iVJpwIvAscBn4mIzU3NzMzMRq0hi4ukccCmiPgA4IJiZmZDGvK0WP4A18uSjmpBPmZmNgbU+w39XwAPSdpMjhgDiIi/aEpWZmY2qtVbXG7Nh5mZ2ZAGLS6SfisinoiItVV2KmkCcDVwAsXw5o8C24EbKW6IuQv444jYK0nAV4AzgJeBP4mI+3I7i4FP52a/0JenpJOAa4HDKX7u+OM5lNrMzFpgqGsu3+ubkPTtCvv9CvB/IuJ3gHcDjwLLgS0RMQvYkvMAC4BZ+VgKXJn5TAJWAqcAJwMrJU3Mda7Mtn3rza8wdzMzG8JQxUWl6WOr6FDSkcAfAtcARMSrEfE8sBDoO0JaC5yd0wuBdVG4i+K7NlOB04HNEbEnIvZSjGSbn8uOjIg782hlXWlbZmbWAkMVlxhg+mAcC/QCfy/pfklX5zf+j4mIZwDy+ehsP439f/GyJ2ODxXtqxM3MrEWGKi7vlvSipJeAd+X0i5JekvRig32OB+YAV0bEiRSjz5YP0l41YtFA/MANS0sldUnq6u3tHTxrMzOr26DFJSLGRcSREfHWiBif033zRzbYZw/QExFbc34DRbF5Nk9pkc+7S+1nlNafDjw9RHx6jXit17c6IjojonPKlCkNvhwzM+uv3qHIlYmIf5H0pKTjImI7MA94JB+LgVX5fHOushG4UNJ6iov3L0TEM5I2AX9Tuoh/GrAiIvbkkdVcYCtwHvA/W/YC3yA6lrdnZPquVWe2pV8zG56WF5f058D1kg4BdgLnUxxF3SRpCfAEcE62vY1iGHI3xVDk8wGyiHweuCfbfS4i9uT0Bbw+FPn2fJiZWYu0pbhExI+AzhqL5tVoG8CyAbazBlhTI95F8R0aMzNrg+H8zLGZmVldXFzMzKxyLi5mZlY5FxczM6uci4uZmVXOxcXMzCrn4mJmZpVzcTEzs8q5uJiZWeVcXMzMrHIuLmZmVjkXFzMzq5yLi5mZVc7FxczMKufiYmZmlXNxMTOzyrm4mJlZ5VxczMysci4uZmZWubYVF0njJN0v6Zacnylpq6Qdkm6UdEjGD8357lzeUdrGioxvl3R6KT4/Y92Slrf6tZmZvdG188jl48CjpflLgcsiYhawF1iS8SXA3oh4J3BZtkPSbGARcDwwH/haFqxxwFeBBcBs4Nxsa2ZmLdKW4iJpOnAmcHXOC3g/sCGbrAXOzumFOU8un5ftFwLrI+KViHgc6AZOzkd3ROyMiFeB9dnWzMxapF1HLpcDnwR+nfNvA56PiH053wNMy+lpwJMAufyFbP9avN86A8UPIGmppC5JXb29vQf7mszMLLW8uEj6ILA7Iu4th2s0jSGWDTd+YDBidUR0RkTnlClTBsnazMyGY3wb+nwvcJakM4DDgCMpjmQmSBqfRyfTgaezfQ8wA+iRNB44CthTivcprzNQ3MzMWqDlRy4RsSIipkdEB8UF+R9ExEeAO4APZ7PFwM05vTHnyeU/iIjI+KIcTTYTmAXcDdwDzMrRZ4dkHxtb8NLMzCy148hlIBcB6yV9AbgfuCbj1wDXSeqmOGJZBBAR2yTdBDwC7AOWRcSvACRdCGwCxgFrImJbS1+JmdkbXFuLS0T8A/APOb2TYqRX/za/AM4ZYP1LgEtqxG8DbqswVTMzGwZ/Q9/MzCrn4mJmZpVzcTEzs8q5uJiZWeVG0mgxsyF1LL+1bX3vWnVm2/o2G2185GJmZpVzcTEzs8q5uJiZWeVcXMzMrHIuLmZmVjkXFzMzq5yLi5mZVc7FxczMKufiYmZmlXNxMTOzyrm4mJlZ5VxczMyscr5xpVmd2nXTTN8w00ajlh+5SJoh6Q5Jj0raJunjGZ8kabOkHfk8MeOSdIWkbkkPSppT2tbibL9D0uJS/CRJD+U6V0hSq1+nmdkbWTtOi+0D/ltE/C4wF1gmaTawHNgSEbOALTkPsACYlY+lwJVQFCNgJXAKcDKwsq8gZZulpfXmt+B1mZlZanlxiYhnIuK+nH4JeBSYBiwE1maztcDZOb0QWBeFu4AJkqYCpwObI2JPROwFNgPzc9mREXFnRASwrrQtMzNrgbZe0JfUAZwIbAWOiYhnoChAwNHZbBrwZGm1nowNFu+pEa/V/1JJXZK6ent7D/blmJlZaltxkfQbwLeBT0TEi4M1rRGLBuIHBiNWR0RnRHROmTJlqJTNzKxObSkukt5MUViuj4jvZPjZPKVFPu/OeA8wo7T6dODpIeLTa8TNzKxF2jFaTMA1wKMR8eXSoo1A34ivxcDNpfh5OWpsLvBCnjbbBJwmaWJeyD8N2JTLXpI0N/s6r7QtMzNrgXZ8z+W9wH8GHpL0o4x9ClgF3CRpCfAEcE4uuw04A+gGXgbOB4iIPZI+D9yT7T4XEXty+gLgWuBw4PZ8mJlZi7S8uETEP1H7ugjAvBrtA1g2wLbWAGtqxLuAEw4iTTMzOwi+/YuZmVXOxcXMzCrn4mJmZpXzjSvNRrh23TATfNNMa5yPXMzMrHIuLmZmVjkXFzMzq5yLi5mZVc7FxczMKufRYmY2IP+0szXKRy5mZlY5FxczM6uci4uZmVXOxcXMzCrn4mJmZpVzcTEzs8p5KLKZjTi+Wefo5yMXMzOrnIuLmZlVbsyeFpM0H/gKMA64OiJWtTklMxsFfFeCaozJIxdJ44CvAguA2cC5kma3NyszszeOsXrkcjLQHRE7ASStBxYCj7Q1KzOzAYy1QQxjtbhMA54szfcAp/RvJGkpsDRn/1XS9n5NJgPPNSXD5nC+zTfacna+zTfacj4gX116UNv77VrBsVpcVCMWBwQiVgOrB9yI1BURnVUm1kzOt/lGW87Ot/lGW86tyndMXnOhOFKZUZqfDjzdplzMzN5wxmpxuQeYJWmmpEOARcDGNudkZvaGMSZPi0XEPkkXApsohiKviYhtDWxqwFNmI5Tzbb7RlrPzbb7RlnNL8lXEAZcizMzMDspYPS1mZmZt5OJiZmaVc3GpQdJ8SdsldUta3uK+Z0i6Q9KjkrZJ+njGJ0naLGlHPk/MuCRdkbk+KGlOaVuLs/0OSYtL8ZMkPZTrXCGp1tDt4eY9TtL9km7J+ZmStmbfN+bACiQdmvPdubyjtI0VGd8u6fRSvPL3Q9IESRskPZb7+j0jeR9L+sv89/CwpBskHTbS9rGkNZJ2S3q4FGv6Ph2ojwbz/WL+m3hQ0nclTWh03zXy/gw339Kyv5YUkiaPlP1LRPhRelAMAPgJcCxwCPAAMLuF/U8F5uT0W4EfU9zC5m+B5RlfDlya02cAt1N8t2cusDXjk4Cd+TwxpyfmsruB9+Q6twMLKsj7r4BvArfk/E3Aopz+OnBBTn8M+HpOLwJuzOnZua8PBWbmezCuWe8HsBb405w+BJgwUvcxxZeCHwcOL+3bPxlp+xj4Q2AO8HAp1vR9OlAfDeZ7GjA+py8t5TvsfTfc96eRfDM+g2Lw0j8Dk0fM/j3YD+lYe+TO3VSaXwGsaGM+NwOnAtuBqRmbCmzP6auAc0vtt+fyc4GrSvGrMjYVeKwU369dgzlOB7YA7wduyX+cz5U+pK/t0/wQvCenx2c79d/Pfe2a8X4AR1L8sVa/+Ijcx7x+x4lJuc9uAU4fifsY6GD/P9ZN36cD9dFIvv2WfQi4vtY+GWrfNfIZaDRfYAPwbmAXrxeXtu9fnxY7UK1bx0xrRyJ5uHwisBU4JiKeAcjno7PZQPkOFu+pET8YlwOfBH6d828Dno+IfTX6eC2vXP5Cth/u6zgYxwK9wN+rOJV3taS3MEL3cUQ8BXwJeAJ4hmKf3cvI3sd9WrFPB+rjYH2U4n/wjeTbyGdg2CSdBTwVEQ/0W9T2/evicqC6bh3T9CSk3wC+DXwiIl4crGmNWDQQb4ikDwK7I+LeOnIabFlL8k3jKU4vXBkRJwI/ozjcH0i79/FEihuvzgR+E3gLxR2/B+pjJOzjoYzoHCVdDOwDru8LDTOvRj4Dw83xCOBi4DO1Fg8zr8r3r4vLgdp+6xhJb6YoLNdHxHcy/Kykqbl8KrA74wPlO1h8eo14o94LnCVpF7Ce4tTY5cAESX1f0i338VpeufwoYE8Dr+Ng9AA9EbE15zdQFJuRuo8/ADweEb0R8UvgO8DvM7L3cZ9W7NOB+mhIXuT+IPCRyHNBDeT7HMN/f4brHRT/4XggP3/Tgfskvb2BfKvfv42eBx6rD4r/1e7MN63vAt3xLexfwDrg8n7xL7L/RbW/zekz2f/C3d0Zn0RxXWFiPh4HJuWye7Jt34W7MyrK/X28fkH/W+x/MfNjOb2M/S9m3pTTx7P/BdOdFBdLm/J+AP8POC6nP5v7d0TuY4o7em8DjsjtrQX+fCTuYw685tL0fTpQHw3mO5/ipzmm9Gs37H033PenkXz7LdvF69dc2r5/W/IHc7Q9KEZa/JhiFMjFLe77DygORx8EfpSPMyjOyW4BduRz3z8IUfww2k+Ah4DO0rY+CnTn4/xSvBN4ONf5O+q8mFhH7u/j9eJyLMXok+78kB2a8cNyvjuXH1ta/+LMaTul0VXNeD+AfwN05X7+Xn7QRuw+Bv478Fhu8zqKP3Ijah8DN1BcE/olxf+El7Rinw7UR4P5dlNck+j77H290X3XyPsz3Hz7Ld/F68Wl7fvXt38xM7PK+ZqLmZlVzsXFzMwq5+JiZmaVc3ExM7PKubiYmVnlXFzMzKxyLi5mZla5/w/ItkxM+XPTJgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.SalePrice.plot.hist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parsing dates\n",
    "When working with time series data, it's a good idea to make sure any date data is the format of a [datetime object](https://docs.python.org/3/library/datetime.html) (a Python data type which encodes specific information about dates)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"../data/bluebook-for-bulldozers/TrainAndValid.csv\",\n",
    "                 low_memory=False,\n",
    "                 parse_dates=[\"saledate\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 412698 entries, 0 to 412697\n",
      "Data columns (total 53 columns):\n",
      " #   Column                    Non-Null Count   Dtype         \n",
      "---  ------                    --------------   -----         \n",
      " 0   SalesID                   412698 non-null  int64         \n",
      " 1   SalePrice                 412698 non-null  float64       \n",
      " 2   MachineID                 412698 non-null  int64         \n",
      " 3   ModelID                   412698 non-null  int64         \n",
      " 4   datasource                412698 non-null  int64         \n",
      " 5   auctioneerID              392562 non-null  float64       \n",
      " 6   YearMade                  412698 non-null  int64         \n",
      " 7   MachineHoursCurrentMeter  147504 non-null  float64       \n",
      " 8   UsageBand                 73670 non-null   object        \n",
      " 9   saledate                  412698 non-null  datetime64[ns]\n",
      " 10  fiModelDesc               412698 non-null  object        \n",
      " 11  fiBaseModel               412698 non-null  object        \n",
      " 12  fiSecondaryDesc           271971 non-null  object        \n",
      " 13  fiModelSeries             58667 non-null   object        \n",
      " 14  fiModelDescriptor         74816 non-null   object        \n",
      " 15  ProductSize               196093 non-null  object        \n",
      " 16  fiProductClassDesc        412698 non-null  object        \n",
      " 17  state                     412698 non-null  object        \n",
      " 18  ProductGroup              412698 non-null  object        \n",
      " 19  ProductGroupDesc          412698 non-null  object        \n",
      " 20  Drive_System              107087 non-null  object        \n",
      " 21  Enclosure                 412364 non-null  object        \n",
      " 22  Forks                     197715 non-null  object        \n",
      " 23  Pad_Type                  81096 non-null   object        \n",
      " 24  Ride_Control              152728 non-null  object        \n",
      " 25  Stick                     81096 non-null   object        \n",
      " 26  Transmission              188007 non-null  object        \n",
      " 27  Turbocharged              81096 non-null   object        \n",
      " 28  Blade_Extension           25983 non-null   object        \n",
      " 29  Blade_Width               25983 non-null   object        \n",
      " 30  Enclosure_Type            25983 non-null   object        \n",
      " 31  Engine_Horsepower         25983 non-null   object        \n",
      " 32  Hydraulics                330133 non-null  object        \n",
      " 33  Pushblock                 25983 non-null   object        \n",
      " 34  Ripper                    106945 non-null  object        \n",
      " 35  Scarifier                 25994 non-null   object        \n",
      " 36  Tip_Control               25983 non-null   object        \n",
      " 37  Tire_Size                 97638 non-null   object        \n",
      " 38  Coupler                   220679 non-null  object        \n",
      " 39  Coupler_System            44974 non-null   object        \n",
      " 40  Grouser_Tracks            44875 non-null   object        \n",
      " 41  Hydraulics_Flow           44875 non-null   object        \n",
      " 42  Track_Type                102193 non-null  object        \n",
      " 43  Undercarriage_Pad_Width   102916 non-null  object        \n",
      " 44  Stick_Length              102261 non-null  object        \n",
      " 45  Thumb                     102332 non-null  object        \n",
      " 46  Pattern_Changer           102261 non-null  object        \n",
      " 47  Grouser_Type              102193 non-null  object        \n",
      " 48  Backhoe_Mounting          80712 non-null   object        \n",
      " 49  Blade_Type                81875 non-null   object        \n",
      " 50  Travel_Controls           81877 non-null   object        \n",
      " 51  Differential_Type         71564 non-null   object        \n",
      " 52  Steering_Controls         71522 non-null   object        \n",
      "dtypes: datetime64[ns](1), float64(3), int64(5), object(44)\n",
      "memory usage: 166.9+ MB\n"
     ]
    }
   ],
   "source": [
    "# With parse_dates... check dtype of \"saledate\"\n",
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7fdf08527f90>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAD4CAYAAAAtrdtxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO29f5RdVZ3g+/nWzQ1UQUslGn1QEJNWGsZ0JIGMpDvv9QA+CUIDNSKCAw/sYQ3r9Wg/4dkZw5hlAuIQJ2Ojru7xDaM8saUhgFqGBifwDK5ewxikYiXGIBkCSEhBS7qToiFVkJvKfn/ccyqnTu29zz6/7j23an/WqlVV+557zj6/9nd/v/v7Q5RSeDwej8eTha52d8Dj8Xg8nYsXIh6Px+PJjBciHo/H48mMFyIej8fjyYwXIh6Px+PJzKx2d6Bo3vWud6kFCxa0uxsej8fTUWzbtu0flFLz0n5v2gmRBQsWMDg42O5ueDweT0chIi9l+Z43Z3k8Ho8nM16IeDwejyczXoh4PB6PJzNeiHg8Ho8nM16IeDwejycz0847y+NpNQNDw2zYvJtXRsY4pbebVSvPoH9pX7u75fG0BC9EPJ4cDAwNc8sPdjLWGAdgeGSMW36wE8ALEs+MINGcJSJ3i8hrIvIrzWd/LiJKRN4V/C8i8g0R2SMivxSRsyPbXi8izwU/10fazxGRncF3viEiErTPFZHHg+0fF5E5xZyyx1McGzbvnhAgIWONcTZs3t2mHnk8rcVlTeQ7wEXxRhE5DfgIsDfS/FHg9ODnRuCbwbZzgbXAucCHgLURofDNYNvwe+GxVgM/UUqdDvwk+N/jqRSvjIylavd4phuJQkQp9XfAAc1HdwL/DohWtboc+K5qshXoFZGTgZXA40qpA0qpg8DjwEXBZ+9QSv1MNatjfRfoj+zrnuDveyLtHk9lOKW3O1W7xzPdyOSdJSKXAcNKqR2xj/qAlyP/7wvabO37NO0A71FKvQoQ/H63pT83isigiAzu378/wxl5PNlYtfIMuuu1SW3d9RqrVp7Rph55PK0l9cK6iPQAXwAu1H2saVMZ2lOhlLoLuAtg2bJlvt6vp2WEi+feO8szU8ninfU+YCGwI1gDPxX4hYh8iKYmcVpk21OBV4L282LtPw3aT9VsD/BbETlZKfVqYPZ6LUNfPZ7S6V/a54WGZ8aS2pyllNqplHq3UmqBUmoBTUFwtlLq74FNwHWBl9Zy4PXAFLUZuFBE5gQL6hcCm4PP3hCR5YFX1nXAj4JDbQJCL67rI+0ej8fjqQguLr73AT8DzhCRfSJyg2XzR4EXgD3AfwX+LYBS6gDwJeDp4Oe2oA3gT4FvBd95Hvhx0L4e+IiIPEfTC2x9ulPzeDweT9lI0ylq+rBs2TLl64l4PB5POkRkm1JqWdrv+dxZHo/H48mMFyIej8fjyYwXIh6Px+PJjBciHo/H48mMFyIej8fjyYwXIh6Px+PJjBciHo/H48mMFyIej8fjyYyvbOjxeDwVoRNLLXsh4vF4PBWgU0steyHimbF04qzPM32xlVqu8nPphcgMpx0DaRUG706d9XmmL51aatkvrM9gwoF0eGQMxbGBdGBoeFodU4dt1ufxtINOLbXshcgMph0DaVUG706d9XmmL51aatkLkRlMOwbSqgzenTrr80xf+pf2ccfHFtPX240Afb3d3PGxxZU3r/o1kRnMKb3dDGsG7zIH0nYcU8eqlWdMWhOBzpj1eaY3nVhq2WsiM5h2qM9VUdk7ddbn8VQNr4nMYMIBs5WeUu04pq0vXmh4PPnw5XE9nhKpgjuzx+NC1vK4XhPxeErCx6J4ZgKJayIicreIvCYiv4q0bRCRZ0XklyLyQxHpjXx2i4jsEZHdIrIy0n5R0LZHRFZH2heKyFMi8pyIbBSR2UH7ccH/e4LPFxR10h5PK6iKO7PHUyYuC+vfAS6KtT0O/L5S6oPA/wRuARCRDwBXA4uC7/xnEamJSA34K+CjwAeATwbbAnwFuFMpdTpwELghaL8BOKiUej9wZ7CdJwMDQ8OsWL+FhasfYcX6LS0P7JupVMWd2eMpk0QhopT6O+BArO0xpdSR4N+twKnB35cD9yul3lZKvQjsAT4U/OxRSr2glDoM3A9cLiICXAA8FHz/HqA/sq97gr8fAj4cbO9JQVUixGciPhbFMxMowsX3XwM/Dv7uA16OfLYvaDO1vxMYiQiksH3SvoLPXw+2n4KI3CgigyIyuH///twnNJ1otUnFaz3HqIo7s8dTJrmEiIh8ATgC3Bs2aTZTGdpt+5raqNRdSqllSqll8+bNs3d6htFKk4rXeibjY1E8M4HM3lkicj3wx8CH1TE/4X3AaZHNTgVeCf7Wtf8D0CsiswJtI7p9uK99IjILOImYWc2TTCsjxDs1lXWZ+FgUz3QnkyYiIhcBnwcuU0qNRj7aBFwdeFYtBE4Hfg48DZweeGLNprn4vikQPk8AHw++fz3wo8i+rg/+/jiwRU23oJYW0EqTil9I9nhmHomaiIjcB5wHvEtE9gFraXpjHQc8Hqx1b1VK/Z9KqV0i8gDwDE0z16eVUuPBfj4DbAZqwN1KqV3BIT4P3C8itwNDwLeD9m8Dfy0ie2hqIFcXcL4zjlZGiFclL5bH42kdPmLdUxjx4Dpoaj1+HcDjqT4+Yt3TdqqUF8vj8bQGL0Q8hTLTF5Jnaq6smXreHi9EPJ7CmKm5smbqeXua+HoiHk9BzNRcWTP1vD1NvBDxeApipro4z9Tz9jTx5iyPJwO6NYCZ6uI8U8/b08RrIh5PSkzpXc4/c96MzJXlc4TNbLwm4mkrNq+eqnr8mNYAnnh2P3d8bHEl+1wm3rV7ZuOFiKdt2Lx6gMp6/NjWAGaqi/NMPW+PFyKeNpLk1VPVZI5+DcDjOYZfE/G0DduMvsoeP34NwOM5htdEPG0jaUafZ7Zf5nqKXwPweI7hhYinbaxaeYY2YWM4o7d9ZqMVEdR+DcDjaeKFiKdtuMzos8z2fXEsj6d1eCHiaSu2GX3W2X6V11M8numGX1j3TDtM6ybee8rjKR4vRDzTDu895fG0Dm/O8kw7TGstACvWbynNo6oTo++nE/4atwdfHtczIyirdG84cA2PjCFA9G2qdwknHj+Lg6ONKZ/5ssHF4ksz5ydreVxvzvLMCMqoeRFNxAiThQRA46ji4GhD+5mvt1EsvqZJ+0gUIiJyt4i8JiK/irTNFZHHReS54PecoF1E5BsiskdEfikiZ0e+c32w/XMicn2k/RwR2Rl85xsiIrZjeDxZKMNjSzdwpcF7ixWH98hrHy6ayHeAi2Jtq4GfKKVOB34S/A/wUeD04OdG4JvQFAjAWuBc4EPA2ohQ+Gawbfi9ixKO4ZnBDAwNs2L9FhaufoQV67cwMDTs9L0yPLbyDlDeW6xJ1nsaxXvktY9EIaKU+jvgQKz5cuCe4O97gP5I+3dVk61Ar4icDKwEHldKHVBKHQQeBy4KPnuHUupnqrk4893YvnTH8MxQTHU8XAadMjy28gxQ3lusSZ57GsV75LWPrGsi71FKvQoQ/H530N4HvBzZbl/QZmvfp2m3HWMKInKjiAyKyOD+/fsznpKn6pjs3jdt3J44g+1f2scdH1tMX283AvT1dudedNUNXDYk+F3EsatIFo2iqLWMMu6vx42iXXxF06YytKdCKXUXcBc0vbPSft/TGdjMRy75sYrOd6VzJR49fGRiMT1KTYSvfuKs0ga1dru3Zs1XVuRaRt772+5r2Klk1UR+G5iiCH6/FrTvA06LbHcq8EpC+6madtsxPDOUJPNRO7xx+pf28eTqC3hx/SU8ufoC1l66SGtWKVuAFGESykNWjaIqaxlVuIadSlYhsgkIPayuB34Uab8u8NJaDrwemKI2AxeKyJxgQf1CYHPw2Rsisjzwyrouti/dMTwzFBfzUbu9cdphVqmCe2tWjaLItYw8C/RVuIa6/hfhdFA2ieYsEbkPOA94l4jso+lltR54QERuAPYCVwabPwpcDOwBRoE/AVBKHRCRLwFPB9vdppQKF+v/lKYHWDfw4+AHyzE8OShKZW+H6h81H+lqjYB9BtuqPrc6TXwV3FuzVnssqjZL3vT/7b6Guv6venAHCDTG1URbVUpER/ER6y2mnXbXoqJ6qxAdnLYPA0PDrHpox8QLCVCvCRs+Xp6ZqVWsWL9FO4D39Xbz5OoLWtKHdj8Tea9Bu6+h6fg65vTU6Zk9q/AxxEesdwDttrsWpbJXQfVPaza69eFdkwQINGd4tz68qwW9LZcquLe22zsqrybR7muYRuM5ONqo1NqNT8DoQFHaQ7uLJRWlsrdb9Q9JYzbSeUzZ2juJqpTrbWe1x6zmtJB2X0NT/11od8E1L0QSKLLUarsH37wvWtH78RTHTC/Xm1Rq2YV2XkNd/+tdMmlNxEY7HUq8OSuBIk037XZnLEplb7fqn4Xe7nqqdk9n0W5zWl50/d9w5Vls+PhZk9pMz2s7J3BeE0mgSO2hiNlSHopS2dut+mdh3WWLWPXgDhpHIwvrXcK6yxa1sVeeIul0bczU/2ibyYGhnRM4L0QSKNJ0U4XBt6gXrV0vbNb1qSpce48nL1V8jr2LbwLtdl30HMPfC4+nPLK6+HpNJIEqSv5OwkVzcNUu2u3d5imOKuepqnLfqogXIg50uq21Xbh4tqXxfmu3d5unGEz3fPClAzzx7P62Dt5FemPOFLx3lqc0XDzb0ni/tdu7zVMMpnt+79a9bQ+iq0IgrYmq5tHymoinUKKmANNqW1RzSKNdlO3d5s0YrcF0z0116GdCHrKkZ6/KGpLXRDyFEU/rYiKqOZyUwu+9zFiAIlLSVHWmWDXSaI6tNlW2Q9vVPXs3bdzOklsfm3iGqqwheU3EUxi6Bz1OVHMYGBrm0OEjU7apd4lRuyhrfSrPov3A0DDrNu1iZOxYCpUqzRSrhk6jNNFqU2U7YrlM783IWGPiGaryeqDXRDyFYXugdZrDhs27tSkdTjx+VssH3qwvaTiLjAqQkKrMFKtG/9I+rjinT1vWNEo7gujaEflue8bCZ6jK64FeE/FkJm7H7e2paxMamtJpm16ekWAfrVyjyBpUmqR9VWGmWEWeeHa/1uRZE+GoUs73u4xnpNXemEnJF18ZGePOq5ZULlI9xAsRTyZ0C331LqFek0nahelBHxgapkuEcU2w6ym93S1ZSIwOQCd11537HiVJSFRhplhFTNftqFK8uP4Sp32sGdjJvVv3TgijTjUhJpn3TuntrnS8mhcinkzoZuCNo4re7jonHGcvmBMKCJ0ACQfusgML40JqZKxBvUuY01NnZLTh/JLaZpFVmSlWkbzphAaGhicJkJBODD4N+3rrw7umaPLRZ6iq8WpeiMxwspoDTDPJ18cabF97ofW7JhNQTWTC/nzzxu2pjmtDd44mIdgzexZDX7T3P4ppFjmnp87aSxdV8qWvAnkXsDds3u3kQt4phAKiE93MvRCZweQxGeWZSdpMGeFxi0p8aTpHk+kg7QDkamboxMGhTPKaZ2z3qZNNiFXVNmx4ITKDyWMyMs0kzz9zHivWb7EODC4CoihXS9M51izrMWlJevGrHCjWTvIMmKZnSMCbEFtMLhdfEblZRHaJyK9E5D4ROV5EForIUyLynIhsFJHZwbbHBf/vCT5fENnPLUH7bhFZGWm/KGjbIyKr8/Q1L9MxkCyP77nOFfKKc/r4/rbhxIA9l6JWRblams5lXKmWFdaqcqBYp6J7hgS4Zvn8GS2Y20FmTURE+oD/C/iAUmpMRB4ArgYuBu5USt0vIv8PcAPwzeD3QaXU+0XkauArwFUi8oHge4uAU4D/T0R+LzjMXwEfAfYBT4vIJqXUM1n7nJVOn0maTClF1KWOnv+K9VucNBtXU0YRqr3pHPsiayNlm5iqHCjWqVTZW2mmkdecNQvoFpEG0AO8ClwA/Kvg83uAdTSFyOXB3wAPAX8pIhK036+Ueht4UUT2AB8KttujlHoBQETuD7ZtuRDp5BTkNgFYdHRumsGyVbZf2zm2qg8ndde1wYimlC8eNzpx/WA6ktmcpZQaBv4TsJem8Hgd2AaMKKXCXBb7gPAu9wEvB989Emz/zmh77Dum9imIyI0iMigig/v37896SkY6eSaZJABDkxE0vaPCz7KY66oYVdvKCGSTyVMModmmdo+nk8hjzppDUzNYCIwADwIf1Wwarl7qXhlladcJOK1Xn1LqLuAuaFY2tHY8A6ZI7E7wAkkSgOFgWoS5rt015E20YsZq0/hGNM8OwMHRBgNDwx07m66Cx1nYh+GRsQlniT5NX6rQ1+lKnoX1/x14USm1XynVAH4A/CHQKyKhcDoVeCX4ex9wGkDw+UnAgWh77Dum9pYyMDTMm29pkgTWzEkCq4SLdlDUwm878g5VBds1tE022lEzowiKyHpcZB+ACW+7eF+q0NfpTB4hshdYLiI9wdrGh2muVzwBfDzY5nrgR8Hfm4L/CT7fopoF3jcBVwfeWwuB04GfA08DpwfeXrNpLr5vytHfTGzYvJvG0anKzQmzW58kMAsunlBFmuv6l/bx5OoLeHH9JTy5+oJc16iTPOJs13DVyjOo1/S2q0710qqCx5ktb1m0L1Xo63QmszlLKfWUiDwE/AI4AgzRNCk9AtwvIrcHbd8OvvJt4K+DhfMDNIUCSqldgWfXM8F+Pq2UGgcQkc8Am4EacLdSalfW/mbFFpndCbh4sRQV2Fck7fCIy2PySLyGFiNrJ6ytxck78SjCvJR0rPDzTlnT7FSTWy7vLKXUWmBtrPkFjnlXRbd9C7jSsJ8vA1/WtD8KPJqnj3mp4gCblqQ1Ad1ahtAcuFes39KWh7nVHnF5hZZtPcikzYZ00rMUkue9KGqCkJT9VtF0Ozd5x2W97msGdnLfUy8zrhQ1ET557mnc3r84075COjmMwNcTScDFHNTpxL20BKZkRm21Kck0ONgGjTzkNXnY1oNsM95OfZbyvBdFmZd0fYgzPDLGocNHqHdNNidmve5rBnbyva17J9ZfxpXie1v3smZgZ+p9Relkk5tPe5LATAlqCrWVFeu3TBmo2xETY0pLAkzyaCrKBFCEycOk8ZlmzNGEk51GnveiKPNStA/DI2OIgO6RaYwr5vTU6Zltzy7twn1PvWxsz6ONdIrJTYcXIg5Mp6CmpEG3Kg+zSYAAE2p++LfOBADpBrgyzJZR99OodgfNmXCnCpCQrO+F6VqH5qc0A3w0+60tsebIaCNVdmYTpufS9ry60Mlmc2/OmkG4uDpWJWCwz3K8UDMymQDWbdqV2qWzaLNl3P00GhA1k1yfddjMUFnNp0kVJot6fmuGCFFTuyvnnzkvVXuV8EJkBuFid63KGlCSvfuVkTFzed2xhpN9OepCvGHzbq44p6+wGBfdtVZAb5Dq5OaN2yvvtqyjCLfr+BpcnDRrAWF/bGtlRT6/nzz3tFTtrjzxrD7Thqm9SnhzVkF0gnuei6kqja27zHMO9/O5B3ZYU7anWWiPnqfOG+b724YL0xBsAi70FOokDxwo1oMoNEMtXP2I1vvZxXyaZMKCpoZwxTl9bNi8m5s3bs/9nIbrHkV7Z1XFjJwFL0QKoFPc81ztri627lacsy4lC0yeWeo+O77elZimpmwXYlOqnDidksgTyrlmedYCkkxY3fXaRHmCIp/T2/sX5xYacUzPS29P9ZN0enNWAXSKe16RpqpWnbPNddb02dpLF7U0Sj9KaF5xESBFHbNVlHHN8jyTtuOGz8ITz+7viHfTtC6fc72+JXhNJCNRU06n1Hou0l25leq3TTOyfZYlSr9LhIWrH8l0bVzMKzo6wQMHyvEgyvNM2mrFPLn6AqC59qSjjOc0j3nXlAGjEzJjeCGSAdfBooqDQ1HuylV3ScwSpQ9Tk/iF+3IhybwCaF19OyXYsKwszVmfSZf+tOo5zWverfr7ZMObszLgMlh00uCQBRczRJUTKMZNYToXzbRmj6TZbXe9xjXL53dsluOqZWl26c+qlWdQi0Wr17qKz8Cdx7w7MDTMobenZgrvlDHEayIZsA0WApX1ziqS/qV9DL50YJKXyhXnHJtRZpmZ5fX2Svv96Ax44epHtNsMj4w51/yw5XLS1bgoilZ6BpYZeJvlPJL6M/jSAcZjecvGjyoGXzpQ6HlkNe+arBpzeuqsvXRRR4whXhPJgEnF7OvtLiQFeicwMDTM97cNT8oh9P1twxPaRtqZWd6aD3m/b/OCcd2PSTv72lVLSnsmpkutjLLOw5ampEiyBumarBo9HVJqArwQyURVAvKKIKvJKUlIpJ2Z5fX2yvt9mxdMkvBLE7BYtImvUzwDk8h6HknXs6w0JXFMwbGjh49Y77HpfRgeGaukGViHN2dlYLokZRwYGmbVQztojB9bTF710A4geTEwSUikXSjM6+2V9/tJXjC6/aQNWCwjtibNeQ8MDbNu066JYMcqmUyy3D+X62lK5OmSpiSNeS1sj15faJZAtt1jmwk0qpGZvl8FvCaSkSIr+LWLWx/eNSFAQhrjii/8cGfibPmkbr35J2xPq63lzdlV1vdtnyfNnuOz5HWbdhWuNbie98DQMKse3DFlgFv10I62znTDa2TSC2z3xUV7yZqmJKt57Q1NKW3bPXZJZ191zdILkRmMKSDu0OHxxJfHNJEL29N68uQ1Eeq+X+8SRg8fcTIL2F5mUz9ss2fdIKQrjGTbjwuu181UGKsxrto2QMWTVMZJuv8u2svt/Yu5dvn8Cc2jJsK1y+cnRpxnXdMzmclMfY2/JyaqFnMWxZuzPInoUluMGARQtD2NJ09eE2H8+yd11zl0+MiEoBweGWPVgzu49eFdjIw2puw/XpsiNIOEXlXQTFMe7ZvNZOfiBh7dPiuu1802CJU5QNlMQrZr5OLN5moyzZKmpIg1PVufokTfE1MyySrHi3gh0mKqlKixu97FWOOo07bxlydLcFTSued1H42/jPGZf+OomiRU4rZm0/FNtvd4XqaQNEkhWxWsZ7O9lzVAJa25mQZkgYmIcxtlBT9C8vMdf5aLyiJc5jmVhTdntZCquWMen2CLjRIfaNKan1p97i6za1dbs8m08cSz+60pzXXM6am3JVhv1cozppSIBajXig+8CzGtud368C4g/zpWmcGPtudb9yybTFFpq1dWLaDThVyaiIj0At8Cfp+mM8G/BnYDG4EFwG+ATyilDoqIAF8HLgZGgU8ppX4R7Od6YE2w29uVUvcE7ecA3wG6gUeBzyrVCSnJ9JSdOdYFl5xfcXTCIa35qdXnnjQ7DHERNjbTRtj3mww5mqJ012tt84bSeQ+V7Z1lWnML24uYdZcV/Gh7vles36KtFVNU9coyAzrLIK856+vAf1NKfVxEZgM9wL8HfqKUWi8iq4HVwOeBjwKnBz/nAt8EzhWRucBaYBnNe7BNRDYppQ4G29wIbKUpRC4Cfpyzz22j3TUDXHN+9XbXOeG45HrUaR52mz+8a0R4Gky5seK4zHptpo3wmtqoShaDIgenIsyyRbjKl13TRrcv07OsaGoOVTBVt5LMQkRE3gH8EfApAKXUYeCwiFwOnBdsdg/wU5pC5HLgu4EmsVVEekXk5GDbx5VSB4L9Pg5cJCI/Bd6hlPpZ0P5doJ8OFiLtTrLmmvNr3WXFz05tmkEZfvDhvkxFrcB91nv+mfP43ta92vaka1oT4fk7LnbsdWfgGu/S213XeqT1RtzD8wi2dtXxcckePJPIsybyu8B+4P8VkSER+ZaInAC8Ryn1KkDw+93B9n1ANNfAvqDN1r5P0z4FEblRRAZFZHD//uqWk2x3pHtSzq8y7a82F9qy/OD7l/bx1U+cpT3unJ6687naSpcmaZF5y6ZWEVf313WXLZqyDlPvEtZdtqjUftxUcunhdr/HVSOPOWsWcDbwZ0qpp0Tk6zRNVyZ0a08qQ/vURqXuAu4CWLZsWalrJnnU53ZHurdiBmW6PklrB2k8mtJQxDW3mSFtGpZLPEInYjrf+HWyXfs1Aztzl5i1CfAytRLd+tLx9S4GXzpQGc/LVpJHiOwD9imlngr+f4imEPmtiJyslHo1MFe9Ftk+Oi07FXglaD8v1v7ToP1UzfZtowj1uZ2LZmW7DyZdn/6lfVbzUhlrI9FjZ8VmhjRd06p71GTFNrvXJbHUXfs1AzsnmQfHlZr4P40gSXKeKNtp5e0jx9zjD442Jp1TJ6QrKYrM5iyl1N8DL4tIOAJ9GHgG2ARcH7RdD/wo+HsTcJ00WQ68Hpi7NgMXisgcEZkDXAhsDj57Q0SWB55d10X21RY6PdmdzX0wnqJjzUBy6pM4tusT7t+W+C7tdWxVvRKb+aITXTLzELrn6nD1mywqs+6qlWdQr9lzYJXltOKyvthJY0Me8npn/Rlwb+CZ9QLwJzQF0wMicgOwF7gy2PZRmu69e2i6+P4JgFLqgIh8CXg62O62cJEd+FOOufj+mDYvqrfbu6oIdDNDnQaRZVZl88By8ZRKcx3DXFBhKo8wIj2pj0n7tJniTKYKF02nSkGmWRkYGrbWjnct5VpoZt2Er5TltJI3Meh0IpcQUUptp+maG+fDmm0V8GnDfu4G7ta0D9KMQakE7fauKos0s6osaShqIk4pQNJcx3Wbdk3JBdU4qli3aVemwdnFFJdVULTLi6hokmbVrvcvT2bdeH90+cBCylzsdo1D6vSxwQUfsZ6C6eqVUdSsynR9XGeY5585z2k7wJjM0NSeRF5TpS0iv9PNoCG2+19PUXI2a2bdNP0p26zokn0X0j3TnYrPneVIdDCIJ+frpNmkjqJmVSazT5jUMIn7nnqZZe+dqzW3xfeZF9fcR8MjYyxY/YjVg2hgaFjrMBAKiulgBgXzcyLAhivPcn4PwmuY1zvL9bktg/iz3mXQrkzu4dMJ6eAsIlqWLVumBgcHC92nLtJ7OnnguESym87XxdbvGimvO47p2ndJM2V9nDk9dYa+eKG1X2n6Eyfutpu0rzBafToEp5X1HmRdL4p7ecVp5Tu6cPUj2uUZAV5cf0npxy8CEdmmlNItT1jx5iwHpos5woTOw+ja5fMTPY5ckyrG92+zfcevq+na12tdUzxz6jVh7aWLEmo/Fu0AACAASURBVPuVJk17nLgHkUsK8OliBi3DEy1PYs6kWX4r39GkIm3TGW/OcmC6mCNsZImlSJNUMbr/NQM7uXfrXqNjTXhdB4aGjeaK18ca3HnVEm59eNeEx9AJs2c59SvPfYubLGz7iroBh/3qZO8sKD7OKU9izjzJM/Og05ySirQVcYyqPi9eiDhwkiEH0EyYZdjIWhf7+9uGrZ6ZClh622O8qSk1GhKuz7wVqYcyMtawmpaS6r+7EGpR4UtuOo94CvCiBt92DS7hcXUFu/IcP88EzeU+Fu0dZfK0Mz1zpuJtWY4B1fTm8+YsB4qeZUwXstSDcDUlHRxtGN03wxm+aRZrMpeF/TKV0k0KXIOmB5FLWdevfsJ9odmVdtWjiZ9vqI0Vcfw8NUWSPJ/KMBlmfeaKOEZVzedeE3HANJs4ONpg4epHKq9ulkWWNCpFmBfCGf7Nhjxc40rRXa8Z+2UyL8XbFryzm60vHJziQaSrJxESn50XkSMqpNU1WaLah4m0x49rUuefOW9KdUjXwd+2JpJXSzJpfKbnN+mZS0Onmc+9EHHApjZHZ4RQTXXTRqsTSuZ1y+ztrk/s3+b1FGoqaUvxupy7raxreNybN27n+Fj54aw5opKOW5bd39WDzfX4OjPNxp+/zOxZxwwivd1151IEecvrpuln+H6bTNthv4swNXZaULMXIg64FDhqdYXCImhHQkmT9nLcrC6nQMGo1cBW56PMRJeml/yk7vqkczPVr7/vqZczCZFWDi5pPNhcj6/bZ+OoohFx1Y4mNXQ5bhnXw6bx2UzbRT1znVZn3a+JOBB3bTRRVXXTRN56DGkTIMYDNuGYm+i6yxY5RQAfHG1MHMdW56OI/powueyK4DTwZsoRZTluGYOL67Oc5vguGmga27+ubnyayHkTNo3PZNrOsoBuotOSenpNxIDOzBOqyCvWb9G+EF0iHbVG4lqPAfTrB2m0mLjWEw6ko4ePTPpOUgRweNzwmKa+Jx1/eGSMmzduZ/ClAyx779xUZohoX0NvpbHGuPPMPW2OKN1xy/bOspkd03pnDQwNs26TOftvnDQlk+N6S+Oo4qaN29mweXdp5qROMjW1Ah+xriEpMjdPhHeVMAnDKL3ddd4+ctTZ/GSKwrYdS3etkqKR+3q7+fvX3zIm8ouXpLUdv94lkzzBXO9d1sj3Fe+by73/5g9SfafVFBWdnvUauRxr6W2PWbMKZ30HbecOFHJdsh6/zPHER6wXSJKLnUsEdpVd8kJcksiNjDW018K0fmFLB29Cd62SopFfGRlLlVLcpnXFXYnHGuN87oEdieaurJHvv/nH6ps9izKp3PrwrkzXyOX9sQmQpH3YTJu2cy/T1BT26aaN272Lb6fj4gUTXURbuPqRVPupCmH/TSVrs6BT6weGhhHspR/i1yrp2tlMC32aPqT1ChtXilUP2euTZL2/VX8uQvIuFCfVH0miiOsUmsZsbsU6U6zt3MuoH+OisbUr2WQSXhPRkDYAytSuoNSKe0XQv7RPO+gmMaennrjIu2ZgJ++75VFu2rg9qXbQlGtoszELTQ8sU7CZrn3VyjOsThE6GuPKWskva8aCTrefJzkoRGfUeUi6Tr0O179LmBKgee/WvaXO9LMEhbpotVnX0srGCxENab1gbGahVkUV5yHtANtdr7H20kVWtT5c03DxRNJdW9s1VcD3tw3ztzte1X6uM4X1L+3jmuXzUwsS20w6yztdZVdNF5IGyKRofldcrtO6yxZN8c6Kc1RN9ZhLytmWlywR5y7HzurVVzbenKUhrRdM/9I+Bl86YEwqWOUYkqQcUHHiHjmmc3Ktl23y8Il7QMWxeUOZXsjb+xdPeGIVYRpI69YZz6fViSRFzbvMqJNMm64Bh/1L+3hwcC9PPn/Aup0raTXEtFHtNkHhYm7NYjFoBV6IGHC1B7ukhoBq2sHTes7URJzdJpNmTfUuYcOVZwFMRHibapebajWY6O0xmznCfbp4pgF0183KepZ1lk4WIJC8XujynL//3Sfw3GuHjJ+7BhyuGdiZKEC6Y1kDTNRr7vElocty1LlkeGRsYg0tSxBkUkBzmv61Gm/OykEa1b2KdvC03kXjSjmb5hLttwKDLx1wsh2nvXYuWr9redPjLdu47iNEoNJmTReS1gVd7tUL+0etn7uuT7hou8fXa0736ITZs5wnjbf8YKfWOzFcQ8sSFBr1+tJSTUsWUIAQEZGaiAyJyN8G/y8UkadE5DkR2Sgis4P244L/9wSfL4js45agfbeIrIy0XxS07RGR1Xn7WjSug3BV7eC2WWNvdx2duTn6gtsWWJPqZTfGFfc99bLWNBIPTFu18gynDLshrzukT3HNQhAm2dQtIIf7cFngheY4kGbxtqgI+yI5/8x5U66X0JwArFi/hfPPnJc4aLvY9otaIxgZbTjdZ5dnBpLf+YOjjcxuwP1L+3hy9QVaQdI4qirr4luEJvJZ4NeR/78C3KmUOh04CNwQtN8AHFRKvR+4M9gOEfkAcDWwCLgI+M+BYKoBfwV8FPgA8Mlg28rg8qD3dtcrawc3zRr7ervZvvZC44z+lcBt0qZF3N6/mGuXz7dqJKZBYGSsMWnAHHzpAI1x96mYq1dc+NK+uP4Sq705ycPm0GFz3ZM44WDrkiKmHWnfk/qkqwUT/j88Msb3tw1zxTnZPP6iuGg0LtOKU3q7ne5z9Hg24e1qlo4e88nVF6R6/zsti28uISIipwKXAN8K/hfgAuChYJN7gP7g78uD/wk+/3Cw/eXA/Uqpt5VSLwJ7gA8FP3uUUi8opQ4D9wfbVgbbg97X283XrlrC9rUXVlKAQLIXms104eKBcnv/Yp6/42Lji2sTMFFt515L5LqJtIOui2lKZ2a59eFdqQSca9+qWFPCRfMea4zzxLP7eXL1BXztqiWZjhNmQ04imv1Xh6vXX3S7JOFdhNuxiVB4mZ6mKprEIb8m8jXg33Eshc07gRGlVDg12weEI2gf8DJA8PnrwfYT7bHvmNqnICI3isigiAzu32+PdC4S0wP5tauWpJ59RGmVGSNJ7ba9cGlmS6b92Exe4X7SeI7FSTPoupq34utfWYPpkqLiqzgbdT12uF3W5185fte2AG8yISU980nCO8kdft1lixL7rcOl0FkVTeKQwztLRP4YeE0ptU1EzgubNZuqhM9M7ToBpx1PlFJ3AXdBM3eWpduFUkZCvCqVxgyPF61jflww+7OlQ1+xfov2euiSOJryY4WBfHldcV0Gvrir5p1XLeFzD+ww5uUqitBRAabe2yrWlHD1Rsvbx7JdWW2el0nCu39pnzWIMus7atPyervriMDNORNLlkUeF98VwGUicjFwPPAOmppJr4jMCrSNU4FXgu33AacB+0RkFnAScCDSHhL9jqm9MhRdt6KV1etcBZaujvkV5/RNqUhX7xIOHT4y4bkS31+8/yvWbzH2Lc1YbUsImTSgDQwNs+rBHRP5s4ZHxlj1oF6AwNR1nF5DkSJXTPe2ijUlXOrqxPuY5fq4nuPsmnDYYErMOvlyEd59lmJoWbEJ52gC1CoWwMtszlJK3aKUOlUptYDmwvgWpdQ1wBPAx4PNrgd+FPy9Kfif4PMtqplCeBNwdeC9tRA4Hfg58DRweuDtNTs4xqas/S0Dk9kp2r70tsdYcutjiSkiws9ND1MZZgwXu7tpmyee3T/FLHDi8bOmrA/YTEq2czo42rAKmZCkeiSjh49YzYHrNu2akoDRVNs9PN6k7ztETSehuw7trClheq7jbqihVhb9Hd7v8Dtpr0+XuA+OSWtRWdaQdJ6A8RiNtC68LuZpm4are/9ufXhXZTz3ygg2/Dxwv4jcDgwB3w7avw38tYjsoamBXA2glNolIg8AzwBHgE8rpcYBROQzwGagBtytlHIvSlAypln84EsHJs3Qozbz+CxCtw9TNG8ZZgwXu7utZkdctV5gSERpqu9hqxkSuo0mMXr4yESg4hXn9PG3O16dNPM9ONqwztzSzpLjebniprqTAtNDmrUS070tszqjiSTtNOyPrjZM+Dtei+bE42c5Xw+L/J6Cy6aZJl8m97MIx83qmjj/OT111l6qj7B31fbTpjQ5ONqYuKbt1k4KESJKqZ8CPw3+foGmZ1V8m7eAKw3f/zLwZU37o8CjRfSxaEwzdFsNjHAbW4qIcJEo+kiVZcbIa3cfHhnjpo3bufmB7Vxz7nzn44Yvlu3FcX2loi/SvVv3aiPMozNS3bpMGkx5uXQvb1JNlBBTIslWYsu8EH1mB4aGjetF8e+s27SLQ28fsWp2cdKsOYkkB5amnXxt2Lxbq5lGzz9u0nvLEhHvap42mchcztG0z1bhI9Yzkse8lJQiQkFLzBhFlVtVyrxArsO2iDjHkrIksR/AqOGFDmdrcdfNE2a7R5xDuvueVBMl7XZl4ZJ5IRob5DprHhlrpBIgkG5G3p3BxTeJJO08reu1q5edcSKhcDYHtstzz+fO0uBSCyBt3qT4d237MFUHLBoX77KaxeTkSnx2aXrYBRj64oXOea3S9kH38vd216nXjjrHerjObNcMuGeybXcQmUv8hyk2qGhOmF0zevfFMU0YwJzYM4kk7Tyt6/VJBseC+HNkmkgomppQT5AD7JTebg4celubDyxraYK8eE0khi7Y6KaN21l622OTFq/SpuKIsuCd3RP7KEITyENSZO3y352T+xjxeJCk/Etpc1IlIZhnuK+PNdjw8bOcNSCXiHNXM1ZIu4PIkoRYUmyQ6TtZOHR43ClKPyx0piOchGXR3pNq1KSpKTQwNKzNZlDvmppMMenajjaOcs3y+Ty5+gJjPrd2lRvxQiSGabZ1cLTBqodiwWEZJ+hbXzgItNcDx4WBoWF+sff1VN9Z8b65k7x1rl0+n9v7F0/aJulFDa9LVtNWd71r0gBjCkaCY2kxhr54IV+7aolTxHFSxLlrGvyQcFLRLpIyL4TPpGm78D7Hn+M8pskQk6nIFITqGu1uwqQRhO1pJn7rNumzGZx4/NRkjy4Tib95qjkxMZUgSFuaoCi8OSuGtRZ3kKUzXBRPa+8Nic6K2+GB40oW88Uzr77B83dcbN0m6UUNsS1Y2nj7yFGtg03cYSGskBgS3lcXj60w4jyewn5gaDi1+S+cVLQLU0xKfELjut2kfT+0I3VamDi6d9K2npjnfXIJNhx86QD3PfXyxH2OCrrw2ANDw8bnSDfYr1p5xqR4JR3hR1ULRPVCJEbSWkcYv1DVesd5WDOwc+LlyLoWcnC0wftueZRxpYx2aRe7smtxozuvWjKltoPpPYwLkrBC4rL3zp0QAGnrg8BU9+60lFWxzmVtL9xmrDE+cc9dC4VF40Kin5u2z4pucDS9p3m1n6QBOkxCGb9ncTdbW3yK7nzCaxV/lnVULRDVm7NiuNjjp6sAiZazzTOwRQfXmzduZ0EsIMrFruxifw9NUScc5zYXqolM0VCiwXHR2Ia0jDXGtantXcgZq6jFJQtw3CtrXKmJwchWxTN8R+JCVGfeC9fcrl3u7gIexTQ4mtYk33zLHlyaRJK5yja5iQpU2/NrGuz7l/axfe2FmOqghe1VM4N7TSRGeCNu3ri9ynVgCiE6Uy3rXKNpwsNB2mUmlaQRRqOIXQROd71mLad768O7cnseZRW8xyW4qmbBJT4ha4qddZumXivb9waGhlOvE4G9nHD/0j7trD0a0xE9fppS12D2WEx61oZHxli4+hFjIO2cnnriYH/i8XVtcOaJxx/TsqpkBveaiIb+pX3cedWS3OksTBSx4JiX+Ey1FUQHmqSZVKJGGOm0ybVRgp9w/6bcRr09+pfWhCkgLmtyxqxrPzZcTIZZMgXbbP2676WNLYlyNKGcsKmQVLQfJm/LJbc+ZtRYbB6LttLLIQr9hKK7XmPtpclZfqu2cJ6E10QMhA+OS3RuGuo1cXqQyqYVPv86oguUtgEiyZ4erfT2T2/pX67enjpDX7xwUptOA0p7e7/6ibO0iQhnzxKOHE3O6aTrZ9GY4hOiAjfLAq3N1q8T5nmes6SFYpf+m44fJhKFdAvxaZ+VmghHlXLK8B1qTJ1WT8QLEQPhDS1SgNhy7LSadgW4mfzpo4u10cXdJ1dfwMLVj2hfrNBEZlpIj8/c4qaK3p46SqXLn9XbXZ/w0Pmbp/ZOOvZY4yhd0rzPaTSbMtbVTUrR62PNcr+n9HZz/pnzpmRi7q7XOP/MecaAP5uJ8Y23m+sR0ec76/ph3HNOh4tZ1PacZ0kV4lpGN+SoUry4/pLE7eLZpHVUIT2ODi9ENOjy4+ShSsIjJE/EvSsuOcDi11qXxM80qxaZmuE0iskLxpQDKYl6l7DuskUTHjq69/2oSj/QpN3eBZPpI17K9uz5J7H1hYMTHnlnzz9pkmCJJ1M0JQgFGD/adIGH/B5Zcc85HS4ZF5Ke87STqbTvjav2oMsmHafd6XFMeCGioQhTT5L/fLsxzeKKEpyhJpG0oOni7WKaVdtm8Ekuj0n3uLte44pz+nji2f1T+r9i/Rbrd9OGD7kONGkWiF0Gu7HGOP/j+QMTQmFcqUn/R7cLzVhJpxZmTS7iOYonftSde5JZ9Pwz51mzB6Q1Ea1aeYaz0034DLrcNxdtuN3pcUx4IaIh782K+tmnefFbybGU3r+cyMPzVkECJEzj7lKFLelaZ7kXNq8eW7baKMfN6mLZe+dOibbP2icTrv79aSterlp5hrUCX4hD1nMg3TkXudYWTfxo0o5s75dt9p4ltiI0ZSaltamJcMU5U9Pm50nbXtU1Ee+dpSHvzRoN8uW4+Oq3k8GXDkxK5FaUaT7u1ms736RrfVJ33bhNb3dd69P/1U+cZa3t4GKOGBlrcFMQ4xL35ClqITyNf3/a7LH9S/ucUri4YrsPZWJK/BgWZkrKc2cTflktBbqJRZxxpfj+tmGrO3SUJI/NvOlcysQLEQ15b1ao0utiD7JUWyuLNL773fUap7/7hNTHSDrfJFfeQ4ePsOCd3VNyXwnNgf64WV0TL6Cusl6UrGbKUKCsGWjOIItYCP/aVUtSJQnM4o77x2edbN1nGofkw0fG2zKInX/mPOM5HhxtGPPchZMXk+DrCwJV47hUIRwYGnZy5x5rjDu7Q6+9dBE1S0jBNcvnV8KCocMLkZIYa4wbPXSqYtt09TwLZ8yP/9/nseJ9c1MfJzxf3QsaL7kapzGut9OH/4+MNXjzrSPUa5IYQZ33un9v614GhoZzL4TXUpSADUmTPTbEZsrp6+3mmuXznbPt2tKul8kTz+7PpAGFk4k0CRPTRPnn9dqMn1P/0j6+euXUbNK93XW+dtUSJ+2nXfg1EQ22mXO9SzInXgzpEplws6zKGomJeG2Te//NHwDwe194lMMpanC4lF01ufImHUV3P3Tum7b6LX//+ltOA8OGzbtze7ZlyUeY5M6qW3uz1W0J7+my986d9L3Rw0eMk58Nm3dzwuwahw7btbk5PXV6Zs8qxPvvlZEx7rxqifbcj5vVZV2QDo9/x8cWO61LZo3ytzGnp85bjaNOea5co9Crts7qNRENthnrhivPMs6a4+hs9tDUAMKZzpT08hXDZMJwDagLF9k/98AO7Qt608btvO+WR1kzsLNwm3v8Ptpmpa4zy1dGxgqvd+KCLcrfNIM2RfKf1F2f0AjD2XoYnW0LhH1lZIx6zT5kCHDJB+1mtDSE+dF0577uskWJ9yGcqNhq5oTkifLXEUaoF5nnqorrrF4T0WCbsUZnCwNDw8asm7UgpmDwpQPcu3WvcTYdTS/fakx1nUPieX6iMyBbbqBwFhqNKbAN0uNK8b2te1nxvrkcOHR4krCxxSUkEV8AN8UVgHsFx5O6j10Tm/eTbX8CU4LyXDDNVE0z6OPrXVPctutdwqHDRyaeWZ1GeOvDu7TayCm93dZBVIA/fN/cKQGMWYnmR7PN0m2Zb9MEFLpEwLtooRJsF9UQ0moYJ3XXEWnG+0T3lTXfWZlk1kRE5DQReUJEfi0iu0Tks0H7XBF5XESeC37PCdpFRL4hIntE5JcicnZkX9cH2z8nItdH2s8RkZ3Bd74h0praXa521DDrpi5D6fhRxeBLB3ji2f1OvvXtwBYBG8/zE58B2XIDPbn6Avp6u1MP/ltfODhl1nbN8vmJC8D1LtEuSuoyusbzIgGpbNzhE9i/tM+okQrNao6m3GsqOGZRs0fTwD4y2phyPU88ftYULTLu/LD20qkz/PD5txWmuvOqJfzmH8eKc/GN3RLTmtr2tc2CYiZctQeX9z5JCz1hdi1R49ERf79GxhocHG1M0TayOFiUjaiMC0QicjJwslLqFyLyO8A2oB/4FHBAKbVeRFYDc5RSnxeRi4E/Ay4GzgW+rpQ6V0TmAoPAMpqPzTbgHKXUQRH5OfBZYCvwKPANpdSPbf1atmyZGhwczHROUdYM7JyS1uLYuTc9dMJ4EFN+rTBvjssV/o1DaoSiSIqVqIlMcZNNqqEiQM/sGqOHx3OtGehqWlzzX3/Gk88fsG5v0liS6tWnrQ0jMJHGQhf1LjQ9aW7vX2zVVF365orpHHT7N607weQZNOjjL3TnHA2ste0/C+E5JB0X0l0HE641WEz3VVfJ0+UYJu0vfh6gTyVTxLMkItuUUsvSfi+zOUsp9SrwavD3GyLya6APuBw4L9jsHuCnwOeD9u+qptTaKiK9gSA6D3hcKXUgOJHHgYtE5KfAO5RSPwvav0tTSFmFSBEMDA2z8emXzcWNgvZwhmCaxYaDYdIgVaQ/v47oQ9vbU+fNt44kVFCbmj01aaajYGLBNW7KSoMu7ckzr76h3TZaO910rKRrnyXtRUhS2o0ks9fwyBhrBnbm9rxJU6TIJuCjs947PrZYOyglnXPajMhJhPfHxYxz/pnzppiO0wYUuixuh9usGdg55XhJqVp0Diau1R+HR8aY01Of4tzTzoJUUNCaiIgsAJYCTwHvCQQMSqlXReTdwWZ9QDQwYV/QZmvfp2nXHf9G4EaA+fOzFb+JsmHzbueF4yTV/cCht62fh/mYyiL+0Lq84ApYettjk/J9pdUuipiNjjXG+cIPdxq9gVyOkeTPn+a8ogFf8dnknVct0c5YVz20w7rPMPI5jyBxySEVohM4cZJs7LaBtuhkkqHQTjLjDAwNs/HnL2ueCcXNG7c7ZU9wxabJh84ipuPphGGarM8HRxvUa0Jvd53XxxrTwztLRE4Evg/cpJT6J9ummjaVoX1qo1J3KaWWKaWWzZuXP9NlkYkJxyz+9QJc9aHTSn0AsgbYHRxtTPIca4dHEpDoTprEuFLWtYc0mVEVGL2h4pHS4D4ZyVKwKY6tBkZ8u+g6iYmsNvYik0lGZ9hJcTKmBIZjjaOFejG5Zj0YHhlj1YNTPS+LWLtojCtOOG5WprWXMsglRESkTlOA3KuU+kHQ/NvATBWum7wWtO8DTot8/VTglYT2UzXtpZO1uFBaFOVn5szz0DbGj9XsCAefsk1vZWAbPNJc/9AmbRLM0UhpcL/2ZdVYNxEVOCbngKzu1kW5acddYZMWvV0SGBaRLSLNpKxxVLFu065JbUVdn6oELEM+7ywBvg38Win1F5GPNgGhh9X1wI8i7dcFXlrLgdcDs9dm4EIRmRN4cl0IbA4+e0NElgfHui6yr1Jp5Utd9sOQ96EdDhLghRwK8oJ1ErbBw/X612vCobePsHD1I9ZZaJjTCaqbMC9KmojuODpvqVUrz8hdEVSkeV+i6WtcqmG6kPd9S/v9uHDTXe96TVJfszIKmWUlz5rICuD/AHaKSLhy+O+B9cADInIDsBe4MvjsUZqeWXuAUeBPAJRSB0TkS8DTwXa3hYvswJ8C3wG6aS6ol76oDtBd77KaoYqk7IEmTepqE+EC960P70pdta8qmF5+U62SOI1x5Vy86uBog6W3PcYlHzyZjU+/XOlrlmY9JWRgaHiKN9HwyNjEc9aM0h7P/A7FHVfCftrWYlwLgaV53+IOKUrlX+szXe+keLI4LVZereTxzvrvmHO4fVizvQI+bdjX3cDdmvZB4Pez9jEreWpe18Q9rUUrvCrC1NVpHtA44Uze1esmT4BgWZgGj7IslwdHG3xv617e8zuz2f/mYWuNkVqb0+C4ptsAe8G28BQPjjbortfodRTQNpIW+cOB3vXZPPT2sdghm+DM4pCiQ5edV3e9bWVxdZRRyCwrPmJdQ54B0FWA9LVwwLi9fzHL3jvXqb6EiTRqfNUEiE1Ylx3o+ds3Diduo3NrbvdiqQnXNYGxxnhhQYemZ89WUrYvKP/7yC9fnXSPR8YarHpwB8gxryjddS+iMF29JtY0MlHSOvNUyVTqhUgbKCrIzJVwtpaHU3q7OfT2kdwzy1ZTprCek6FGexKtTmERd1U+/8x52mqOIWWXVNZhGjCTSsoue+9cnnh2/5SJgkvCzqxrJxKo4Wm1Ste0O9D+uJA4XohoqHdBWUsirX4AiqgXH+2zaeZXNHMcgiLbTVlaTKs8b3SBb9GKfboZeprBrghs70tSBt+0z2r0umfNutBTb2ZtSEuasgzh9VixfkslMvl6IaJhVq2LxtHipUgrTVghWdXycF0jNAuEs9XuehdHlCp1YS/U1NYM7EwsQ5pEuOA7+NKBKWnPqzSbi9IqU4XLsxGfobdCgPQFiR7zDo5pJyDR6+4SlKkjmrXhlh/snMifl3Q+LpktuoSJFDCfe3AH40ePmeM+92AzqLUdgsQLEQ1leWa5vBBF1wrIOqsNBUj8ZSq7OJHQDAAMU88UgaIZGR71lIrX6W4X8Sy7rdRUXZ+N6HYug11eXE29XYLVYSEN8ese96LqzWC6HGuMT3JoiU5o4hkKXLwoj6qm9vEPb749IUBCxo8qvvDDnW0RIr6eSAtJWpfQRULfHCnLmoU8s9rQV7+wrKwOhAP+TRu3F+4aq8te226KrDWRFtdnIz5Dr9daE4ybRF4B+qHMyQAAC5lJREFUknTdo0GZQ1+80Jix20a8iwq4N6iQGT+WS8bq4ZEx3j6in8jlze6QFa+JtJCkmZ9uwA4fOltSNxtZ1XKA2bO6KhUZOx1J415bNOefOS/RXGiaoUfjRIp06T5ulvu8No9WVBOxuvearAFFZJhQoHWeuL1/MU+98I8899qh3MdoJV4TaSFJMz/TgB0+dFlwzZWk4+0jR43V8aYLOYOrK4kuklzH3+541bofAa44Z6qQ61/ax9AXL+Q36y8pvITB20eOWvscJU3eszjR6qLRVDVJlQOLmlTF9zMwNMyiL/63jhMg4IVIy3CxdduETJ6H1yVXkomRsUZq4dNJVNj5KxNpyqcm2fddc7sVnYLDNVli2rxngj4vXjQtji3lPBTn9BDdz5qBndy0cXtuc1R3vT3DuRciGopKMijBj6ute9XKM4wDtuvDmzQLzZKN15RSOYm+3u6J2Wpa4VUG1y6fbx1MTNiq5uWhjGSWSYNgWlwmL2U4bLn0Oc3EKpxEHTV0NtxXUsr5LNpP/EmLTigHhoa5N6UHoi7PVhdwx8c+mLpvReCFiIak8cV1+PnD981Nla7ZtLiWJiFe0iw0at5KQ9pxIk1Z0S5plhUtm2XvnZs4mMS5dvl8+pf2FZ7Zuaw6MqY1Al27LiVHHJfJS1kpOJKERBatICmlfNLnWdZE7rxqiXERP226k77ebjZceRZfi+3zLzT1bFqFFyIaRixBZGlqh2994WDqY9/ev9j60NlwnYWG5q2ytANdn22p5GsifPlfLp70YszpqdMTUc9dxvCk9Y1oWu6kAagmMqnU6SfPPc26fUhYMCiJDVeeVcpLbxJ2uvaklByukxdjXrLEb2bbb4irVh0VlklZi5M+T2tWntNTt9Z6SbO/a5fPn/i+a/2YVuC9szSYolXDyna2+uRRsgZmZfXYsaniOo+TPJ5bNnR+/uHxdXb4xtFm3RKXl2HpbY9pI8VdAhSjxzaVlDUJ7FCY3PvU3inmm2hgZujJY6vd3tfbXdpLbyvVHMeWnHNOT31SZUsbpmt5xTl9mYNFXQRYdDZvuta1rsn5q1xLGps+TxvJfskHT7Z+nmZ/ZdceyooXIhpMgT+hl5Tr4Nuq4lYhpgfypO76lPQWYR3tOz62eOIlDFNaFJ2F1yX1isuMbGBomDffmlrPpF6TiQHn9v7FTgOXbrAII/Nv3rhd6/p5e/9innh2/5RrHAqQqPC0nU+ZwYQmt1eT1hkm58wT4GobeOMJEF1x1b6jE66BoWHWbdo1MVkwCcKkSZrtc927b3tfkgb+NBO56DNVdFByHrwQ0dC/tM+Y8faVkbEpL83xhvojriaQojDNCEWmBtaFZi7d7H/h6kcy90EnN10CFl3s2xs279amsjhh9qxJ52CqLRFfA4gPQDpBG24XkrTwGj0f3WDe210v9WU3PQM2wVVErIppH2svXZRa262JZOpPK2JuTALTFG2eNDly0aZCQi8412e1Vfg1EQNJZUOjNslff+mjXLt8/oTmEbentwpT9TfTGo/pAc/jxqizpiS9SK62d9N+4gu7ay9dNCWqOiktt+t6UtLCa4jJtl7GYnqUoioAFt2fNLR68pUW3XqE63Nh21/SGmX4bhXtgZcXr4kYSDuju71/ccuFhg5TwRvdLMf0gJtU7BNm16jXuqzxBboXwWb3TZOU0rSf+HlkqdbnqmG4PhdZ+lAU7YyC19G/tM95HbGn3lWJ9ygtWTTAOEmTrXCy5PqstgovRAy0cxAomrQPuMu5rxnYOWVB1rTPtIvYRZxH2oG0DAFVtcG8nbjY/rvrNf5DSq2lKhQxXiQtskfdkNNMCstGVJWK9RbAsmXL1ODgYLu7UTnKWIhLs8+ijl/WgqJu8T+LoPOYSVsAa6Zhc0CJPotlPasisk0ptSz197wQ8XiaVMnjxTMzCZ/BqLekzuRbxrM6bYWIiFwEfB2oAd9SSq23be+FiMfj8aQnqxCptHeWiNSAvwI+CnwA+KSIfKC9vfJ4PB5PSKWFCPAhYI9S6gWl1GHgfuDyNvfJ4/F4PAFVFyJ9QLRG6r6gbRIicqOIDIrI4P791UwN4PF4PNORqgsRXd6QKYs4Sqm7lFLLlFLL5s3LXqjG4/F4POmouhDZB0TDV08FXmlTXzwej8cTo9LeWSIyC/ifwIeBYeBp4F8ppXZZvrMfeCnnod8F/EPOfZSF71s2fN+yUeW+QbX712l9e69SKrUpp9IR60qpIyLyGWAzTRffu20CJPhObnuWiAxmcXVrBb5v2fB9y0aV+wbV7t9M6VulhQiAUupR4NF298Pj8Xg8U6n6mojH4/F4KowXInruancHLPi+ZcP3LRtV7htUu38zom+VXlj3eDweT7XxmojH4/F4MuOFiMfj8XgyMyOEiIicJiJPiMivRWSXiHw2aJ8rIo+LyHPB7zlBu4jIN0Rkj4j8UkTOju3vHSIyLCJ/WaW+ici4iGwPfjZVrG/zReSxYF/PiMiCKvRNRM6PXLPtIvKWiPRXoW/BZ/8x2Mevg210WRza2b+viMivgp+r2tC3M0XkZyLytoj8eWxfF4nI7qDfqyvWt7tF5DUR+VXefhXZN9N+rCilpv0PcDJwdvD379AMYPwA8B+B1UH7auArwd8XAz+mmXZlOfBUbH9fB/4G+Msq9Q14s6rXDfgp8JHg7xOBnqr0LbLPucCBqvQN+EPgSZoxUjXgZ8B5VbmvwCXA4zRDBU4ABoF3tLhv7wb+OfBl4M8j+6kBzwO/C8wGdgAfqELfgs/+CDgb+FWb3lXTddPux3rsIk6g036AHwEfAXYDJ0cu3u7g7/8CfDKyfXS7c2hmE/4UBQiRgvtWqBApqm/Bw/zfq9i32D5uBO6tSt+APwC2Ad1AD81B+p9VqH+rgDWR9m8Dn2hl3yLbrYsNhn8AbI78fwtwSxX6FmlfQEFCpOi+xfdjO9aMMGdFCcwoS4GngPcopV4FCH6/O9hMmz1YRLqAr9J8eSrVt+Dv46WZzXhrXpNMwX37PWBERH4gIkMiskGatWKq0LcoVwP3FdWvvH1TSv0MeAJ4NfjZrJT6dVX6R3N2/1ER6RGRdwHnMznXXSv6ZsIpA3ib+lYqRfUtth8jlY9YLxIRORH4PnCTUuqfLOZlU/bgfws8qpR6uQDTdNF9A5ivlHpFRH4X2CIiO5VSz1egb7OA/43mA7kX2EhTk/t2BfoW7udkYDHNFDuFkLdvIvJ+4J/RTDwK8LiI/JFS6u+q0D+l1GMi8s+B/wHsp2luO9Livhl3oWkrJJ6hgL6VRlF9i+/Htu2M0UREpE7zotyrlPpB0PzbYPAIB5HXgnZT9uA/AD4jIr8B/hNwnYhYy/W2sG8opcLfL9Bcg1hakb7tA4ZUs7jYEWCApj24Cn0L+QTwQ6VUI2+/CuzbvwS2KqXeVEq9SXNdYnmF+odS6stKqSVKqY/QHLifa3HfTJSSAbygvpVCUX0z7MfIjBAi0hTH3wZ+rZT6i8hHm4Drg7+vp2n/C9uvkybLgdeVUq8qpa5RSs1XSi0A/hz4rlIql9dHUX0TkTkiclywz3cBK4BnqtA3mtmX54hImBzzggr1LeSTFGTKKrBve4F/ISKzghf7XwC5zVkFPnM1EXlnsM8PAh8EHmtx30w8DZwuIgtFZDZNU2Uuj8UC+1Y4RfXNsh8zZSzqVO0H+F9pqrK/BLYHPxcD7wR+QnP29BNgbrC90Kzt/jywE1im2eenKMY7q5C+0fTk2UnTTr0TuKEqfQs++0iwn53Ad4DZFerbApqlBrqq9LzR9DD6LzQFxzPAX1Ssf8cH/XoG2AosaUPf/heaWsc/ASPB3+8IPruYpnfR88AXKta3+2iuczWC9lzva1F9M+3Hdmyf9sTj8Xg8mZkR5iyPx+PxlIMXIh6Px+PJjBciHo/H48mMFyIej8fjyYwXIh6Px+PJjBciHo/H48mMFyIej8fjycz/DzLilwwiWR4hAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots()\n",
    "ax.scatter(df[\"saledate\"][:1000], df[\"SalePrice\"][:1000])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SalesID</th>\n",
       "      <th>SalePrice</th>\n",
       "      <th>MachineID</th>\n",
       "      <th>ModelID</th>\n",
       "      <th>datasource</th>\n",
       "      <th>auctioneerID</th>\n",
       "      <th>YearMade</th>\n",
       "      <th>MachineHoursCurrentMeter</th>\n",
       "      <th>UsageBand</th>\n",
       "      <th>saledate</th>\n",
       "      <th>...</th>\n",
       "      <th>Undercarriage_Pad_Width</th>\n",
       "      <th>Stick_Length</th>\n",
       "      <th>Thumb</th>\n",
       "      <th>Pattern_Changer</th>\n",
       "      <th>Grouser_Type</th>\n",
       "      <th>Backhoe_Mounting</th>\n",
       "      <th>Blade_Type</th>\n",
       "      <th>Travel_Controls</th>\n",
       "      <th>Differential_Type</th>\n",
       "      <th>Steering_Controls</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1139246</td>\n",
       "      <td>66000.0</td>\n",
       "      <td>999089</td>\n",
       "      <td>3157</td>\n",
       "      <td>121</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2004</td>\n",
       "      <td>68.0</td>\n",
       "      <td>Low</td>\n",
       "      <td>2006-11-16</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "      <td>Conventional</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1139248</td>\n",
       "      <td>57000.0</td>\n",
       "      <td>117657</td>\n",
       "      <td>77</td>\n",
       "      <td>121</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1996</td>\n",
       "      <td>4640.0</td>\n",
       "      <td>Low</td>\n",
       "      <td>2004-03-26</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "      <td>Conventional</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1139249</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>434808</td>\n",
       "      <td>7009</td>\n",
       "      <td>121</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2001</td>\n",
       "      <td>2838.0</td>\n",
       "      <td>High</td>\n",
       "      <td>2004-02-26</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1139251</td>\n",
       "      <td>38500.0</td>\n",
       "      <td>1026470</td>\n",
       "      <td>332</td>\n",
       "      <td>121</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2001</td>\n",
       "      <td>3486.0</td>\n",
       "      <td>High</td>\n",
       "      <td>2011-05-19</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1139253</td>\n",
       "      <td>11000.0</td>\n",
       "      <td>1057373</td>\n",
       "      <td>17311</td>\n",
       "      <td>121</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2007</td>\n",
       "      <td>722.0</td>\n",
       "      <td>Medium</td>\n",
       "      <td>2009-07-23</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 53 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   SalesID  SalePrice  MachineID  ModelID  datasource  auctioneerID  YearMade  \\\n",
       "0  1139246    66000.0     999089     3157         121           3.0      2004   \n",
       "1  1139248    57000.0     117657       77         121           3.0      1996   \n",
       "2  1139249    10000.0     434808     7009         121           3.0      2001   \n",
       "3  1139251    38500.0    1026470      332         121           3.0      2001   \n",
       "4  1139253    11000.0    1057373    17311         121           3.0      2007   \n",
       "\n",
       "   MachineHoursCurrentMeter UsageBand   saledate  ... Undercarriage_Pad_Width  \\\n",
       "0                      68.0       Low 2006-11-16  ...                     NaN   \n",
       "1                    4640.0       Low 2004-03-26  ...                     NaN   \n",
       "2                    2838.0      High 2004-02-26  ...                     NaN   \n",
       "3                    3486.0      High 2011-05-19  ...                     NaN   \n",
       "4                     722.0    Medium 2009-07-23  ...                     NaN   \n",
       "\n",
       "  Stick_Length Thumb Pattern_Changer Grouser_Type Backhoe_Mounting Blade_Type  \\\n",
       "0          NaN   NaN             NaN          NaN              NaN        NaN   \n",
       "1          NaN   NaN             NaN          NaN              NaN        NaN   \n",
       "2          NaN   NaN             NaN          NaN              NaN        NaN   \n",
       "3          NaN   NaN             NaN          NaN              NaN        NaN   \n",
       "4          NaN   NaN             NaN          NaN              NaN        NaN   \n",
       "\n",
       "  Travel_Controls Differential_Type Steering_Controls  \n",
       "0             NaN          Standard      Conventional  \n",
       "1             NaN          Standard      Conventional  \n",
       "2             NaN               NaN               NaN  \n",
       "3             NaN               NaN               NaN  \n",
       "4             NaN               NaN               NaN  \n",
       "\n",
       "[5 rows x 53 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>SalesID</th>\n",
       "      <td>1139246</td>\n",
       "      <td>1139248</td>\n",
       "      <td>1139249</td>\n",
       "      <td>1139251</td>\n",
       "      <td>1139253</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SalePrice</th>\n",
       "      <td>66000</td>\n",
       "      <td>57000</td>\n",
       "      <td>10000</td>\n",
       "      <td>38500</td>\n",
       "      <td>11000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MachineID</th>\n",
       "      <td>999089</td>\n",
       "      <td>117657</td>\n",
       "      <td>434808</td>\n",
       "      <td>1026470</td>\n",
       "      <td>1057373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ModelID</th>\n",
       "      <td>3157</td>\n",
       "      <td>77</td>\n",
       "      <td>7009</td>\n",
       "      <td>332</td>\n",
       "      <td>17311</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>datasource</th>\n",
       "      <td>121</td>\n",
       "      <td>121</td>\n",
       "      <td>121</td>\n",
       "      <td>121</td>\n",
       "      <td>121</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>auctioneerID</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>YearMade</th>\n",
       "      <td>2004</td>\n",
       "      <td>1996</td>\n",
       "      <td>2001</td>\n",
       "      <td>2001</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MachineHoursCurrentMeter</th>\n",
       "      <td>68</td>\n",
       "      <td>4640</td>\n",
       "      <td>2838</td>\n",
       "      <td>3486</td>\n",
       "      <td>722</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UsageBand</th>\n",
       "      <td>Low</td>\n",
       "      <td>Low</td>\n",
       "      <td>High</td>\n",
       "      <td>High</td>\n",
       "      <td>Medium</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saledate</th>\n",
       "      <td>2006-11-16 00:00:00</td>\n",
       "      <td>2004-03-26 00:00:00</td>\n",
       "      <td>2004-02-26 00:00:00</td>\n",
       "      <td>2011-05-19 00:00:00</td>\n",
       "      <td>2009-07-23 00:00:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiModelDesc</th>\n",
       "      <td>521D</td>\n",
       "      <td>950FII</td>\n",
       "      <td>226</td>\n",
       "      <td>PC120-6E</td>\n",
       "      <td>S175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiBaseModel</th>\n",
       "      <td>521</td>\n",
       "      <td>950</td>\n",
       "      <td>226</td>\n",
       "      <td>PC120</td>\n",
       "      <td>S175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiSecondaryDesc</th>\n",
       "      <td>D</td>\n",
       "      <td>F</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiModelSeries</th>\n",
       "      <td>NaN</td>\n",
       "      <td>II</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-6E</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiModelDescriptor</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ProductSize</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Medium</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Small</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiProductClassDesc</th>\n",
       "      <td>Wheel Loader - 110.0 to 120.0 Horsepower</td>\n",
       "      <td>Wheel Loader - 150.0 to 175.0 Horsepower</td>\n",
       "      <td>Skid Steer Loader - 1351.0 to 1601.0 Lb Operat...</td>\n",
       "      <td>Hydraulic Excavator, Track - 12.0 to 14.0 Metr...</td>\n",
       "      <td>Skid Steer Loader - 1601.0 to 1751.0 Lb Operat...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>state</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>North Carolina</td>\n",
       "      <td>New York</td>\n",
       "      <td>Texas</td>\n",
       "      <td>New York</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ProductGroup</th>\n",
       "      <td>WL</td>\n",
       "      <td>WL</td>\n",
       "      <td>SSL</td>\n",
       "      <td>TEX</td>\n",
       "      <td>SSL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ProductGroupDesc</th>\n",
       "      <td>Wheel Loader</td>\n",
       "      <td>Wheel Loader</td>\n",
       "      <td>Skid Steer Loaders</td>\n",
       "      <td>Track Excavators</td>\n",
       "      <td>Skid Steer Loaders</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Drive_System</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Enclosure</th>\n",
       "      <td>EROPS w AC</td>\n",
       "      <td>EROPS w AC</td>\n",
       "      <td>OROPS</td>\n",
       "      <td>EROPS w AC</td>\n",
       "      <td>EROPS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Forks</th>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pad_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ride_Control</th>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Stick</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Transmission</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Turbocharged</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blade_Extension</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blade_Width</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Enclosure_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Engine_Horsepower</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hydraulics</th>\n",
       "      <td>2 Valve</td>\n",
       "      <td>2 Valve</td>\n",
       "      <td>Auxiliary</td>\n",
       "      <td>2 Valve</td>\n",
       "      <td>Auxiliary</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pushblock</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ripper</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Scarifier</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tip_Control</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tire_Size</th>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>23.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Coupler</th>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>None or Unspecified</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Coupler_System</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Grouser_Tracks</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hydraulics_Flow</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Track_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Undercarriage_Pad_Width</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Stick_Length</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Thumb</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pattern_Changer</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Grouser_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Backhoe_Mounting</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blade_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Travel_Controls</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Differential_Type</th>\n",
       "      <td>Standard</td>\n",
       "      <td>Standard</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Steering_Controls</th>\n",
       "      <td>Conventional</td>\n",
       "      <td>Conventional</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                 0  \\\n",
       "SalesID                                                    1139246   \n",
       "SalePrice                                                    66000   \n",
       "MachineID                                                   999089   \n",
       "ModelID                                                       3157   \n",
       "datasource                                                     121   \n",
       "auctioneerID                                                     3   \n",
       "YearMade                                                      2004   \n",
       "MachineHoursCurrentMeter                                        68   \n",
       "UsageBand                                                      Low   \n",
       "saledate                                       2006-11-16 00:00:00   \n",
       "fiModelDesc                                                   521D   \n",
       "fiBaseModel                                                    521   \n",
       "fiSecondaryDesc                                                  D   \n",
       "fiModelSeries                                                  NaN   \n",
       "fiModelDescriptor                                              NaN   \n",
       "ProductSize                                                    NaN   \n",
       "fiProductClassDesc        Wheel Loader - 110.0 to 120.0 Horsepower   \n",
       "state                                                      Alabama   \n",
       "ProductGroup                                                    WL   \n",
       "ProductGroupDesc                                      Wheel Loader   \n",
       "Drive_System                                                   NaN   \n",
       "Enclosure                                               EROPS w AC   \n",
       "Forks                                          None or Unspecified   \n",
       "Pad_Type                                                       NaN   \n",
       "Ride_Control                                   None or Unspecified   \n",
       "Stick                                                          NaN   \n",
       "Transmission                                                   NaN   \n",
       "Turbocharged                                                   NaN   \n",
       "Blade_Extension                                                NaN   \n",
       "Blade_Width                                                    NaN   \n",
       "Enclosure_Type                                                 NaN   \n",
       "Engine_Horsepower                                              NaN   \n",
       "Hydraulics                                                 2 Valve   \n",
       "Pushblock                                                      NaN   \n",
       "Ripper                                                         NaN   \n",
       "Scarifier                                                      NaN   \n",
       "Tip_Control                                                    NaN   \n",
       "Tire_Size                                      None or Unspecified   \n",
       "Coupler                                        None or Unspecified   \n",
       "Coupler_System                                                 NaN   \n",
       "Grouser_Tracks                                                 NaN   \n",
       "Hydraulics_Flow                                                NaN   \n",
       "Track_Type                                                     NaN   \n",
       "Undercarriage_Pad_Width                                        NaN   \n",
       "Stick_Length                                                   NaN   \n",
       "Thumb                                                          NaN   \n",
       "Pattern_Changer                                                NaN   \n",
       "Grouser_Type                                                   NaN   \n",
       "Backhoe_Mounting                                               NaN   \n",
       "Blade_Type                                                     NaN   \n",
       "Travel_Controls                                                NaN   \n",
       "Differential_Type                                         Standard   \n",
       "Steering_Controls                                     Conventional   \n",
       "\n",
       "                                                                 1  \\\n",
       "SalesID                                                    1139248   \n",
       "SalePrice                                                    57000   \n",
       "MachineID                                                   117657   \n",
       "ModelID                                                         77   \n",
       "datasource                                                     121   \n",
       "auctioneerID                                                     3   \n",
       "YearMade                                                      1996   \n",
       "MachineHoursCurrentMeter                                      4640   \n",
       "UsageBand                                                      Low   \n",
       "saledate                                       2004-03-26 00:00:00   \n",
       "fiModelDesc                                                 950FII   \n",
       "fiBaseModel                                                    950   \n",
       "fiSecondaryDesc                                                  F   \n",
       "fiModelSeries                                                   II   \n",
       "fiModelDescriptor                                              NaN   \n",
       "ProductSize                                                 Medium   \n",
       "fiProductClassDesc        Wheel Loader - 150.0 to 175.0 Horsepower   \n",
       "state                                               North Carolina   \n",
       "ProductGroup                                                    WL   \n",
       "ProductGroupDesc                                      Wheel Loader   \n",
       "Drive_System                                                   NaN   \n",
       "Enclosure                                               EROPS w AC   \n",
       "Forks                                          None or Unspecified   \n",
       "Pad_Type                                                       NaN   \n",
       "Ride_Control                                   None or Unspecified   \n",
       "Stick                                                          NaN   \n",
       "Transmission                                                   NaN   \n",
       "Turbocharged                                                   NaN   \n",
       "Blade_Extension                                                NaN   \n",
       "Blade_Width                                                    NaN   \n",
       "Enclosure_Type                                                 NaN   \n",
       "Engine_Horsepower                                              NaN   \n",
       "Hydraulics                                                 2 Valve   \n",
       "Pushblock                                                      NaN   \n",
       "Ripper                                                         NaN   \n",
       "Scarifier                                                      NaN   \n",
       "Tip_Control                                                    NaN   \n",
       "Tire_Size                                                     23.5   \n",
       "Coupler                                        None or Unspecified   \n",
       "Coupler_System                                                 NaN   \n",
       "Grouser_Tracks                                                 NaN   \n",
       "Hydraulics_Flow                                                NaN   \n",
       "Track_Type                                                     NaN   \n",
       "Undercarriage_Pad_Width                                        NaN   \n",
       "Stick_Length                                                   NaN   \n",
       "Thumb                                                          NaN   \n",
       "Pattern_Changer                                                NaN   \n",
       "Grouser_Type                                                   NaN   \n",
       "Backhoe_Mounting                                               NaN   \n",
       "Blade_Type                                                     NaN   \n",
       "Travel_Controls                                                NaN   \n",
       "Differential_Type                                         Standard   \n",
       "Steering_Controls                                     Conventional   \n",
       "\n",
       "                                                                          2  \\\n",
       "SalesID                                                             1139249   \n",
       "SalePrice                                                             10000   \n",
       "MachineID                                                            434808   \n",
       "ModelID                                                                7009   \n",
       "datasource                                                              121   \n",
       "auctioneerID                                                              3   \n",
       "YearMade                                                               2001   \n",
       "MachineHoursCurrentMeter                                               2838   \n",
       "UsageBand                                                              High   \n",
       "saledate                                                2004-02-26 00:00:00   \n",
       "fiModelDesc                                                             226   \n",
       "fiBaseModel                                                             226   \n",
       "fiSecondaryDesc                                                         NaN   \n",
       "fiModelSeries                                                           NaN   \n",
       "fiModelDescriptor                                                       NaN   \n",
       "ProductSize                                                             NaN   \n",
       "fiProductClassDesc        Skid Steer Loader - 1351.0 to 1601.0 Lb Operat...   \n",
       "state                                                              New York   \n",
       "ProductGroup                                                            SSL   \n",
       "ProductGroupDesc                                         Skid Steer Loaders   \n",
       "Drive_System                                                            NaN   \n",
       "Enclosure                                                             OROPS   \n",
       "Forks                                                   None or Unspecified   \n",
       "Pad_Type                                                                NaN   \n",
       "Ride_Control                                                            NaN   \n",
       "Stick                                                                   NaN   \n",
       "Transmission                                                            NaN   \n",
       "Turbocharged                                                            NaN   \n",
       "Blade_Extension                                                         NaN   \n",
       "Blade_Width                                                             NaN   \n",
       "Enclosure_Type                                                          NaN   \n",
       "Engine_Horsepower                                                       NaN   \n",
       "Hydraulics                                                        Auxiliary   \n",
       "Pushblock                                                               NaN   \n",
       "Ripper                                                                  NaN   \n",
       "Scarifier                                                               NaN   \n",
       "Tip_Control                                                             NaN   \n",
       "Tire_Size                                                               NaN   \n",
       "Coupler                                                 None or Unspecified   \n",
       "Coupler_System                                          None or Unspecified   \n",
       "Grouser_Tracks                                          None or Unspecified   \n",
       "Hydraulics_Flow                                                    Standard   \n",
       "Track_Type                                                              NaN   \n",
       "Undercarriage_Pad_Width                                                 NaN   \n",
       "Stick_Length                                                            NaN   \n",
       "Thumb                                                                   NaN   \n",
       "Pattern_Changer                                                         NaN   \n",
       "Grouser_Type                                                            NaN   \n",
       "Backhoe_Mounting                                                        NaN   \n",
       "Blade_Type                                                              NaN   \n",
       "Travel_Controls                                                         NaN   \n",
       "Differential_Type                                                       NaN   \n",
       "Steering_Controls                                                       NaN   \n",
       "\n",
       "                                                                          3  \\\n",
       "SalesID                                                             1139251   \n",
       "SalePrice                                                             38500   \n",
       "MachineID                                                           1026470   \n",
       "ModelID                                                                 332   \n",
       "datasource                                                              121   \n",
       "auctioneerID                                                              3   \n",
       "YearMade                                                               2001   \n",
       "MachineHoursCurrentMeter                                               3486   \n",
       "UsageBand                                                              High   \n",
       "saledate                                                2011-05-19 00:00:00   \n",
       "fiModelDesc                                                        PC120-6E   \n",
       "fiBaseModel                                                           PC120   \n",
       "fiSecondaryDesc                                                         NaN   \n",
       "fiModelSeries                                                           -6E   \n",
       "fiModelDescriptor                                                       NaN   \n",
       "ProductSize                                                           Small   \n",
       "fiProductClassDesc        Hydraulic Excavator, Track - 12.0 to 14.0 Metr...   \n",
       "state                                                                 Texas   \n",
       "ProductGroup                                                            TEX   \n",
       "ProductGroupDesc                                           Track Excavators   \n",
       "Drive_System                                                            NaN   \n",
       "Enclosure                                                        EROPS w AC   \n",
       "Forks                                                                   NaN   \n",
       "Pad_Type                                                                NaN   \n",
       "Ride_Control                                                            NaN   \n",
       "Stick                                                                   NaN   \n",
       "Transmission                                                            NaN   \n",
       "Turbocharged                                                            NaN   \n",
       "Blade_Extension                                                         NaN   \n",
       "Blade_Width                                                             NaN   \n",
       "Enclosure_Type                                                          NaN   \n",
       "Engine_Horsepower                                                       NaN   \n",
       "Hydraulics                                                          2 Valve   \n",
       "Pushblock                                                               NaN   \n",
       "Ripper                                                                  NaN   \n",
       "Scarifier                                                               NaN   \n",
       "Tip_Control                                                             NaN   \n",
       "Tire_Size                                                               NaN   \n",
       "Coupler                                                 None or Unspecified   \n",
       "Coupler_System                                                          NaN   \n",
       "Grouser_Tracks                                                          NaN   \n",
       "Hydraulics_Flow                                                         NaN   \n",
       "Track_Type                                                              NaN   \n",
       "Undercarriage_Pad_Width                                                 NaN   \n",
       "Stick_Length                                                            NaN   \n",
       "Thumb                                                                   NaN   \n",
       "Pattern_Changer                                                         NaN   \n",
       "Grouser_Type                                                            NaN   \n",
       "Backhoe_Mounting                                                        NaN   \n",
       "Blade_Type                                                              NaN   \n",
       "Travel_Controls                                                         NaN   \n",
       "Differential_Type                                                       NaN   \n",
       "Steering_Controls                                                       NaN   \n",
       "\n",
       "                                                                          4  \n",
       "SalesID                                                             1139253  \n",
       "SalePrice                                                             11000  \n",
       "MachineID                                                           1057373  \n",
       "ModelID                                                               17311  \n",
       "datasource                                                              121  \n",
       "auctioneerID                                                              3  \n",
       "YearMade                                                               2007  \n",
       "MachineHoursCurrentMeter                                                722  \n",
       "UsageBand                                                            Medium  \n",
       "saledate                                                2009-07-23 00:00:00  \n",
       "fiModelDesc                                                            S175  \n",
       "fiBaseModel                                                            S175  \n",
       "fiSecondaryDesc                                                         NaN  \n",
       "fiModelSeries                                                           NaN  \n",
       "fiModelDescriptor                                                       NaN  \n",
       "ProductSize                                                             NaN  \n",
       "fiProductClassDesc        Skid Steer Loader - 1601.0 to 1751.0 Lb Operat...  \n",
       "state                                                              New York  \n",
       "ProductGroup                                                            SSL  \n",
       "ProductGroupDesc                                         Skid Steer Loaders  \n",
       "Drive_System                                                            NaN  \n",
       "Enclosure                                                             EROPS  \n",
       "Forks                                                   None or Unspecified  \n",
       "Pad_Type                                                                NaN  \n",
       "Ride_Control                                                            NaN  \n",
       "Stick                                                                   NaN  \n",
       "Transmission                                                            NaN  \n",
       "Turbocharged                                                            NaN  \n",
       "Blade_Extension                                                         NaN  \n",
       "Blade_Width                                                             NaN  \n",
       "Enclosure_Type                                                          NaN  \n",
       "Engine_Horsepower                                                       NaN  \n",
       "Hydraulics                                                        Auxiliary  \n",
       "Pushblock                                                               NaN  \n",
       "Ripper                                                                  NaN  \n",
       "Scarifier                                                               NaN  \n",
       "Tip_Control                                                             NaN  \n",
       "Tire_Size                                                               NaN  \n",
       "Coupler                                                 None or Unspecified  \n",
       "Coupler_System                                          None or Unspecified  \n",
       "Grouser_Tracks                                          None or Unspecified  \n",
       "Hydraulics_Flow                                                    Standard  \n",
       "Track_Type                                                              NaN  \n",
       "Undercarriage_Pad_Width                                                 NaN  \n",
       "Stick_Length                                                            NaN  \n",
       "Thumb                                                                   NaN  \n",
       "Pattern_Changer                                                         NaN  \n",
       "Grouser_Type                                                            NaN  \n",
       "Backhoe_Mounting                                                        NaN  \n",
       "Blade_Type                                                              NaN  \n",
       "Travel_Controls                                                         NaN  \n",
       "Differential_Type                                                       NaN  \n",
       "Steering_Controls                                                       NaN  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head().T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    2006-11-16\n",
       "1    2004-03-26\n",
       "2    2004-02-26\n",
       "3    2011-05-19\n",
       "4    2009-07-23\n",
       "5    2008-12-18\n",
       "6    2004-08-26\n",
       "7    2005-11-17\n",
       "8    2009-08-27\n",
       "9    2007-08-09\n",
       "10   2008-08-21\n",
       "11   2006-08-24\n",
       "12   2005-10-20\n",
       "13   2006-01-26\n",
       "14   2006-01-03\n",
       "15   2006-11-16\n",
       "16   2007-06-14\n",
       "17   2010-01-28\n",
       "18   2006-03-09\n",
       "19   2005-11-17\n",
       "Name: saledate, dtype: datetime64[ns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.saledate.head(20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sort DataFrame by saledate\n",
    "\n",
    "As we're working on a time series problem and trying to predict future examples given past examples, it makes sense to sort our data by date."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "205615   1989-01-17\n",
       "274835   1989-01-31\n",
       "141296   1989-01-31\n",
       "212552   1989-01-31\n",
       "62755    1989-01-31\n",
       "54653    1989-01-31\n",
       "81383    1989-01-31\n",
       "204924   1989-01-31\n",
       "135376   1989-01-31\n",
       "113390   1989-01-31\n",
       "113394   1989-01-31\n",
       "116419   1989-01-31\n",
       "32138    1989-01-31\n",
       "127610   1989-01-31\n",
       "76171    1989-01-31\n",
       "127000   1989-01-31\n",
       "128130   1989-01-31\n",
       "127626   1989-01-31\n",
       "55455    1989-01-31\n",
       "55454    1989-01-31\n",
       "Name: saledate, dtype: datetime64[ns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Sort DataFrame in date order\n",
    "df.sort_values(by=[\"saledate\"], inplace=True, ascending=True)\n",
    "df.saledate.head(20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Make a copy of the original DataFrame\n",
    "\n",
    "Since we're going to be manipulating the data, we'll make a copy of the original DataFrame and perform our changes there.\n",
    "\n",
    "This will keep the original DataFrame in tact if we need it again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make a copy of the original DataFrame to perform edits on\n",
    "df_tmp = df.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Add datetime parameters for saledate column\n",
    "\n",
    "Why?\n",
    "\n",
    "So we can enrich our dataset with as much information as possible.\n",
    "\n",
    "Because we imported the data using `read_csv()` and we asked pandas to parse the dates using `parase_dates=[\"saledate\"]`, we can now access the [different datetime attributes](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DatetimeIndex.html) of the `saledate` column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add datetime parameters for saledate\n",
    "df_tmp[\"saleYear\"] = df_tmp.saledate.dt.year\n",
    "df_tmp[\"saleMonth\"] = df_tmp.saledate.dt.month\n",
    "df_tmp[\"saleDay\"] = df_tmp.saledate.dt.day\n",
    "df_tmp[\"saleDayofweek\"] = df_tmp.saledate.dt.dayofweek\n",
    "df_tmp[\"saleDayofyear\"] = df_tmp.saledate.dt.dayofyear\n",
    "\n",
    "# Drop original saledate\n",
    "df_tmp.drop(\"saledate\", axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could add more of these style of columns, such as, whether it was the start or end of a quarter but these will do for now.\n",
    "\n",
    "**Challenge:** See what other [datetime attributes](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DatetimeIndex.html) you can add to `df_tmp` using a similar technique to what we've used above. Hint: check the bottom of the pandas.DatetimeIndex docs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>205615</th>\n",
       "      <th>274835</th>\n",
       "      <th>141296</th>\n",
       "      <th>212552</th>\n",
       "      <th>62755</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>SalesID</th>\n",
       "      <td>1646770</td>\n",
       "      <td>1821514</td>\n",
       "      <td>1505138</td>\n",
       "      <td>1671174</td>\n",
       "      <td>1329056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SalePrice</th>\n",
       "      <td>9500</td>\n",
       "      <td>14000</td>\n",
       "      <td>50000</td>\n",
       "      <td>16000</td>\n",
       "      <td>22000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MachineID</th>\n",
       "      <td>1126363</td>\n",
       "      <td>1194089</td>\n",
       "      <td>1473654</td>\n",
       "      <td>1327630</td>\n",
       "      <td>1336053</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ModelID</th>\n",
       "      <td>8434</td>\n",
       "      <td>10150</td>\n",
       "      <td>4139</td>\n",
       "      <td>8591</td>\n",
       "      <td>4089</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>datasource</th>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>auctioneerID</th>\n",
       "      <td>18</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>YearMade</th>\n",
       "      <td>1974</td>\n",
       "      <td>1980</td>\n",
       "      <td>1978</td>\n",
       "      <td>1980</td>\n",
       "      <td>1984</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MachineHoursCurrentMeter</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UsageBand</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiModelDesc</th>\n",
       "      <td>TD20</td>\n",
       "      <td>A66</td>\n",
       "      <td>D7G</td>\n",
       "      <td>A62</td>\n",
       "      <td>D3B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiBaseModel</th>\n",
       "      <td>TD20</td>\n",
       "      <td>A66</td>\n",
       "      <td>D7</td>\n",
       "      <td>A62</td>\n",
       "      <td>D3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiSecondaryDesc</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>G</td>\n",
       "      <td>NaN</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiModelSeries</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiModelDescriptor</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ProductSize</th>\n",
       "      <td>Medium</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Large</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiProductClassDesc</th>\n",
       "      <td>Track Type Tractor, Dozer - 105.0 to 130.0 Hor...</td>\n",
       "      <td>Wheel Loader - 120.0 to 135.0 Horsepower</td>\n",
       "      <td>Track Type Tractor, Dozer - 190.0 to 260.0 Hor...</td>\n",
       "      <td>Wheel Loader - Unidentified</td>\n",
       "      <td>Track Type Tractor, Dozer - 20.0 to 75.0 Horse...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>state</th>\n",
       "      <td>Texas</td>\n",
       "      <td>Florida</td>\n",
       "      <td>Florida</td>\n",
       "      <td>Florida</td>\n",
       "      <td>Florida</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ProductGroup</th>\n",
       "      <td>TTT</td>\n",
       "      <td>WL</td>\n",
       "      <td>TTT</td>\n",
       "      <td>WL</td>\n",
       "      <td>TTT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ProductGroupDesc</th>\n",
       "      <td>Track Type Tractors</td>\n",
       "      <td>Wheel Loader</td>\n",
       "      <td>Track Type Tractors</td>\n",
       "      <td>Wheel Loader</td>\n",
       "      <td>Track Type Tractors</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Drive_System</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Enclosure</th>\n",
       "      <td>OROPS</td>\n",
       "      <td>OROPS</td>\n",
       "      <td>OROPS</td>\n",
       "      <td>EROPS</td>\n",
       "      <td>OROPS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Forks</th>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pad_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ride_Control</th>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Stick</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Transmission</th>\n",
       "      <td>Direct Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Turbocharged</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blade_Extension</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blade_Width</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Enclosure_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Engine_Horsepower</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hydraulics</th>\n",
       "      <td>2 Valve</td>\n",
       "      <td>2 Valve</td>\n",
       "      <td>2 Valve</td>\n",
       "      <td>2 Valve</td>\n",
       "      <td>2 Valve</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pushblock</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ripper</th>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Scarifier</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tip_Control</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tire_Size</th>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Coupler</th>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Coupler_System</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Grouser_Tracks</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hydraulics_Flow</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Track_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Undercarriage_Pad_Width</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Stick_Length</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Thumb</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pattern_Changer</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Grouser_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Backhoe_Mounting</th>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blade_Type</th>\n",
       "      <td>Straight</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Straight</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PAT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Travel_Controls</th>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Lever</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Differential_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Steering_Controls</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Conventional</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Conventional</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleYear</th>\n",
       "      <td>1989</td>\n",
       "      <td>1989</td>\n",
       "      <td>1989</td>\n",
       "      <td>1989</td>\n",
       "      <td>1989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleMonth</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleDay</th>\n",
       "      <td>17</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleDayofweek</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleDayofyear</th>\n",
       "      <td>17</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                     205615  \\\n",
       "SalesID                                                             1646770   \n",
       "SalePrice                                                              9500   \n",
       "MachineID                                                           1126363   \n",
       "ModelID                                                                8434   \n",
       "datasource                                                              132   \n",
       "auctioneerID                                                             18   \n",
       "YearMade                                                               1974   \n",
       "MachineHoursCurrentMeter                                                NaN   \n",
       "UsageBand                                                               NaN   \n",
       "fiModelDesc                                                            TD20   \n",
       "fiBaseModel                                                            TD20   \n",
       "fiSecondaryDesc                                                         NaN   \n",
       "fiModelSeries                                                           NaN   \n",
       "fiModelDescriptor                                                       NaN   \n",
       "ProductSize                                                          Medium   \n",
       "fiProductClassDesc        Track Type Tractor, Dozer - 105.0 to 130.0 Hor...   \n",
       "state                                                                 Texas   \n",
       "ProductGroup                                                            TTT   \n",
       "ProductGroupDesc                                        Track Type Tractors   \n",
       "Drive_System                                                            NaN   \n",
       "Enclosure                                                             OROPS   \n",
       "Forks                                                                   NaN   \n",
       "Pad_Type                                                                NaN   \n",
       "Ride_Control                                                            NaN   \n",
       "Stick                                                                   NaN   \n",
       "Transmission                                                   Direct Drive   \n",
       "Turbocharged                                                            NaN   \n",
       "Blade_Extension                                                         NaN   \n",
       "Blade_Width                                                             NaN   \n",
       "Enclosure_Type                                                          NaN   \n",
       "Engine_Horsepower                                                       NaN   \n",
       "Hydraulics                                                          2 Valve   \n",
       "Pushblock                                                               NaN   \n",
       "Ripper                                                  None or Unspecified   \n",
       "Scarifier                                                               NaN   \n",
       "Tip_Control                                                             NaN   \n",
       "Tire_Size                                                               NaN   \n",
       "Coupler                                                                 NaN   \n",
       "Coupler_System                                                          NaN   \n",
       "Grouser_Tracks                                                          NaN   \n",
       "Hydraulics_Flow                                                         NaN   \n",
       "Track_Type                                                              NaN   \n",
       "Undercarriage_Pad_Width                                                 NaN   \n",
       "Stick_Length                                                            NaN   \n",
       "Thumb                                                                   NaN   \n",
       "Pattern_Changer                                                         NaN   \n",
       "Grouser_Type                                                            NaN   \n",
       "Backhoe_Mounting                                        None or Unspecified   \n",
       "Blade_Type                                                         Straight   \n",
       "Travel_Controls                                         None or Unspecified   \n",
       "Differential_Type                                                       NaN   \n",
       "Steering_Controls                                                       NaN   \n",
       "saleYear                                                               1989   \n",
       "saleMonth                                                                 1   \n",
       "saleDay                                                                  17   \n",
       "saleDayofweek                                                             1   \n",
       "saleDayofyear                                                            17   \n",
       "\n",
       "                                                            274835  \\\n",
       "SalesID                                                    1821514   \n",
       "SalePrice                                                    14000   \n",
       "MachineID                                                  1194089   \n",
       "ModelID                                                      10150   \n",
       "datasource                                                     132   \n",
       "auctioneerID                                                    99   \n",
       "YearMade                                                      1980   \n",
       "MachineHoursCurrentMeter                                       NaN   \n",
       "UsageBand                                                      NaN   \n",
       "fiModelDesc                                                    A66   \n",
       "fiBaseModel                                                    A66   \n",
       "fiSecondaryDesc                                                NaN   \n",
       "fiModelSeries                                                  NaN   \n",
       "fiModelDescriptor                                              NaN   \n",
       "ProductSize                                                    NaN   \n",
       "fiProductClassDesc        Wheel Loader - 120.0 to 135.0 Horsepower   \n",
       "state                                                      Florida   \n",
       "ProductGroup                                                    WL   \n",
       "ProductGroupDesc                                      Wheel Loader   \n",
       "Drive_System                                                   NaN   \n",
       "Enclosure                                                    OROPS   \n",
       "Forks                                          None or Unspecified   \n",
       "Pad_Type                                                       NaN   \n",
       "Ride_Control                                   None or Unspecified   \n",
       "Stick                                                          NaN   \n",
       "Transmission                                                   NaN   \n",
       "Turbocharged                                                   NaN   \n",
       "Blade_Extension                                                NaN   \n",
       "Blade_Width                                                    NaN   \n",
       "Enclosure_Type                                                 NaN   \n",
       "Engine_Horsepower                                              NaN   \n",
       "Hydraulics                                                 2 Valve   \n",
       "Pushblock                                                      NaN   \n",
       "Ripper                                                         NaN   \n",
       "Scarifier                                                      NaN   \n",
       "Tip_Control                                                    NaN   \n",
       "Tire_Size                                      None or Unspecified   \n",
       "Coupler                                        None or Unspecified   \n",
       "Coupler_System                                                 NaN   \n",
       "Grouser_Tracks                                                 NaN   \n",
       "Hydraulics_Flow                                                NaN   \n",
       "Track_Type                                                     NaN   \n",
       "Undercarriage_Pad_Width                                        NaN   \n",
       "Stick_Length                                                   NaN   \n",
       "Thumb                                                          NaN   \n",
       "Pattern_Changer                                                NaN   \n",
       "Grouser_Type                                                   NaN   \n",
       "Backhoe_Mounting                                               NaN   \n",
       "Blade_Type                                                     NaN   \n",
       "Travel_Controls                                                NaN   \n",
       "Differential_Type                                         Standard   \n",
       "Steering_Controls                                     Conventional   \n",
       "saleYear                                                      1989   \n",
       "saleMonth                                                        1   \n",
       "saleDay                                                         31   \n",
       "saleDayofweek                                                    1   \n",
       "saleDayofyear                                                   31   \n",
       "\n",
       "                                                                     141296  \\\n",
       "SalesID                                                             1505138   \n",
       "SalePrice                                                             50000   \n",
       "MachineID                                                           1473654   \n",
       "ModelID                                                                4139   \n",
       "datasource                                                              132   \n",
       "auctioneerID                                                             99   \n",
       "YearMade                                                               1978   \n",
       "MachineHoursCurrentMeter                                                NaN   \n",
       "UsageBand                                                               NaN   \n",
       "fiModelDesc                                                             D7G   \n",
       "fiBaseModel                                                              D7   \n",
       "fiSecondaryDesc                                                           G   \n",
       "fiModelSeries                                                           NaN   \n",
       "fiModelDescriptor                                                       NaN   \n",
       "ProductSize                                                           Large   \n",
       "fiProductClassDesc        Track Type Tractor, Dozer - 190.0 to 260.0 Hor...   \n",
       "state                                                               Florida   \n",
       "ProductGroup                                                            TTT   \n",
       "ProductGroupDesc                                        Track Type Tractors   \n",
       "Drive_System                                                            NaN   \n",
       "Enclosure                                                             OROPS   \n",
       "Forks                                                                   NaN   \n",
       "Pad_Type                                                                NaN   \n",
       "Ride_Control                                                            NaN   \n",
       "Stick                                                                   NaN   \n",
       "Transmission                                                       Standard   \n",
       "Turbocharged                                                            NaN   \n",
       "Blade_Extension                                                         NaN   \n",
       "Blade_Width                                                             NaN   \n",
       "Enclosure_Type                                                          NaN   \n",
       "Engine_Horsepower                                                       NaN   \n",
       "Hydraulics                                                          2 Valve   \n",
       "Pushblock                                                               NaN   \n",
       "Ripper                                                  None or Unspecified   \n",
       "Scarifier                                                               NaN   \n",
       "Tip_Control                                                             NaN   \n",
       "Tire_Size                                                               NaN   \n",
       "Coupler                                                                 NaN   \n",
       "Coupler_System                                                          NaN   \n",
       "Grouser_Tracks                                                          NaN   \n",
       "Hydraulics_Flow                                                         NaN   \n",
       "Track_Type                                                              NaN   \n",
       "Undercarriage_Pad_Width                                                 NaN   \n",
       "Stick_Length                                                            NaN   \n",
       "Thumb                                                                   NaN   \n",
       "Pattern_Changer                                                         NaN   \n",
       "Grouser_Type                                                            NaN   \n",
       "Backhoe_Mounting                                        None or Unspecified   \n",
       "Blade_Type                                                         Straight   \n",
       "Travel_Controls                                         None or Unspecified   \n",
       "Differential_Type                                                       NaN   \n",
       "Steering_Controls                                                       NaN   \n",
       "saleYear                                                               1989   \n",
       "saleMonth                                                                 1   \n",
       "saleDay                                                                  31   \n",
       "saleDayofweek                                                             1   \n",
       "saleDayofyear                                                            31   \n",
       "\n",
       "                                               212552  \\\n",
       "SalesID                                       1671174   \n",
       "SalePrice                                       16000   \n",
       "MachineID                                     1327630   \n",
       "ModelID                                          8591   \n",
       "datasource                                        132   \n",
       "auctioneerID                                       99   \n",
       "YearMade                                         1980   \n",
       "MachineHoursCurrentMeter                          NaN   \n",
       "UsageBand                                         NaN   \n",
       "fiModelDesc                                       A62   \n",
       "fiBaseModel                                       A62   \n",
       "fiSecondaryDesc                                   NaN   \n",
       "fiModelSeries                                     NaN   \n",
       "fiModelDescriptor                                 NaN   \n",
       "ProductSize                                       NaN   \n",
       "fiProductClassDesc        Wheel Loader - Unidentified   \n",
       "state                                         Florida   \n",
       "ProductGroup                                       WL   \n",
       "ProductGroupDesc                         Wheel Loader   \n",
       "Drive_System                                      NaN   \n",
       "Enclosure                                       EROPS   \n",
       "Forks                             None or Unspecified   \n",
       "Pad_Type                                          NaN   \n",
       "Ride_Control                      None or Unspecified   \n",
       "Stick                                             NaN   \n",
       "Transmission                                      NaN   \n",
       "Turbocharged                                      NaN   \n",
       "Blade_Extension                                   NaN   \n",
       "Blade_Width                                       NaN   \n",
       "Enclosure_Type                                    NaN   \n",
       "Engine_Horsepower                                 NaN   \n",
       "Hydraulics                                    2 Valve   \n",
       "Pushblock                                         NaN   \n",
       "Ripper                                            NaN   \n",
       "Scarifier                                         NaN   \n",
       "Tip_Control                                       NaN   \n",
       "Tire_Size                         None or Unspecified   \n",
       "Coupler                           None or Unspecified   \n",
       "Coupler_System                                    NaN   \n",
       "Grouser_Tracks                                    NaN   \n",
       "Hydraulics_Flow                                   NaN   \n",
       "Track_Type                                        NaN   \n",
       "Undercarriage_Pad_Width                           NaN   \n",
       "Stick_Length                                      NaN   \n",
       "Thumb                                             NaN   \n",
       "Pattern_Changer                                   NaN   \n",
       "Grouser_Type                                      NaN   \n",
       "Backhoe_Mounting                                  NaN   \n",
       "Blade_Type                                        NaN   \n",
       "Travel_Controls                                   NaN   \n",
       "Differential_Type                            Standard   \n",
       "Steering_Controls                        Conventional   \n",
       "saleYear                                         1989   \n",
       "saleMonth                                           1   \n",
       "saleDay                                            31   \n",
       "saleDayofweek                                       1   \n",
       "saleDayofyear                                      31   \n",
       "\n",
       "                                                                     62755   \n",
       "SalesID                                                             1329056  \n",
       "SalePrice                                                             22000  \n",
       "MachineID                                                           1336053  \n",
       "ModelID                                                                4089  \n",
       "datasource                                                              132  \n",
       "auctioneerID                                                             99  \n",
       "YearMade                                                               1984  \n",
       "MachineHoursCurrentMeter                                                NaN  \n",
       "UsageBand                                                               NaN  \n",
       "fiModelDesc                                                             D3B  \n",
       "fiBaseModel                                                              D3  \n",
       "fiSecondaryDesc                                                           B  \n",
       "fiModelSeries                                                           NaN  \n",
       "fiModelDescriptor                                                       NaN  \n",
       "ProductSize                                                             NaN  \n",
       "fiProductClassDesc        Track Type Tractor, Dozer - 20.0 to 75.0 Horse...  \n",
       "state                                                               Florida  \n",
       "ProductGroup                                                            TTT  \n",
       "ProductGroupDesc                                        Track Type Tractors  \n",
       "Drive_System                                                            NaN  \n",
       "Enclosure                                                             OROPS  \n",
       "Forks                                                                   NaN  \n",
       "Pad_Type                                                                NaN  \n",
       "Ride_Control                                                            NaN  \n",
       "Stick                                                                   NaN  \n",
       "Transmission                                                       Standard  \n",
       "Turbocharged                                                            NaN  \n",
       "Blade_Extension                                                         NaN  \n",
       "Blade_Width                                                             NaN  \n",
       "Enclosure_Type                                                          NaN  \n",
       "Engine_Horsepower                                                       NaN  \n",
       "Hydraulics                                                          2 Valve  \n",
       "Pushblock                                                               NaN  \n",
       "Ripper                                                  None or Unspecified  \n",
       "Scarifier                                                               NaN  \n",
       "Tip_Control                                                             NaN  \n",
       "Tire_Size                                                               NaN  \n",
       "Coupler                                                                 NaN  \n",
       "Coupler_System                                                          NaN  \n",
       "Grouser_Tracks                                                          NaN  \n",
       "Hydraulics_Flow                                                         NaN  \n",
       "Track_Type                                                              NaN  \n",
       "Undercarriage_Pad_Width                                                 NaN  \n",
       "Stick_Length                                                            NaN  \n",
       "Thumb                                                                   NaN  \n",
       "Pattern_Changer                                                         NaN  \n",
       "Grouser_Type                                                            NaN  \n",
       "Backhoe_Mounting                                        None or Unspecified  \n",
       "Blade_Type                                                              PAT  \n",
       "Travel_Controls                                                       Lever  \n",
       "Differential_Type                                                       NaN  \n",
       "Steering_Controls                                                       NaN  \n",
       "saleYear                                                               1989  \n",
       "saleMonth                                                                 1  \n",
       "saleDay                                                                  31  \n",
       "saleDayofweek                                                             1  \n",
       "saleDayofyear                                                            31  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_tmp.head().T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Florida           67320\n",
       "Texas             53110\n",
       "California        29761\n",
       "Washington        16222\n",
       "Georgia           14633\n",
       "Maryland          13322\n",
       "Mississippi       13240\n",
       "Ohio              12369\n",
       "Illinois          11540\n",
       "Colorado          11529\n",
       "New Jersey        11156\n",
       "North Carolina    10636\n",
       "Tennessee         10298\n",
       "Alabama           10292\n",
       "Pennsylvania      10234\n",
       "South Carolina     9951\n",
       "Arizona            9364\n",
       "New York           8639\n",
       "Connecticut        8276\n",
       "Minnesota          7885\n",
       "Missouri           7178\n",
       "Nevada             6932\n",
       "Louisiana          6627\n",
       "Kentucky           5351\n",
       "Maine              5096\n",
       "Indiana            4124\n",
       "Arkansas           3933\n",
       "New Mexico         3631\n",
       "Utah               3046\n",
       "Unspecified        2801\n",
       "Wisconsin          2745\n",
       "New Hampshire      2738\n",
       "Virginia           2353\n",
       "Idaho              2025\n",
       "Oregon             1911\n",
       "Michigan           1831\n",
       "Wyoming            1672\n",
       "Montana            1336\n",
       "Iowa               1336\n",
       "Oklahoma           1326\n",
       "Nebraska            866\n",
       "West Virginia       840\n",
       "Kansas              667\n",
       "Delaware            510\n",
       "North Dakota        480\n",
       "Alaska              430\n",
       "Massachusetts       347\n",
       "Vermont             300\n",
       "South Dakota        244\n",
       "Hawaii              118\n",
       "Rhode Island         83\n",
       "Puerto Rico          42\n",
       "Washington DC         2\n",
       "Name: state, dtype: int64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check the different values of different columns\n",
    "df_tmp.state.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Modelling\n",
    "\n",
    "We've explored our dataset a little as well as enriched it with some datetime attributes, now let's try to model.\n",
    "\n",
    "Why model so early?\n",
    "\n",
    "We know the evaluation metric we're heading towards. We could spend more time doing exploratory data analysis (EDA), finding more out about the data ourselves but what we'll do instead is use a machine learning model to help us do EDA.\n",
    "\n",
    "Remember, one of the biggest goals of starting any new machine learning project is reducing the time between experiments.\n",
    "\n",
    "Following the [Scikit-Learn machine learning map](https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html), we find a [RandomForestRegressor()](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html#sklearn-ensemble-randomforestregressor) might be a good candidate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "could not convert string to float: 'Low'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-17-907907c0c28e>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mmodel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mRandomForestRegressor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mn_jobs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdf_tmp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdrop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"SalePrice\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdf_tmp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSalePrice\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.7/site-packages/sklearn/ensemble/_forest.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, X, y, sample_weight)\u001b[0m\n\u001b[1;32m    293\u001b[0m         \"\"\"\n\u001b[1;32m    294\u001b[0m         \u001b[0;31m# Validate or convert input data\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 295\u001b[0;31m         \u001b[0mX\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcheck_array\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maccept_sparse\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"csc\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mDTYPE\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    296\u001b[0m         \u001b[0my\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcheck_array\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0my\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maccept_sparse\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'csc'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mensure_2d\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    297\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0msample_weight\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.7/site-packages/sklearn/utils/validation.py\u001b[0m in \u001b[0;36mcheck_array\u001b[0;34m(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)\u001b[0m\n\u001b[1;32m    529\u001b[0m                     \u001b[0marray\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marray\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mastype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdtype\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcasting\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"unsafe\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcopy\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    530\u001b[0m                 \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 531\u001b[0;31m                     \u001b[0marray\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0masarray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marray\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0morder\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0morder\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdtype\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    532\u001b[0m             \u001b[0;32mexcept\u001b[0m \u001b[0mComplexWarning\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    533\u001b[0m                 raise ValueError(\"Complex data not supported\\n\"\n",
      "\u001b[0;32m~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.7/site-packages/numpy/core/_asarray.py\u001b[0m in \u001b[0;36masarray\u001b[0;34m(a, dtype, order)\u001b[0m\n\u001b[1;32m     83\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     84\u001b[0m     \"\"\"\n\u001b[0;32m---> 85\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0marray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcopy\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0morder\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0morder\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     86\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     87\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: could not convert string to float: 'Low'"
     ]
    }
   ],
   "source": [
    "# This won't work since we've got missing numbers and categories\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "\n",
    "model = RandomForestRegressor(n_jobs=-1)\n",
    "model.fit(df_tmp.drop(\"SalePrice\", axis=1), df_tmp.SalePrice)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 412698 entries, 205615 to 409203\n",
      "Data columns (total 57 columns):\n",
      " #   Column                    Non-Null Count   Dtype  \n",
      "---  ------                    --------------   -----  \n",
      " 0   SalesID                   412698 non-null  int64  \n",
      " 1   SalePrice                 412698 non-null  float64\n",
      " 2   MachineID                 412698 non-null  int64  \n",
      " 3   ModelID                   412698 non-null  int64  \n",
      " 4   datasource                412698 non-null  int64  \n",
      " 5   auctioneerID              392562 non-null  float64\n",
      " 6   YearMade                  412698 non-null  int64  \n",
      " 7   MachineHoursCurrentMeter  147504 non-null  float64\n",
      " 8   UsageBand                 73670 non-null   object \n",
      " 9   fiModelDesc               412698 non-null  object \n",
      " 10  fiBaseModel               412698 non-null  object \n",
      " 11  fiSecondaryDesc           271971 non-null  object \n",
      " 12  fiModelSeries             58667 non-null   object \n",
      " 13  fiModelDescriptor         74816 non-null   object \n",
      " 14  ProductSize               196093 non-null  object \n",
      " 15  fiProductClassDesc        412698 non-null  object \n",
      " 16  state                     412698 non-null  object \n",
      " 17  ProductGroup              412698 non-null  object \n",
      " 18  ProductGroupDesc          412698 non-null  object \n",
      " 19  Drive_System              107087 non-null  object \n",
      " 20  Enclosure                 412364 non-null  object \n",
      " 21  Forks                     197715 non-null  object \n",
      " 22  Pad_Type                  81096 non-null   object \n",
      " 23  Ride_Control              152728 non-null  object \n",
      " 24  Stick                     81096 non-null   object \n",
      " 25  Transmission              188007 non-null  object \n",
      " 26  Turbocharged              81096 non-null   object \n",
      " 27  Blade_Extension           25983 non-null   object \n",
      " 28  Blade_Width               25983 non-null   object \n",
      " 29  Enclosure_Type            25983 non-null   object \n",
      " 30  Engine_Horsepower         25983 non-null   object \n",
      " 31  Hydraulics                330133 non-null  object \n",
      " 32  Pushblock                 25983 non-null   object \n",
      " 33  Ripper                    106945 non-null  object \n",
      " 34  Scarifier                 25994 non-null   object \n",
      " 35  Tip_Control               25983 non-null   object \n",
      " 36  Tire_Size                 97638 non-null   object \n",
      " 37  Coupler                   220679 non-null  object \n",
      " 38  Coupler_System            44974 non-null   object \n",
      " 39  Grouser_Tracks            44875 non-null   object \n",
      " 40  Hydraulics_Flow           44875 non-null   object \n",
      " 41  Track_Type                102193 non-null  object \n",
      " 42  Undercarriage_Pad_Width   102916 non-null  object \n",
      " 43  Stick_Length              102261 non-null  object \n",
      " 44  Thumb                     102332 non-null  object \n",
      " 45  Pattern_Changer           102261 non-null  object \n",
      " 46  Grouser_Type              102193 non-null  object \n",
      " 47  Backhoe_Mounting          80712 non-null   object \n",
      " 48  Blade_Type                81875 non-null   object \n",
      " 49  Travel_Controls           81877 non-null   object \n",
      " 50  Differential_Type         71564 non-null   object \n",
      " 51  Steering_Controls         71522 non-null   object \n",
      " 52  saleYear                  412698 non-null  int64  \n",
      " 53  saleMonth                 412698 non-null  int64  \n",
      " 54  saleDay                   412698 non-null  int64  \n",
      " 55  saleDayofweek             412698 non-null  int64  \n",
      " 56  saleDayofyear             412698 non-null  int64  \n",
      "dtypes: float64(3), int64(10), object(44)\n",
      "memory usage: 182.6+ MB\n"
     ]
    }
   ],
   "source": [
    "# Check for missing categories and different datatypes\n",
    "df_tmp.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SalesID                          0\n",
       "SalePrice                        0\n",
       "MachineID                        0\n",
       "ModelID                          0\n",
       "datasource                       0\n",
       "auctioneerID                 20136\n",
       "YearMade                         0\n",
       "MachineHoursCurrentMeter    265194\n",
       "UsageBand                   339028\n",
       "fiModelDesc                      0\n",
       "fiBaseModel                      0\n",
       "fiSecondaryDesc             140727\n",
       "fiModelSeries               354031\n",
       "fiModelDescriptor           337882\n",
       "ProductSize                 216605\n",
       "fiProductClassDesc               0\n",
       "state                            0\n",
       "ProductGroup                     0\n",
       "ProductGroupDesc                 0\n",
       "Drive_System                305611\n",
       "Enclosure                      334\n",
       "Forks                       214983\n",
       "Pad_Type                    331602\n",
       "Ride_Control                259970\n",
       "Stick                       331602\n",
       "Transmission                224691\n",
       "Turbocharged                331602\n",
       "Blade_Extension             386715\n",
       "Blade_Width                 386715\n",
       "Enclosure_Type              386715\n",
       "Engine_Horsepower           386715\n",
       "Hydraulics                   82565\n",
       "Pushblock                   386715\n",
       "Ripper                      305753\n",
       "Scarifier                   386704\n",
       "Tip_Control                 386715\n",
       "Tire_Size                   315060\n",
       "Coupler                     192019\n",
       "Coupler_System              367724\n",
       "Grouser_Tracks              367823\n",
       "Hydraulics_Flow             367823\n",
       "Track_Type                  310505\n",
       "Undercarriage_Pad_Width     309782\n",
       "Stick_Length                310437\n",
       "Thumb                       310366\n",
       "Pattern_Changer             310437\n",
       "Grouser_Type                310505\n",
       "Backhoe_Mounting            331986\n",
       "Blade_Type                  330823\n",
       "Travel_Controls             330821\n",
       "Differential_Type           341134\n",
       "Steering_Controls           341176\n",
       "saleYear                         0\n",
       "saleMonth                        0\n",
       "saleDay                          0\n",
       "saleDayofweek                    0\n",
       "saleDayofyear                    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check for missing values\n",
    "df_tmp.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convert strings to categories\n",
    "\n",
    "One way to help turn all of our data into numbers is to convert the columns with the string datatype into a category datatype.\n",
    "\n",
    "To do this we can use the [pandas types API](https://pandas.pydata.org/pandas-docs/stable/reference/general_utility_functions.html#data-types-related-functionality) which allows us to interact and manipulate the types of data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>205615</th>\n",
       "      <th>274835</th>\n",
       "      <th>141296</th>\n",
       "      <th>212552</th>\n",
       "      <th>62755</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>SalesID</th>\n",
       "      <td>1646770</td>\n",
       "      <td>1821514</td>\n",
       "      <td>1505138</td>\n",
       "      <td>1671174</td>\n",
       "      <td>1329056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SalePrice</th>\n",
       "      <td>9500</td>\n",
       "      <td>14000</td>\n",
       "      <td>50000</td>\n",
       "      <td>16000</td>\n",
       "      <td>22000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MachineID</th>\n",
       "      <td>1126363</td>\n",
       "      <td>1194089</td>\n",
       "      <td>1473654</td>\n",
       "      <td>1327630</td>\n",
       "      <td>1336053</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ModelID</th>\n",
       "      <td>8434</td>\n",
       "      <td>10150</td>\n",
       "      <td>4139</td>\n",
       "      <td>8591</td>\n",
       "      <td>4089</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>datasource</th>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>auctioneerID</th>\n",
       "      <td>18</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>YearMade</th>\n",
       "      <td>1974</td>\n",
       "      <td>1980</td>\n",
       "      <td>1978</td>\n",
       "      <td>1980</td>\n",
       "      <td>1984</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MachineHoursCurrentMeter</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UsageBand</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiModelDesc</th>\n",
       "      <td>TD20</td>\n",
       "      <td>A66</td>\n",
       "      <td>D7G</td>\n",
       "      <td>A62</td>\n",
       "      <td>D3B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiBaseModel</th>\n",
       "      <td>TD20</td>\n",
       "      <td>A66</td>\n",
       "      <td>D7</td>\n",
       "      <td>A62</td>\n",
       "      <td>D3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiSecondaryDesc</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>G</td>\n",
       "      <td>NaN</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiModelSeries</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiModelDescriptor</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ProductSize</th>\n",
       "      <td>Medium</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Large</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiProductClassDesc</th>\n",
       "      <td>Track Type Tractor, Dozer - 105.0 to 130.0 Hor...</td>\n",
       "      <td>Wheel Loader - 120.0 to 135.0 Horsepower</td>\n",
       "      <td>Track Type Tractor, Dozer - 190.0 to 260.0 Hor...</td>\n",
       "      <td>Wheel Loader - Unidentified</td>\n",
       "      <td>Track Type Tractor, Dozer - 20.0 to 75.0 Horse...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>state</th>\n",
       "      <td>Texas</td>\n",
       "      <td>Florida</td>\n",
       "      <td>Florida</td>\n",
       "      <td>Florida</td>\n",
       "      <td>Florida</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ProductGroup</th>\n",
       "      <td>TTT</td>\n",
       "      <td>WL</td>\n",
       "      <td>TTT</td>\n",
       "      <td>WL</td>\n",
       "      <td>TTT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ProductGroupDesc</th>\n",
       "      <td>Track Type Tractors</td>\n",
       "      <td>Wheel Loader</td>\n",
       "      <td>Track Type Tractors</td>\n",
       "      <td>Wheel Loader</td>\n",
       "      <td>Track Type Tractors</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Drive_System</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Enclosure</th>\n",
       "      <td>OROPS</td>\n",
       "      <td>OROPS</td>\n",
       "      <td>OROPS</td>\n",
       "      <td>EROPS</td>\n",
       "      <td>OROPS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Forks</th>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pad_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ride_Control</th>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Stick</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Transmission</th>\n",
       "      <td>Direct Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Turbocharged</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blade_Extension</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blade_Width</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Enclosure_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Engine_Horsepower</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hydraulics</th>\n",
       "      <td>2 Valve</td>\n",
       "      <td>2 Valve</td>\n",
       "      <td>2 Valve</td>\n",
       "      <td>2 Valve</td>\n",
       "      <td>2 Valve</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pushblock</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ripper</th>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Scarifier</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tip_Control</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tire_Size</th>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Coupler</th>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Coupler_System</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Grouser_Tracks</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hydraulics_Flow</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Track_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Undercarriage_Pad_Width</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Stick_Length</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Thumb</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pattern_Changer</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Grouser_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Backhoe_Mounting</th>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blade_Type</th>\n",
       "      <td>Straight</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Straight</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PAT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Travel_Controls</th>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Lever</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Differential_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Steering_Controls</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Conventional</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Conventional</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleYear</th>\n",
       "      <td>1989</td>\n",
       "      <td>1989</td>\n",
       "      <td>1989</td>\n",
       "      <td>1989</td>\n",
       "      <td>1989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleMonth</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleDay</th>\n",
       "      <td>17</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleDayofweek</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleDayofyear</th>\n",
       "      <td>17</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                     205615  \\\n",
       "SalesID                                                             1646770   \n",
       "SalePrice                                                              9500   \n",
       "MachineID                                                           1126363   \n",
       "ModelID                                                                8434   \n",
       "datasource                                                              132   \n",
       "auctioneerID                                                             18   \n",
       "YearMade                                                               1974   \n",
       "MachineHoursCurrentMeter                                                NaN   \n",
       "UsageBand                                                               NaN   \n",
       "fiModelDesc                                                            TD20   \n",
       "fiBaseModel                                                            TD20   \n",
       "fiSecondaryDesc                                                         NaN   \n",
       "fiModelSeries                                                           NaN   \n",
       "fiModelDescriptor                                                       NaN   \n",
       "ProductSize                                                          Medium   \n",
       "fiProductClassDesc        Track Type Tractor, Dozer - 105.0 to 130.0 Hor...   \n",
       "state                                                                 Texas   \n",
       "ProductGroup                                                            TTT   \n",
       "ProductGroupDesc                                        Track Type Tractors   \n",
       "Drive_System                                                            NaN   \n",
       "Enclosure                                                             OROPS   \n",
       "Forks                                                                   NaN   \n",
       "Pad_Type                                                                NaN   \n",
       "Ride_Control                                                            NaN   \n",
       "Stick                                                                   NaN   \n",
       "Transmission                                                   Direct Drive   \n",
       "Turbocharged                                                            NaN   \n",
       "Blade_Extension                                                         NaN   \n",
       "Blade_Width                                                             NaN   \n",
       "Enclosure_Type                                                          NaN   \n",
       "Engine_Horsepower                                                       NaN   \n",
       "Hydraulics                                                          2 Valve   \n",
       "Pushblock                                                               NaN   \n",
       "Ripper                                                  None or Unspecified   \n",
       "Scarifier                                                               NaN   \n",
       "Tip_Control                                                             NaN   \n",
       "Tire_Size                                                               NaN   \n",
       "Coupler                                                                 NaN   \n",
       "Coupler_System                                                          NaN   \n",
       "Grouser_Tracks                                                          NaN   \n",
       "Hydraulics_Flow                                                         NaN   \n",
       "Track_Type                                                              NaN   \n",
       "Undercarriage_Pad_Width                                                 NaN   \n",
       "Stick_Length                                                            NaN   \n",
       "Thumb                                                                   NaN   \n",
       "Pattern_Changer                                                         NaN   \n",
       "Grouser_Type                                                            NaN   \n",
       "Backhoe_Mounting                                        None or Unspecified   \n",
       "Blade_Type                                                         Straight   \n",
       "Travel_Controls                                         None or Unspecified   \n",
       "Differential_Type                                                       NaN   \n",
       "Steering_Controls                                                       NaN   \n",
       "saleYear                                                               1989   \n",
       "saleMonth                                                                 1   \n",
       "saleDay                                                                  17   \n",
       "saleDayofweek                                                             1   \n",
       "saleDayofyear                                                            17   \n",
       "\n",
       "                                                            274835  \\\n",
       "SalesID                                                    1821514   \n",
       "SalePrice                                                    14000   \n",
       "MachineID                                                  1194089   \n",
       "ModelID                                                      10150   \n",
       "datasource                                                     132   \n",
       "auctioneerID                                                    99   \n",
       "YearMade                                                      1980   \n",
       "MachineHoursCurrentMeter                                       NaN   \n",
       "UsageBand                                                      NaN   \n",
       "fiModelDesc                                                    A66   \n",
       "fiBaseModel                                                    A66   \n",
       "fiSecondaryDesc                                                NaN   \n",
       "fiModelSeries                                                  NaN   \n",
       "fiModelDescriptor                                              NaN   \n",
       "ProductSize                                                    NaN   \n",
       "fiProductClassDesc        Wheel Loader - 120.0 to 135.0 Horsepower   \n",
       "state                                                      Florida   \n",
       "ProductGroup                                                    WL   \n",
       "ProductGroupDesc                                      Wheel Loader   \n",
       "Drive_System                                                   NaN   \n",
       "Enclosure                                                    OROPS   \n",
       "Forks                                          None or Unspecified   \n",
       "Pad_Type                                                       NaN   \n",
       "Ride_Control                                   None or Unspecified   \n",
       "Stick                                                          NaN   \n",
       "Transmission                                                   NaN   \n",
       "Turbocharged                                                   NaN   \n",
       "Blade_Extension                                                NaN   \n",
       "Blade_Width                                                    NaN   \n",
       "Enclosure_Type                                                 NaN   \n",
       "Engine_Horsepower                                              NaN   \n",
       "Hydraulics                                                 2 Valve   \n",
       "Pushblock                                                      NaN   \n",
       "Ripper                                                         NaN   \n",
       "Scarifier                                                      NaN   \n",
       "Tip_Control                                                    NaN   \n",
       "Tire_Size                                      None or Unspecified   \n",
       "Coupler                                        None or Unspecified   \n",
       "Coupler_System                                                 NaN   \n",
       "Grouser_Tracks                                                 NaN   \n",
       "Hydraulics_Flow                                                NaN   \n",
       "Track_Type                                                     NaN   \n",
       "Undercarriage_Pad_Width                                        NaN   \n",
       "Stick_Length                                                   NaN   \n",
       "Thumb                                                          NaN   \n",
       "Pattern_Changer                                                NaN   \n",
       "Grouser_Type                                                   NaN   \n",
       "Backhoe_Mounting                                               NaN   \n",
       "Blade_Type                                                     NaN   \n",
       "Travel_Controls                                                NaN   \n",
       "Differential_Type                                         Standard   \n",
       "Steering_Controls                                     Conventional   \n",
       "saleYear                                                      1989   \n",
       "saleMonth                                                        1   \n",
       "saleDay                                                         31   \n",
       "saleDayofweek                                                    1   \n",
       "saleDayofyear                                                   31   \n",
       "\n",
       "                                                                     141296  \\\n",
       "SalesID                                                             1505138   \n",
       "SalePrice                                                             50000   \n",
       "MachineID                                                           1473654   \n",
       "ModelID                                                                4139   \n",
       "datasource                                                              132   \n",
       "auctioneerID                                                             99   \n",
       "YearMade                                                               1978   \n",
       "MachineHoursCurrentMeter                                                NaN   \n",
       "UsageBand                                                               NaN   \n",
       "fiModelDesc                                                             D7G   \n",
       "fiBaseModel                                                              D7   \n",
       "fiSecondaryDesc                                                           G   \n",
       "fiModelSeries                                                           NaN   \n",
       "fiModelDescriptor                                                       NaN   \n",
       "ProductSize                                                           Large   \n",
       "fiProductClassDesc        Track Type Tractor, Dozer - 190.0 to 260.0 Hor...   \n",
       "state                                                               Florida   \n",
       "ProductGroup                                                            TTT   \n",
       "ProductGroupDesc                                        Track Type Tractors   \n",
       "Drive_System                                                            NaN   \n",
       "Enclosure                                                             OROPS   \n",
       "Forks                                                                   NaN   \n",
       "Pad_Type                                                                NaN   \n",
       "Ride_Control                                                            NaN   \n",
       "Stick                                                                   NaN   \n",
       "Transmission                                                       Standard   \n",
       "Turbocharged                                                            NaN   \n",
       "Blade_Extension                                                         NaN   \n",
       "Blade_Width                                                             NaN   \n",
       "Enclosure_Type                                                          NaN   \n",
       "Engine_Horsepower                                                       NaN   \n",
       "Hydraulics                                                          2 Valve   \n",
       "Pushblock                                                               NaN   \n",
       "Ripper                                                  None or Unspecified   \n",
       "Scarifier                                                               NaN   \n",
       "Tip_Control                                                             NaN   \n",
       "Tire_Size                                                               NaN   \n",
       "Coupler                                                                 NaN   \n",
       "Coupler_System                                                          NaN   \n",
       "Grouser_Tracks                                                          NaN   \n",
       "Hydraulics_Flow                                                         NaN   \n",
       "Track_Type                                                              NaN   \n",
       "Undercarriage_Pad_Width                                                 NaN   \n",
       "Stick_Length                                                            NaN   \n",
       "Thumb                                                                   NaN   \n",
       "Pattern_Changer                                                         NaN   \n",
       "Grouser_Type                                                            NaN   \n",
       "Backhoe_Mounting                                        None or Unspecified   \n",
       "Blade_Type                                                         Straight   \n",
       "Travel_Controls                                         None or Unspecified   \n",
       "Differential_Type                                                       NaN   \n",
       "Steering_Controls                                                       NaN   \n",
       "saleYear                                                               1989   \n",
       "saleMonth                                                                 1   \n",
       "saleDay                                                                  31   \n",
       "saleDayofweek                                                             1   \n",
       "saleDayofyear                                                            31   \n",
       "\n",
       "                                               212552  \\\n",
       "SalesID                                       1671174   \n",
       "SalePrice                                       16000   \n",
       "MachineID                                     1327630   \n",
       "ModelID                                          8591   \n",
       "datasource                                        132   \n",
       "auctioneerID                                       99   \n",
       "YearMade                                         1980   \n",
       "MachineHoursCurrentMeter                          NaN   \n",
       "UsageBand                                         NaN   \n",
       "fiModelDesc                                       A62   \n",
       "fiBaseModel                                       A62   \n",
       "fiSecondaryDesc                                   NaN   \n",
       "fiModelSeries                                     NaN   \n",
       "fiModelDescriptor                                 NaN   \n",
       "ProductSize                                       NaN   \n",
       "fiProductClassDesc        Wheel Loader - Unidentified   \n",
       "state                                         Florida   \n",
       "ProductGroup                                       WL   \n",
       "ProductGroupDesc                         Wheel Loader   \n",
       "Drive_System                                      NaN   \n",
       "Enclosure                                       EROPS   \n",
       "Forks                             None or Unspecified   \n",
       "Pad_Type                                          NaN   \n",
       "Ride_Control                      None or Unspecified   \n",
       "Stick                                             NaN   \n",
       "Transmission                                      NaN   \n",
       "Turbocharged                                      NaN   \n",
       "Blade_Extension                                   NaN   \n",
       "Blade_Width                                       NaN   \n",
       "Enclosure_Type                                    NaN   \n",
       "Engine_Horsepower                                 NaN   \n",
       "Hydraulics                                    2 Valve   \n",
       "Pushblock                                         NaN   \n",
       "Ripper                                            NaN   \n",
       "Scarifier                                         NaN   \n",
       "Tip_Control                                       NaN   \n",
       "Tire_Size                         None or Unspecified   \n",
       "Coupler                           None or Unspecified   \n",
       "Coupler_System                                    NaN   \n",
       "Grouser_Tracks                                    NaN   \n",
       "Hydraulics_Flow                                   NaN   \n",
       "Track_Type                                        NaN   \n",
       "Undercarriage_Pad_Width                           NaN   \n",
       "Stick_Length                                      NaN   \n",
       "Thumb                                             NaN   \n",
       "Pattern_Changer                                   NaN   \n",
       "Grouser_Type                                      NaN   \n",
       "Backhoe_Mounting                                  NaN   \n",
       "Blade_Type                                        NaN   \n",
       "Travel_Controls                                   NaN   \n",
       "Differential_Type                            Standard   \n",
       "Steering_Controls                        Conventional   \n",
       "saleYear                                         1989   \n",
       "saleMonth                                           1   \n",
       "saleDay                                            31   \n",
       "saleDayofweek                                       1   \n",
       "saleDayofyear                                      31   \n",
       "\n",
       "                                                                     62755   \n",
       "SalesID                                                             1329056  \n",
       "SalePrice                                                             22000  \n",
       "MachineID                                                           1336053  \n",
       "ModelID                                                                4089  \n",
       "datasource                                                              132  \n",
       "auctioneerID                                                             99  \n",
       "YearMade                                                               1984  \n",
       "MachineHoursCurrentMeter                                                NaN  \n",
       "UsageBand                                                               NaN  \n",
       "fiModelDesc                                                             D3B  \n",
       "fiBaseModel                                                              D3  \n",
       "fiSecondaryDesc                                                           B  \n",
       "fiModelSeries                                                           NaN  \n",
       "fiModelDescriptor                                                       NaN  \n",
       "ProductSize                                                             NaN  \n",
       "fiProductClassDesc        Track Type Tractor, Dozer - 20.0 to 75.0 Horse...  \n",
       "state                                                               Florida  \n",
       "ProductGroup                                                            TTT  \n",
       "ProductGroupDesc                                        Track Type Tractors  \n",
       "Drive_System                                                            NaN  \n",
       "Enclosure                                                             OROPS  \n",
       "Forks                                                                   NaN  \n",
       "Pad_Type                                                                NaN  \n",
       "Ride_Control                                                            NaN  \n",
       "Stick                                                                   NaN  \n",
       "Transmission                                                       Standard  \n",
       "Turbocharged                                                            NaN  \n",
       "Blade_Extension                                                         NaN  \n",
       "Blade_Width                                                             NaN  \n",
       "Enclosure_Type                                                          NaN  \n",
       "Engine_Horsepower                                                       NaN  \n",
       "Hydraulics                                                          2 Valve  \n",
       "Pushblock                                                               NaN  \n",
       "Ripper                                                  None or Unspecified  \n",
       "Scarifier                                                               NaN  \n",
       "Tip_Control                                                             NaN  \n",
       "Tire_Size                                                               NaN  \n",
       "Coupler                                                                 NaN  \n",
       "Coupler_System                                                          NaN  \n",
       "Grouser_Tracks                                                          NaN  \n",
       "Hydraulics_Flow                                                         NaN  \n",
       "Track_Type                                                              NaN  \n",
       "Undercarriage_Pad_Width                                                 NaN  \n",
       "Stick_Length                                                            NaN  \n",
       "Thumb                                                                   NaN  \n",
       "Pattern_Changer                                                         NaN  \n",
       "Grouser_Type                                                            NaN  \n",
       "Backhoe_Mounting                                        None or Unspecified  \n",
       "Blade_Type                                                              PAT  \n",
       "Travel_Controls                                                       Lever  \n",
       "Differential_Type                                                       NaN  \n",
       "Steering_Controls                                                       NaN  \n",
       "saleYear                                                               1989  \n",
       "saleMonth                                                                 1  \n",
       "saleDay                                                                  31  \n",
       "saleDayofweek                                                             1  \n",
       "saleDayofyear                                                            31  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_tmp.head().T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.api.types.is_string_dtype(df_tmp[\"UsageBand\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "UsageBand\n",
      "fiModelDesc\n",
      "fiBaseModel\n",
      "fiSecondaryDesc\n",
      "fiModelSeries\n",
      "fiModelDescriptor\n",
      "ProductSize\n",
      "fiProductClassDesc\n",
      "state\n",
      "ProductGroup\n",
      "ProductGroupDesc\n",
      "Drive_System\n",
      "Enclosure\n",
      "Forks\n",
      "Pad_Type\n",
      "Ride_Control\n",
      "Stick\n",
      "Transmission\n",
      "Turbocharged\n",
      "Blade_Extension\n",
      "Blade_Width\n",
      "Enclosure_Type\n",
      "Engine_Horsepower\n",
      "Hydraulics\n",
      "Pushblock\n",
      "Ripper\n",
      "Scarifier\n",
      "Tip_Control\n",
      "Tire_Size\n",
      "Coupler\n",
      "Coupler_System\n",
      "Grouser_Tracks\n",
      "Hydraulics_Flow\n",
      "Track_Type\n",
      "Undercarriage_Pad_Width\n",
      "Stick_Length\n",
      "Thumb\n",
      "Pattern_Changer\n",
      "Grouser_Type\n",
      "Backhoe_Mounting\n",
      "Blade_Type\n",
      "Travel_Controls\n",
      "Differential_Type\n",
      "Steering_Controls\n"
     ]
    }
   ],
   "source": [
    "# These columns contain strings\n",
    "for label, content in df_tmp.items():\n",
    "    if pd.api.types.is_string_dtype(content):\n",
    "        print(label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is a key: key1\n",
      "This is a value: hello\n",
      "This is a key: key2\n",
      "This is a value: world!\n"
     ]
    }
   ],
   "source": [
    "# If you're wondering what df.items() does, let's use a dictionary as an example\n",
    "random_dict = {\"key1\": \"hello\",\n",
    "               \"key2\": \"world!\"}\n",
    "\n",
    "for key, value in random_dict.items():\n",
    "    print(f\"This is a key: {key}\")\n",
    "    print(f\"This is a value: {value}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This will turn all of the string values into category values\n",
    "for label, content in df_tmp.items():\n",
    "    if pd.api.types.is_string_dtype(content):\n",
    "        df_tmp[label] = content.astype(\"category\").cat.as_ordered()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 412698 entries, 205615 to 409203\n",
      "Data columns (total 57 columns):\n",
      " #   Column                    Non-Null Count   Dtype   \n",
      "---  ------                    --------------   -----   \n",
      " 0   SalesID                   412698 non-null  int64   \n",
      " 1   SalePrice                 412698 non-null  float64 \n",
      " 2   MachineID                 412698 non-null  int64   \n",
      " 3   ModelID                   412698 non-null  int64   \n",
      " 4   datasource                412698 non-null  int64   \n",
      " 5   auctioneerID              392562 non-null  float64 \n",
      " 6   YearMade                  412698 non-null  int64   \n",
      " 7   MachineHoursCurrentMeter  147504 non-null  float64 \n",
      " 8   UsageBand                 73670 non-null   category\n",
      " 9   fiModelDesc               412698 non-null  category\n",
      " 10  fiBaseModel               412698 non-null  category\n",
      " 11  fiSecondaryDesc           271971 non-null  category\n",
      " 12  fiModelSeries             58667 non-null   category\n",
      " 13  fiModelDescriptor         74816 non-null   category\n",
      " 14  ProductSize               196093 non-null  category\n",
      " 15  fiProductClassDesc        412698 non-null  category\n",
      " 16  state                     412698 non-null  category\n",
      " 17  ProductGroup              412698 non-null  category\n",
      " 18  ProductGroupDesc          412698 non-null  category\n",
      " 19  Drive_System              107087 non-null  category\n",
      " 20  Enclosure                 412364 non-null  category\n",
      " 21  Forks                     197715 non-null  category\n",
      " 22  Pad_Type                  81096 non-null   category\n",
      " 23  Ride_Control              152728 non-null  category\n",
      " 24  Stick                     81096 non-null   category\n",
      " 25  Transmission              188007 non-null  category\n",
      " 26  Turbocharged              81096 non-null   category\n",
      " 27  Blade_Extension           25983 non-null   category\n",
      " 28  Blade_Width               25983 non-null   category\n",
      " 29  Enclosure_Type            25983 non-null   category\n",
      " 30  Engine_Horsepower         25983 non-null   category\n",
      " 31  Hydraulics                330133 non-null  category\n",
      " 32  Pushblock                 25983 non-null   category\n",
      " 33  Ripper                    106945 non-null  category\n",
      " 34  Scarifier                 25994 non-null   category\n",
      " 35  Tip_Control               25983 non-null   category\n",
      " 36  Tire_Size                 97638 non-null   category\n",
      " 37  Coupler                   220679 non-null  category\n",
      " 38  Coupler_System            44974 non-null   category\n",
      " 39  Grouser_Tracks            44875 non-null   category\n",
      " 40  Hydraulics_Flow           44875 non-null   category\n",
      " 41  Track_Type                102193 non-null  category\n",
      " 42  Undercarriage_Pad_Width   102916 non-null  category\n",
      " 43  Stick_Length              102261 non-null  category\n",
      " 44  Thumb                     102332 non-null  category\n",
      " 45  Pattern_Changer           102261 non-null  category\n",
      " 46  Grouser_Type              102193 non-null  category\n",
      " 47  Backhoe_Mounting          80712 non-null   category\n",
      " 48  Blade_Type                81875 non-null   category\n",
      " 49  Travel_Controls           81877 non-null   category\n",
      " 50  Differential_Type         71564 non-null   category\n",
      " 51  Steering_Controls         71522 non-null   category\n",
      " 52  saleYear                  412698 non-null  int64   \n",
      " 53  saleMonth                 412698 non-null  int64   \n",
      " 54  saleDay                   412698 non-null  int64   \n",
      " 55  saleDayofweek             412698 non-null  int64   \n",
      " 56  saleDayofyear             412698 non-null  int64   \n",
      "dtypes: category(44), float64(3), int64(10)\n",
      "memory usage: 63.3 MB\n"
     ]
    }
   ],
   "source": [
    "df_tmp.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Alabama', 'Alaska', 'Arizona', 'Arkansas', 'California', 'Colorado',\n",
       "       'Connecticut', 'Delaware', 'Florida', 'Georgia', 'Hawaii', 'Idaho',\n",
       "       'Illinois', 'Indiana', 'Iowa', 'Kansas', 'Kentucky', 'Louisiana',\n",
       "       'Maine', 'Maryland', 'Massachusetts', 'Michigan', 'Minnesota',\n",
       "       'Mississippi', 'Missouri', 'Montana', 'Nebraska', 'Nevada',\n",
       "       'New Hampshire', 'New Jersey', 'New Mexico', 'New York',\n",
       "       'North Carolina', 'North Dakota', 'Ohio', 'Oklahoma', 'Oregon',\n",
       "       'Pennsylvania', 'Puerto Rico', 'Rhode Island', 'South Carolina',\n",
       "       'South Dakota', 'Tennessee', 'Texas', 'Unspecified', 'Utah', 'Vermont',\n",
       "       'Virginia', 'Washington', 'Washington DC', 'West Virginia', 'Wisconsin',\n",
       "       'Wyoming'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_tmp.state.cat.categories"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "205615    43\n",
       "274835     8\n",
       "141296     8\n",
       "212552     8\n",
       "62755      8\n",
       "          ..\n",
       "410879     4\n",
       "412476     4\n",
       "411927     4\n",
       "407124     4\n",
       "409203     4\n",
       "Length: 412698, dtype: int8"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_tmp.state.cat.codes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All of our data is categorical and thus we can now turn the categories into numbers, however it's still missing values..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SalesID                     0.000000\n",
       "SalePrice                   0.000000\n",
       "MachineID                   0.000000\n",
       "ModelID                     0.000000\n",
       "datasource                  0.000000\n",
       "auctioneerID                0.048791\n",
       "YearMade                    0.000000\n",
       "MachineHoursCurrentMeter    0.642586\n",
       "UsageBand                   0.821492\n",
       "fiModelDesc                 0.000000\n",
       "fiBaseModel                 0.000000\n",
       "fiSecondaryDesc             0.340993\n",
       "fiModelSeries               0.857845\n",
       "fiModelDescriptor           0.818715\n",
       "ProductSize                 0.524851\n",
       "fiProductClassDesc          0.000000\n",
       "state                       0.000000\n",
       "ProductGroup                0.000000\n",
       "ProductGroupDesc            0.000000\n",
       "Drive_System                0.740520\n",
       "Enclosure                   0.000809\n",
       "Forks                       0.520921\n",
       "Pad_Type                    0.803498\n",
       "Ride_Control                0.629928\n",
       "Stick                       0.803498\n",
       "Transmission                0.544444\n",
       "Turbocharged                0.803498\n",
       "Blade_Extension             0.937041\n",
       "Blade_Width                 0.937041\n",
       "Enclosure_Type              0.937041\n",
       "Engine_Horsepower           0.937041\n",
       "Hydraulics                  0.200062\n",
       "Pushblock                   0.937041\n",
       "Ripper                      0.740864\n",
       "Scarifier                   0.937014\n",
       "Tip_Control                 0.937041\n",
       "Tire_Size                   0.763415\n",
       "Coupler                     0.465277\n",
       "Coupler_System              0.891024\n",
       "Grouser_Tracks              0.891264\n",
       "Hydraulics_Flow             0.891264\n",
       "Track_Type                  0.752378\n",
       "Undercarriage_Pad_Width     0.750626\n",
       "Stick_Length                0.752213\n",
       "Thumb                       0.752041\n",
       "Pattern_Changer             0.752213\n",
       "Grouser_Type                0.752378\n",
       "Backhoe_Mounting            0.804428\n",
       "Blade_Type                  0.801610\n",
       "Travel_Controls             0.801606\n",
       "Differential_Type           0.826595\n",
       "Steering_Controls           0.826697\n",
       "saleYear                    0.000000\n",
       "saleMonth                   0.000000\n",
       "saleDay                     0.000000\n",
       "saleDayofweek               0.000000\n",
       "saleDayofyear               0.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_tmp.isnull().sum()/len(df_tmp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the format it's in, it's still good to be worked with, let's save it to file and reimport it so we can continue on.\n",
    "\n",
    "### Save Processed Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save preprocessed data\n",
    "df_tmp.to_csv(\"../data/bluebook-for-bulldozers/train_tmp.csv\",\n",
    "              index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>SalesID</th>\n",
       "      <td>1646770</td>\n",
       "      <td>1821514</td>\n",
       "      <td>1505138</td>\n",
       "      <td>1671174</td>\n",
       "      <td>1329056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SalePrice</th>\n",
       "      <td>9500</td>\n",
       "      <td>14000</td>\n",
       "      <td>50000</td>\n",
       "      <td>16000</td>\n",
       "      <td>22000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MachineID</th>\n",
       "      <td>1126363</td>\n",
       "      <td>1194089</td>\n",
       "      <td>1473654</td>\n",
       "      <td>1327630</td>\n",
       "      <td>1336053</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ModelID</th>\n",
       "      <td>8434</td>\n",
       "      <td>10150</td>\n",
       "      <td>4139</td>\n",
       "      <td>8591</td>\n",
       "      <td>4089</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>datasource</th>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>auctioneerID</th>\n",
       "      <td>18</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>YearMade</th>\n",
       "      <td>1974</td>\n",
       "      <td>1980</td>\n",
       "      <td>1978</td>\n",
       "      <td>1980</td>\n",
       "      <td>1984</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MachineHoursCurrentMeter</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UsageBand</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiModelDesc</th>\n",
       "      <td>TD20</td>\n",
       "      <td>A66</td>\n",
       "      <td>D7G</td>\n",
       "      <td>A62</td>\n",
       "      <td>D3B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiBaseModel</th>\n",
       "      <td>TD20</td>\n",
       "      <td>A66</td>\n",
       "      <td>D7</td>\n",
       "      <td>A62</td>\n",
       "      <td>D3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiSecondaryDesc</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>G</td>\n",
       "      <td>NaN</td>\n",
       "      <td>B</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiModelSeries</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiModelDescriptor</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ProductSize</th>\n",
       "      <td>Medium</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Large</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fiProductClassDesc</th>\n",
       "      <td>Track Type Tractor, Dozer - 105.0 to 130.0 Hor...</td>\n",
       "      <td>Wheel Loader - 120.0 to 135.0 Horsepower</td>\n",
       "      <td>Track Type Tractor, Dozer - 190.0 to 260.0 Hor...</td>\n",
       "      <td>Wheel Loader - Unidentified</td>\n",
       "      <td>Track Type Tractor, Dozer - 20.0 to 75.0 Horse...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>state</th>\n",
       "      <td>Texas</td>\n",
       "      <td>Florida</td>\n",
       "      <td>Florida</td>\n",
       "      <td>Florida</td>\n",
       "      <td>Florida</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ProductGroup</th>\n",
       "      <td>TTT</td>\n",
       "      <td>WL</td>\n",
       "      <td>TTT</td>\n",
       "      <td>WL</td>\n",
       "      <td>TTT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ProductGroupDesc</th>\n",
       "      <td>Track Type Tractors</td>\n",
       "      <td>Wheel Loader</td>\n",
       "      <td>Track Type Tractors</td>\n",
       "      <td>Wheel Loader</td>\n",
       "      <td>Track Type Tractors</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Drive_System</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Enclosure</th>\n",
       "      <td>OROPS</td>\n",
       "      <td>OROPS</td>\n",
       "      <td>OROPS</td>\n",
       "      <td>EROPS</td>\n",
       "      <td>OROPS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Forks</th>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pad_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ride_Control</th>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Stick</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Transmission</th>\n",
       "      <td>Direct Drive</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Turbocharged</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blade_Extension</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blade_Width</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Enclosure_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Engine_Horsepower</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hydraulics</th>\n",
       "      <td>2 Valve</td>\n",
       "      <td>2 Valve</td>\n",
       "      <td>2 Valve</td>\n",
       "      <td>2 Valve</td>\n",
       "      <td>2 Valve</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pushblock</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ripper</th>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Scarifier</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tip_Control</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tire_Size</th>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Coupler</th>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Coupler_System</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Grouser_Tracks</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hydraulics_Flow</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Track_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Undercarriage_Pad_Width</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Stick_Length</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Thumb</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pattern_Changer</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Grouser_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Backhoe_Mounting</th>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blade_Type</th>\n",
       "      <td>Straight</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Straight</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PAT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Travel_Controls</th>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Lever</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Differential_Type</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Steering_Controls</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Conventional</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Conventional</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleYear</th>\n",
       "      <td>1989</td>\n",
       "      <td>1989</td>\n",
       "      <td>1989</td>\n",
       "      <td>1989</td>\n",
       "      <td>1989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleMonth</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleDay</th>\n",
       "      <td>17</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleDayofweek</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>saleDayofyear</th>\n",
       "      <td>17</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                          0  \\\n",
       "SalesID                                                             1646770   \n",
       "SalePrice                                                              9500   \n",
       "MachineID                                                           1126363   \n",
       "ModelID                                                                8434   \n",
       "datasource                                                              132   \n",
       "auctioneerID                                                             18   \n",
       "YearMade                                                               1974   \n",
       "MachineHoursCurrentMeter                                                NaN   \n",
       "UsageBand                                                               NaN   \n",
       "fiModelDesc                                                            TD20   \n",
       "fiBaseModel                                                            TD20   \n",
       "fiSecondaryDesc                                                         NaN   \n",
       "fiModelSeries                                                           NaN   \n",
       "fiModelDescriptor                                                       NaN   \n",
       "ProductSize                                                          Medium   \n",
       "fiProductClassDesc        Track Type Tractor, Dozer - 105.0 to 130.0 Hor...   \n",
       "state                                                                 Texas   \n",
       "ProductGroup                                                            TTT   \n",
       "ProductGroupDesc                                        Track Type Tractors   \n",
       "Drive_System                                                            NaN   \n",
       "Enclosure                                                             OROPS   \n",
       "Forks                                                                   NaN   \n",
       "Pad_Type                                                                NaN   \n",
       "Ride_Control                                                            NaN   \n",
       "Stick                                                                   NaN   \n",
       "Transmission                                                   Direct Drive   \n",
       "Turbocharged                                                            NaN   \n",
       "Blade_Extension                                                         NaN   \n",
       "Blade_Width                                                             NaN   \n",
       "Enclosure_Type                                                          NaN   \n",
       "Engine_Horsepower                                                       NaN   \n",
       "Hydraulics                                                          2 Valve   \n",
       "Pushblock                                                               NaN   \n",
       "Ripper                                                  None or Unspecified   \n",
       "Scarifier                                                               NaN   \n",
       "Tip_Control                                                             NaN   \n",
       "Tire_Size                                                               NaN   \n",
       "Coupler                                                                 NaN   \n",
       "Coupler_System                                                          NaN   \n",
       "Grouser_Tracks                                                          NaN   \n",
       "Hydraulics_Flow                                                         NaN   \n",
       "Track_Type                                                              NaN   \n",
       "Undercarriage_Pad_Width                                                 NaN   \n",
       "Stick_Length                                                            NaN   \n",
       "Thumb                                                                   NaN   \n",
       "Pattern_Changer                                                         NaN   \n",
       "Grouser_Type                                                            NaN   \n",
       "Backhoe_Mounting                                        None or Unspecified   \n",
       "Blade_Type                                                         Straight   \n",
       "Travel_Controls                                         None or Unspecified   \n",
       "Differential_Type                                                       NaN   \n",
       "Steering_Controls                                                       NaN   \n",
       "saleYear                                                               1989   \n",
       "saleMonth                                                                 1   \n",
       "saleDay                                                                  17   \n",
       "saleDayofweek                                                             1   \n",
       "saleDayofyear                                                            17   \n",
       "\n",
       "                                                                 1  \\\n",
       "SalesID                                                    1821514   \n",
       "SalePrice                                                    14000   \n",
       "MachineID                                                  1194089   \n",
       "ModelID                                                      10150   \n",
       "datasource                                                     132   \n",
       "auctioneerID                                                    99   \n",
       "YearMade                                                      1980   \n",
       "MachineHoursCurrentMeter                                       NaN   \n",
       "UsageBand                                                      NaN   \n",
       "fiModelDesc                                                    A66   \n",
       "fiBaseModel                                                    A66   \n",
       "fiSecondaryDesc                                                NaN   \n",
       "fiModelSeries                                                  NaN   \n",
       "fiModelDescriptor                                              NaN   \n",
       "ProductSize                                                    NaN   \n",
       "fiProductClassDesc        Wheel Loader - 120.0 to 135.0 Horsepower   \n",
       "state                                                      Florida   \n",
       "ProductGroup                                                    WL   \n",
       "ProductGroupDesc                                      Wheel Loader   \n",
       "Drive_System                                                   NaN   \n",
       "Enclosure                                                    OROPS   \n",
       "Forks                                          None or Unspecified   \n",
       "Pad_Type                                                       NaN   \n",
       "Ride_Control                                   None or Unspecified   \n",
       "Stick                                                          NaN   \n",
       "Transmission                                                   NaN   \n",
       "Turbocharged                                                   NaN   \n",
       "Blade_Extension                                                NaN   \n",
       "Blade_Width                                                    NaN   \n",
       "Enclosure_Type                                                 NaN   \n",
       "Engine_Horsepower                                              NaN   \n",
       "Hydraulics                                                 2 Valve   \n",
       "Pushblock                                                      NaN   \n",
       "Ripper                                                         NaN   \n",
       "Scarifier                                                      NaN   \n",
       "Tip_Control                                                    NaN   \n",
       "Tire_Size                                      None or Unspecified   \n",
       "Coupler                                        None or Unspecified   \n",
       "Coupler_System                                                 NaN   \n",
       "Grouser_Tracks                                                 NaN   \n",
       "Hydraulics_Flow                                                NaN   \n",
       "Track_Type                                                     NaN   \n",
       "Undercarriage_Pad_Width                                        NaN   \n",
       "Stick_Length                                                   NaN   \n",
       "Thumb                                                          NaN   \n",
       "Pattern_Changer                                                NaN   \n",
       "Grouser_Type                                                   NaN   \n",
       "Backhoe_Mounting                                               NaN   \n",
       "Blade_Type                                                     NaN   \n",
       "Travel_Controls                                                NaN   \n",
       "Differential_Type                                         Standard   \n",
       "Steering_Controls                                     Conventional   \n",
       "saleYear                                                      1989   \n",
       "saleMonth                                                        1   \n",
       "saleDay                                                         31   \n",
       "saleDayofweek                                                    1   \n",
       "saleDayofyear                                                   31   \n",
       "\n",
       "                                                                          2  \\\n",
       "SalesID                                                             1505138   \n",
       "SalePrice                                                             50000   \n",
       "MachineID                                                           1473654   \n",
       "ModelID                                                                4139   \n",
       "datasource                                                              132   \n",
       "auctioneerID                                                             99   \n",
       "YearMade                                                               1978   \n",
       "MachineHoursCurrentMeter                                                NaN   \n",
       "UsageBand                                                               NaN   \n",
       "fiModelDesc                                                             D7G   \n",
       "fiBaseModel                                                              D7   \n",
       "fiSecondaryDesc                                                           G   \n",
       "fiModelSeries                                                           NaN   \n",
       "fiModelDescriptor                                                       NaN   \n",
       "ProductSize                                                           Large   \n",
       "fiProductClassDesc        Track Type Tractor, Dozer - 190.0 to 260.0 Hor...   \n",
       "state                                                               Florida   \n",
       "ProductGroup                                                            TTT   \n",
       "ProductGroupDesc                                        Track Type Tractors   \n",
       "Drive_System                                                            NaN   \n",
       "Enclosure                                                             OROPS   \n",
       "Forks                                                                   NaN   \n",
       "Pad_Type                                                                NaN   \n",
       "Ride_Control                                                            NaN   \n",
       "Stick                                                                   NaN   \n",
       "Transmission                                                       Standard   \n",
       "Turbocharged                                                            NaN   \n",
       "Blade_Extension                                                         NaN   \n",
       "Blade_Width                                                             NaN   \n",
       "Enclosure_Type                                                          NaN   \n",
       "Engine_Horsepower                                                       NaN   \n",
       "Hydraulics                                                          2 Valve   \n",
       "Pushblock                                                               NaN   \n",
       "Ripper                                                  None or Unspecified   \n",
       "Scarifier                                                               NaN   \n",
       "Tip_Control                                                             NaN   \n",
       "Tire_Size                                                               NaN   \n",
       "Coupler                                                                 NaN   \n",
       "Coupler_System                                                          NaN   \n",
       "Grouser_Tracks                                                          NaN   \n",
       "Hydraulics_Flow                                                         NaN   \n",
       "Track_Type                                                              NaN   \n",
       "Undercarriage_Pad_Width                                                 NaN   \n",
       "Stick_Length                                                            NaN   \n",
       "Thumb                                                                   NaN   \n",
       "Pattern_Changer                                                         NaN   \n",
       "Grouser_Type                                                            NaN   \n",
       "Backhoe_Mounting                                        None or Unspecified   \n",
       "Blade_Type                                                         Straight   \n",
       "Travel_Controls                                         None or Unspecified   \n",
       "Differential_Type                                                       NaN   \n",
       "Steering_Controls                                                       NaN   \n",
       "saleYear                                                               1989   \n",
       "saleMonth                                                                 1   \n",
       "saleDay                                                                  31   \n",
       "saleDayofweek                                                             1   \n",
       "saleDayofyear                                                            31   \n",
       "\n",
       "                                                    3  \\\n",
       "SalesID                                       1671174   \n",
       "SalePrice                                       16000   \n",
       "MachineID                                     1327630   \n",
       "ModelID                                          8591   \n",
       "datasource                                        132   \n",
       "auctioneerID                                       99   \n",
       "YearMade                                         1980   \n",
       "MachineHoursCurrentMeter                          NaN   \n",
       "UsageBand                                         NaN   \n",
       "fiModelDesc                                       A62   \n",
       "fiBaseModel                                       A62   \n",
       "fiSecondaryDesc                                   NaN   \n",
       "fiModelSeries                                     NaN   \n",
       "fiModelDescriptor                                 NaN   \n",
       "ProductSize                                       NaN   \n",
       "fiProductClassDesc        Wheel Loader - Unidentified   \n",
       "state                                         Florida   \n",
       "ProductGroup                                       WL   \n",
       "ProductGroupDesc                         Wheel Loader   \n",
       "Drive_System                                      NaN   \n",
       "Enclosure                                       EROPS   \n",
       "Forks                             None or Unspecified   \n",
       "Pad_Type                                          NaN   \n",
       "Ride_Control                      None or Unspecified   \n",
       "Stick                                             NaN   \n",
       "Transmission                                      NaN   \n",
       "Turbocharged                                      NaN   \n",
       "Blade_Extension                                   NaN   \n",
       "Blade_Width                                       NaN   \n",
       "Enclosure_Type                                    NaN   \n",
       "Engine_Horsepower                                 NaN   \n",
       "Hydraulics                                    2 Valve   \n",
       "Pushblock                                         NaN   \n",
       "Ripper                                            NaN   \n",
       "Scarifier                                         NaN   \n",
       "Tip_Control                                       NaN   \n",
       "Tire_Size                         None or Unspecified   \n",
       "Coupler                           None or Unspecified   \n",
       "Coupler_System                                    NaN   \n",
       "Grouser_Tracks                                    NaN   \n",
       "Hydraulics_Flow                                   NaN   \n",
       "Track_Type                                        NaN   \n",
       "Undercarriage_Pad_Width                           NaN   \n",
       "Stick_Length                                      NaN   \n",
       "Thumb                                             NaN   \n",
       "Pattern_Changer                                   NaN   \n",
       "Grouser_Type                                      NaN   \n",
       "Backhoe_Mounting                                  NaN   \n",
       "Blade_Type                                        NaN   \n",
       "Travel_Controls                                   NaN   \n",
       "Differential_Type                            Standard   \n",
       "Steering_Controls                        Conventional   \n",
       "saleYear                                         1989   \n",
       "saleMonth                                           1   \n",
       "saleDay                                            31   \n",
       "saleDayofweek                                       1   \n",
       "saleDayofyear                                      31   \n",
       "\n",
       "                                                                          4  \n",
       "SalesID                                                             1329056  \n",
       "SalePrice                                                             22000  \n",
       "MachineID                                                           1336053  \n",
       "ModelID                                                                4089  \n",
       "datasource                                                              132  \n",
       "auctioneerID                                                             99  \n",
       "YearMade                                                               1984  \n",
       "MachineHoursCurrentMeter                                                NaN  \n",
       "UsageBand                                                               NaN  \n",
       "fiModelDesc                                                             D3B  \n",
       "fiBaseModel                                                              D3  \n",
       "fiSecondaryDesc                                                           B  \n",
       "fiModelSeries                                                           NaN  \n",
       "fiModelDescriptor                                                       NaN  \n",
       "ProductSize                                                             NaN  \n",
       "fiProductClassDesc        Track Type Tractor, Dozer - 20.0 to 75.0 Horse...  \n",
       "state                                                               Florida  \n",
       "ProductGroup                                                            TTT  \n",
       "ProductGroupDesc                                        Track Type Tractors  \n",
       "Drive_System                                                            NaN  \n",
       "Enclosure                                                             OROPS  \n",
       "Forks                                                                   NaN  \n",
       "Pad_Type                                                                NaN  \n",
       "Ride_Control                                                            NaN  \n",
       "Stick                                                                   NaN  \n",
       "Transmission                                                       Standard  \n",
       "Turbocharged                                                            NaN  \n",
       "Blade_Extension                                                         NaN  \n",
       "Blade_Width                                                             NaN  \n",
       "Enclosure_Type                                                          NaN  \n",
       "Engine_Horsepower                                                       NaN  \n",
       "Hydraulics                                                          2 Valve  \n",
       "Pushblock                                                               NaN  \n",
       "Ripper                                                  None or Unspecified  \n",
       "Scarifier                                                               NaN  \n",
       "Tip_Control                                                             NaN  \n",
       "Tire_Size                                                               NaN  \n",
       "Coupler                                                                 NaN  \n",
       "Coupler_System                                                          NaN  \n",
       "Grouser_Tracks                                                          NaN  \n",
       "Hydraulics_Flow                                                         NaN  \n",
       "Track_Type                                                              NaN  \n",
       "Undercarriage_Pad_Width                                                 NaN  \n",
       "Stick_Length                                                            NaN  \n",
       "Thumb                                                                   NaN  \n",
       "Pattern_Changer                                                         NaN  \n",
       "Grouser_Type                                                            NaN  \n",
       "Backhoe_Mounting                                        None or Unspecified  \n",
       "Blade_Type                                                              PAT  \n",
       "Travel_Controls                                                       Lever  \n",
       "Differential_Type                                                       NaN  \n",
       "Steering_Controls                                                       NaN  \n",
       "saleYear                                                               1989  \n",
       "saleMonth                                                                 1  \n",
       "saleDay                                                                  31  \n",
       "saleDayofweek                                                             1  \n",
       "saleDayofyear                                                            31  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Import preprocessed data\n",
    "df_tmp = pd.read_csv(\"../data/bluebook-for-bulldozers/train_tmp.csv\",\n",
    "                     low_memory=False)\n",
    "df_tmp.head().T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Excellent, our processed DataFrame has the columns we added to it but it's still missing values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SalesID                          0\n",
       "SalePrice                        0\n",
       "MachineID                        0\n",
       "ModelID                          0\n",
       "datasource                       0\n",
       "auctioneerID                 20136\n",
       "YearMade                         0\n",
       "MachineHoursCurrentMeter    265194\n",
       "UsageBand                   339028\n",
       "fiModelDesc                      0\n",
       "fiBaseModel                      0\n",
       "fiSecondaryDesc             140727\n",
       "fiModelSeries               354031\n",
       "fiModelDescriptor           337882\n",
       "ProductSize                 216605\n",
       "fiProductClassDesc               0\n",
       "state                            0\n",
       "ProductGroup                     0\n",
       "ProductGroupDesc                 0\n",
       "Drive_System                305611\n",
       "Enclosure                      334\n",
       "Forks                       214983\n",
       "Pad_Type                    331602\n",
       "Ride_Control                259970\n",
       "Stick                       331602\n",
       "Transmission                224691\n",
       "Turbocharged                331602\n",
       "Blade_Extension             386715\n",
       "Blade_Width                 386715\n",
       "Enclosure_Type              386715\n",
       "Engine_Horsepower           386715\n",
       "Hydraulics                   82565\n",
       "Pushblock                   386715\n",
       "Ripper                      305753\n",
       "Scarifier                   386704\n",
       "Tip_Control                 386715\n",
       "Tire_Size                   315060\n",
       "Coupler                     192019\n",
       "Coupler_System              367724\n",
       "Grouser_Tracks              367823\n",
       "Hydraulics_Flow             367823\n",
       "Track_Type                  310505\n",
       "Undercarriage_Pad_Width     309782\n",
       "Stick_Length                310437\n",
       "Thumb                       310366\n",
       "Pattern_Changer             310437\n",
       "Grouser_Type                310505\n",
       "Backhoe_Mounting            331986\n",
       "Blade_Type                  330823\n",
       "Travel_Controls             330821\n",
       "Differential_Type           341134\n",
       "Steering_Controls           341176\n",
       "saleYear                         0\n",
       "saleMonth                        0\n",
       "saleDay                          0\n",
       "saleDayofweek                    0\n",
       "saleDayofyear                    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check missing values\n",
    "df_tmp.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fill missing values\n",
    "\n",
    "From our experience with machine learning models. We know two things:\n",
    "1. All of our data has to be numerical\n",
    "2. There can't be any missing values\n",
    "\n",
    "And as we've seen using `df_tmp.isna().sum()` our data still has plenty of missing values.\n",
    "\n",
    "Let's fill them.\n",
    "\n",
    "### Filling numerical values first\n",
    "\n",
    "We're going to fill any column with missing values with the median of that column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SalesID\n",
      "SalePrice\n",
      "MachineID\n",
      "ModelID\n",
      "datasource\n",
      "auctioneerID\n",
      "YearMade\n",
      "MachineHoursCurrentMeter\n",
      "saleYear\n",
      "saleMonth\n",
      "saleDay\n",
      "saleDayofweek\n",
      "saleDayofyear\n"
     ]
    }
   ],
   "source": [
    "for label, content in df_tmp.items():\n",
    "    if pd.api.types.is_numeric_dtype(content):\n",
    "        print(label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "auctioneerID\n",
      "MachineHoursCurrentMeter\n"
     ]
    }
   ],
   "source": [
    "# Check for which numeric columns have null values\n",
    "for label, content in df_tmp.items():\n",
    "    if pd.api.types.is_numeric_dtype(content):\n",
    "        if pd.isnull(content).sum():\n",
    "            print(label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fill numeric rows with the median\n",
    "for label, content in df_tmp.items():\n",
    "    if pd.api.types.is_numeric_dtype(content):\n",
    "        if pd.isnull(content).sum():\n",
    "            # Add a binary column which tells if the data was missing our not\n",
    "            df_tmp[label+\"_is_missing\"] = pd.isnull(content)\n",
    "            # Fill missing numeric values with median since it's more robust than the mean\n",
    "            df_tmp[label] = content.fillna(content.median())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why add a binary column indicating whether the data was missing or not?\n",
    "\n",
    "We can easily fill all of the missing numeric values in our dataset with the median. However, a numeric value may be missing for a reason. In other words, absence of evidence may be evidence of absence. Adding a binary column which indicates whether the value was missing or not helps to retain this information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check if there's any null values\n",
    "for label, content in df_tmp.items():\n",
    "    if pd.api.types.is_numeric_dtype(content):\n",
    "        if pd.isnull(content).sum():\n",
    "            print(label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False    392562\n",
       "True      20136\n",
       "Name: auctioneerID_is_missing, dtype: int64"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check to see how many examples were missing\n",
    "df_tmp.auctioneerID_is_missing.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filling and turning categorical variables to numbers\n",
    "\n",
    "Now we've filled the numeric values, we'll do the same with the categorical values at the same time as turning them into numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "UsageBand\n",
      "fiModelDesc\n",
      "fiBaseModel\n",
      "fiSecondaryDesc\n",
      "fiModelSeries\n",
      "fiModelDescriptor\n",
      "ProductSize\n",
      "fiProductClassDesc\n",
      "state\n",
      "ProductGroup\n",
      "ProductGroupDesc\n",
      "Drive_System\n",
      "Enclosure\n",
      "Forks\n",
      "Pad_Type\n",
      "Ride_Control\n",
      "Stick\n",
      "Transmission\n",
      "Turbocharged\n",
      "Blade_Extension\n",
      "Blade_Width\n",
      "Enclosure_Type\n",
      "Engine_Horsepower\n",
      "Hydraulics\n",
      "Pushblock\n",
      "Ripper\n",
      "Scarifier\n",
      "Tip_Control\n",
      "Tire_Size\n",
      "Coupler\n",
      "Coupler_System\n",
      "Grouser_Tracks\n",
      "Hydraulics_Flow\n",
      "Track_Type\n",
      "Undercarriage_Pad_Width\n",
      "Stick_Length\n",
      "Thumb\n",
      "Pattern_Changer\n",
      "Grouser_Type\n",
      "Backhoe_Mounting\n",
      "Blade_Type\n",
      "Travel_Controls\n",
      "Differential_Type\n",
      "Steering_Controls\n"
     ]
    }
   ],
   "source": [
    "# Check columns which *aren't* numeric\n",
    "for label, content in df_tmp.items():\n",
    "    if not pd.api.types.is_numeric_dtype(content):\n",
    "        print(label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Turn categorical variables into numbers\n",
    "for label, content in df_tmp.items():\n",
    "    # Check columns which *aren't* numeric\n",
    "    if not pd.api.types.is_numeric_dtype(content):\n",
    "        # Add binary column to inidicate whether sample had missing value\n",
    "        df_tmp[label+\"_is_missing\"] = pd.isnull(content)\n",
    "        # We add the +1 because pandas encodes missing categories as -1\n",
    "        df_tmp[label] = pd.Categorical(content).codes+1        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 412698 entries, 0 to 412697\n",
      "Columns: 103 entries, SalesID to Steering_Controls_is_missing\n",
      "dtypes: bool(46), float64(3), int16(4), int64(10), int8(40)\n",
      "memory usage: 77.9 MB\n"
     ]
    }
   ],
   "source": [
    "df_tmp.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SalesID                         0\n",
       "SalePrice                       0\n",
       "MachineID                       0\n",
       "ModelID                         0\n",
       "datasource                      0\n",
       "                               ..\n",
       "Backhoe_Mounting_is_missing     0\n",
       "Blade_Type_is_missing           0\n",
       "Travel_Controls_is_missing      0\n",
       "Differential_Type_is_missing    0\n",
       "Steering_Controls_is_missing    0\n",
       "Length: 103, dtype: int64"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_tmp.isna().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>SalesID</th>\n",
       "      <td>1646770</td>\n",
       "      <td>1821514</td>\n",
       "      <td>1505138</td>\n",
       "      <td>1671174</td>\n",
       "      <td>1329056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SalePrice</th>\n",
       "      <td>9500</td>\n",
       "      <td>14000</td>\n",
       "      <td>50000</td>\n",
       "      <td>16000</td>\n",
       "      <td>22000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MachineID</th>\n",
       "      <td>1126363</td>\n",
       "      <td>1194089</td>\n",
       "      <td>1473654</td>\n",
       "      <td>1327630</td>\n",
       "      <td>1336053</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ModelID</th>\n",
       "      <td>8434</td>\n",
       "      <td>10150</td>\n",
       "      <td>4139</td>\n",
       "      <td>8591</td>\n",
       "      <td>4089</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>datasource</th>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "      <td>132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Backhoe_Mounting_is_missing</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Blade_Type_is_missing</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Travel_Controls_is_missing</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Differential_Type_is_missing</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Steering_Controls_is_missing</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>103 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    0        1        2        3        4\n",
       "SalesID                       1646770  1821514  1505138  1671174  1329056\n",
       "SalePrice                        9500    14000    50000    16000    22000\n",
       "MachineID                     1126363  1194089  1473654  1327630  1336053\n",
       "ModelID                          8434    10150     4139     8591     4089\n",
       "datasource                        132      132      132      132      132\n",
       "...                               ...      ...      ...      ...      ...\n",
       "Backhoe_Mounting_is_missing     False     True    False     True    False\n",
       "Blade_Type_is_missing           False     True    False     True    False\n",
       "Travel_Controls_is_missing      False     True    False     True    False\n",
       "Differential_Type_is_missing     True    False     True    False     True\n",
       "Steering_Controls_is_missing     True    False     True    False     True\n",
       "\n",
       "[103 rows x 5 columns]"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_tmp.head().T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now all of our data is numeric and there are no missing values, we should be able to build a machine learning model!\n",
    "\n",
    "Let's reinstantiate our trusty [RandomForestRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html).\n",
    "\n",
    "This will take a few minutes which is too long for interacting with it. So what we'll do is create a subset of rows to work with."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 25min 34s, sys: 10.4 s, total: 25min 44s\n",
      "Wall time: 1min 49s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor(bootstrap=True, ccp_alpha=0.0, criterion='mse',\n",
       "                      max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
       "                      max_samples=None, min_impurity_decrease=0.0,\n",
       "                      min_impurity_split=None, min_samples_leaf=1,\n",
       "                      min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
       "                      n_estimators=100, n_jobs=-1, oob_score=False,\n",
       "                      random_state=None, verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "# Instantiate model\n",
    "model = RandomForestRegressor(n_jobs=-1)\n",
    "\n",
    "# Fit the model\n",
    "model.fit(df_tmp.drop(\"SalePrice\", axis=1), df_tmp.SalePrice)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.987621368497284"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Score the model\n",
    "model.score(df_tmp.drop(\"SalePrice\", axis=1), df_tmp.SalePrice)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question:** Why is this metric not reliable?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Splitting data into train/valid sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SalesID</th>\n",
       "      <th>SalePrice</th>\n",
       "      <th>MachineID</th>\n",
       "      <th>ModelID</th>\n",
       "      <th>datasource</th>\n",
       "      <th>auctioneerID</th>\n",
       "      <th>YearMade</th>\n",
       "      <th>MachineHoursCurrentMeter</th>\n",
       "      <th>UsageBand</th>\n",
       "      <th>fiModelDesc</th>\n",
       "      <th>...</th>\n",
       "      <th>Undercarriage_Pad_Width_is_missing</th>\n",
       "      <th>Stick_Length_is_missing</th>\n",
       "      <th>Thumb_is_missing</th>\n",
       "      <th>Pattern_Changer_is_missing</th>\n",
       "      <th>Grouser_Type_is_missing</th>\n",
       "      <th>Backhoe_Mounting_is_missing</th>\n",
       "      <th>Blade_Type_is_missing</th>\n",
       "      <th>Travel_Controls_is_missing</th>\n",
       "      <th>Differential_Type_is_missing</th>\n",
       "      <th>Steering_Controls_is_missing</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1646770</td>\n",
       "      <td>9500.0</td>\n",
       "      <td>1126363</td>\n",
       "      <td>8434</td>\n",
       "      <td>132</td>\n",
       "      <td>18.0</td>\n",
       "      <td>1974</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4593</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1821514</td>\n",
       "      <td>14000.0</td>\n",
       "      <td>1194089</td>\n",
       "      <td>10150</td>\n",
       "      <td>132</td>\n",
       "      <td>99.0</td>\n",
       "      <td>1980</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1820</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1505138</td>\n",
       "      <td>50000.0</td>\n",
       "      <td>1473654</td>\n",
       "      <td>4139</td>\n",
       "      <td>132</td>\n",
       "      <td>99.0</td>\n",
       "      <td>1978</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2348</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1671174</td>\n",
       "      <td>16000.0</td>\n",
       "      <td>1327630</td>\n",
       "      <td>8591</td>\n",
       "      <td>132</td>\n",
       "      <td>99.0</td>\n",
       "      <td>1980</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1819</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1329056</td>\n",
       "      <td>22000.0</td>\n",
       "      <td>1336053</td>\n",
       "      <td>4089</td>\n",
       "      <td>132</td>\n",
       "      <td>99.0</td>\n",
       "      <td>1984</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2119</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 103 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   SalesID  SalePrice  MachineID  ModelID  datasource  auctioneerID  YearMade  \\\n",
       "0  1646770     9500.0    1126363     8434         132          18.0      1974   \n",
       "1  1821514    14000.0    1194089    10150         132          99.0      1980   \n",
       "2  1505138    50000.0    1473654     4139         132          99.0      1978   \n",
       "3  1671174    16000.0    1327630     8591         132          99.0      1980   \n",
       "4  1329056    22000.0    1336053     4089         132          99.0      1984   \n",
       "\n",
       "   MachineHoursCurrentMeter  UsageBand  fiModelDesc  ...  \\\n",
       "0                       0.0          0         4593  ...   \n",
       "1                       0.0          0         1820  ...   \n",
       "2                       0.0          0         2348  ...   \n",
       "3                       0.0          0         1819  ...   \n",
       "4                       0.0          0         2119  ...   \n",
       "\n",
       "   Undercarriage_Pad_Width_is_missing  Stick_Length_is_missing  \\\n",
       "0                                True                     True   \n",
       "1                                True                     True   \n",
       "2                                True                     True   \n",
       "3                                True                     True   \n",
       "4                                True                     True   \n",
       "\n",
       "   Thumb_is_missing  Pattern_Changer_is_missing  Grouser_Type_is_missing  \\\n",
       "0              True                        True                     True   \n",
       "1              True                        True                     True   \n",
       "2              True                        True                     True   \n",
       "3              True                        True                     True   \n",
       "4              True                        True                     True   \n",
       "\n",
       "   Backhoe_Mounting_is_missing  Blade_Type_is_missing  \\\n",
       "0                        False                  False   \n",
       "1                         True                   True   \n",
       "2                        False                  False   \n",
       "3                         True                   True   \n",
       "4                        False                  False   \n",
       "\n",
       "   Travel_Controls_is_missing  Differential_Type_is_missing  \\\n",
       "0                       False                          True   \n",
       "1                        True                         False   \n",
       "2                       False                          True   \n",
       "3                        True                         False   \n",
       "4                       False                          True   \n",
       "\n",
       "   Steering_Controls_is_missing  \n",
       "0                          True  \n",
       "1                         False  \n",
       "2                          True  \n",
       "3                         False  \n",
       "4                          True  \n",
       "\n",
       "[5 rows x 103 columns]"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_tmp.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "According to the [Kaggle data page](https://www.kaggle.com/c/bluebook-for-bulldozers/data), the validation set and test set are split according to dates.\n",
    "\n",
    "This makes sense since we're working on a time series problem.\n",
    "\n",
    "E.g. using past events to try and predict future events.\n",
    "\n",
    "Knowing this, randomly splitting our data into train and test sets using something like `train_test_split()` wouldn't work.\n",
    "\n",
    "Instead, we split our data into training, validation and test sets using the date each sample occured.\n",
    "\n",
    "In our case:\n",
    "* Training = all samples up until 2011\n",
    "* Valid = all samples form January 1, 2012 - April 30, 2012\n",
    "* Test = all samples from May 1, 2012 - November 2012\n",
    "\n",
    "For more on making good training, validation and test sets, check out the post [How (and why) to create a good validation set](https://www.fast.ai/2017/11/13/validation-sets/) by Rachel Thomas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2009    43849\n",
       "2008    39767\n",
       "2011    35197\n",
       "2010    33390\n",
       "2007    32208\n",
       "2006    21685\n",
       "2005    20463\n",
       "2004    19879\n",
       "2001    17594\n",
       "2000    17415\n",
       "2002    17246\n",
       "2003    15254\n",
       "1998    13046\n",
       "1999    12793\n",
       "2012    11573\n",
       "1997     9785\n",
       "1996     8829\n",
       "1995     8530\n",
       "1994     7929\n",
       "1993     6303\n",
       "1992     5519\n",
       "1991     5109\n",
       "1989     4806\n",
       "1990     4529\n",
       "Name: saleYear, dtype: int64"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_tmp.saleYear.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(11573, 401125)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Split data into training and validation\n",
    "df_val = df_tmp[df_tmp.saleYear == 2012]\n",
    "df_train = df_tmp[df_tmp.saleYear != 2012]\n",
    "\n",
    "len(df_val), len(df_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((401125, 102), (401125,), (11573, 102), (11573,))"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Split data into X & y\n",
    "X_train, y_train = df_train.drop(\"SalePrice\", axis=1), df_train.SalePrice\n",
    "X_valid, y_valid = df_val.drop(\"SalePrice\", axis=1), df_val.SalePrice\n",
    "\n",
    "X_train.shape, y_train.shape, X_valid.shape, y_valid.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Building an evaluation function\n",
    "\n",
    "According to Kaggle for the Bluebook for Bulldozers competition, [the evaluation function](https://www.kaggle.com/c/bluebook-for-bulldozers/overview/evaluation) they use is root mean squared log error (RMSLE).\n",
    "\n",
    "**RMSLE** = generally you don't care as much if you're off by $10 as much as you'd care if you were off by 10%, you care more about ratios rather than differences. **MAE** (mean absolute error) is more about exact differences.\n",
    "\n",
    "It's important to understand the evaluation metric you're going for.\n",
    "\n",
    "Since Scikit-Learn doesn't have a function built-in for RMSLE, we'll create our own.\n",
    "\n",
    "We can do this by taking the square root of Scikit-Learn's [mean_squared_log_error](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.mean_squared_log_error.html#sklearn.metrics.mean_squared_log_error) (MSLE). MSLE is the same as taking the log of mean squared error (MSE).\n",
    "\n",
    "We'll also calculate the MAE and R^2 for fun."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create evaluation function (the competition uses Root Mean Square Log Error)\n",
    "from sklearn.metrics import mean_squared_log_error, mean_absolute_error\n",
    "\n",
    "def rmsle(y_test, y_preds):\n",
    "    return np.sqrt(mean_squared_log_error(y_test, y_preds))\n",
    "\n",
    "# Create function to evaluate our model\n",
    "def show_scores(model):\n",
    "    train_preds = model.predict(X_train)\n",
    "    val_preds = model.predict(X_valid)\n",
    "    scores = {\"Training MAE\": mean_absolute_error(y_train, train_preds),\n",
    "              \"Valid MAE\": mean_absolute_error(y_valid, val_preds),\n",
    "              \"Training RMSLE\": rmsle(y_train, train_preds),\n",
    "              \"Valid RMSLE\": rmsle(y_valid, val_preds),\n",
    "              \"Training R^2\": model.score(X_train, y_train),\n",
    "              \"Valid R^2\": model.score(X_valid, y_valid)}\n",
    "    return scores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Testing our model on a subset (to tune the hyperparameters)\n",
    "\n",
    "Retraing an entire model would take far too long to continuing experimenting as fast as we want to.\n",
    "\n",
    "So what we'll do is take a sample of the training set and tune the hyperparameters on that before training a larger model.\n",
    "\n",
    "If you're experiments are taking longer than 10-seconds (give or take how long you have to wait), you should be trying to speed things up. You can speed things up by sampling less data or using a faster computer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This takes too long...\n",
    "\n",
    "# %%time\n",
    "# # Retrain a model on training data\n",
    "# model.fit(X_train, y_train)\n",
    "# show_scores(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "401125"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Depending on your computer (mine is a MacBook Pro), making calculations on ~400,000 rows may take a while...\n",
    "\n",
    "Let's alter the number of samples each `n_estimator` in the [`RandomForestRegressor`](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html) see's using the `max_samples` parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Change max samples in RandomForestRegressor\n",
    "model = RandomForestRegressor(n_jobs=-1,\n",
    "                              max_samples=10000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Setting `max_samples` to 10000 means every `n_estimator` (default 100) in our `RandomForestRegressor` will only see 10000 random samples from our DataFrame instead of the entire 400,000.\n",
    "\n",
    "In other words, we'll be looking at 40x less samples which means we'll get faster computation speeds but we should expect our results to worsen (simple the model has less samples to learn patterns from)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 49.2 s, sys: 897 ms, total: 50.1 s\n",
      "Wall time: 5.07 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor(bootstrap=True, ccp_alpha=0.0, criterion='mse',\n",
       "                      max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
       "                      max_samples=10000, min_impurity_decrease=0.0,\n",
       "                      min_impurity_split=None, min_samples_leaf=1,\n",
       "                      min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
       "                      n_estimators=100, n_jobs=-1, oob_score=False,\n",
       "                      random_state=None, verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "# Cutting down the max number of samples each tree can see improves training time\n",
    "model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Training MAE': 5570.442946637581,\n",
       " 'Valid MAE': 7152.537319623261,\n",
       " 'Training RMSLE': 0.2579466072796448,\n",
       " 'Valid RMSLE': 0.2934498553410761,\n",
       " 'Training R^2': 0.8601748180348622,\n",
       " 'Valid R^2': 0.8325057428307028}"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show_scores(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Beautiful, that took far less time than the model with all the data.\n",
    "\n",
    "With this, let's try tune some hyperparameters."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hyperparameter tuning with RandomizedSearchCV\n",
    "\n",
    "You can increase `n_iter` to try more combinations of hyperparameters but in our case, we'll try 20 and see where it gets us.\n",
    "\n",
    "Remember, we're trying to reduce the amount of time it takes between experiments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 20 candidates, totalling 100 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n",
      "[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:  6.4min finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 6min 28s, sys: 7.53 s, total: 6min 36s\n",
      "Wall time: 6min 36s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "RandomizedSearchCV(cv=5, error_score=nan,\n",
       "                   estimator=RandomForestRegressor(bootstrap=True,\n",
       "                                                   ccp_alpha=0.0,\n",
       "                                                   criterion='mse',\n",
       "                                                   max_depth=None,\n",
       "                                                   max_features='auto',\n",
       "                                                   max_leaf_nodes=None,\n",
       "                                                   max_samples=None,\n",
       "                                                   min_impurity_decrease=0.0,\n",
       "                                                   min_impurity_split=None,\n",
       "                                                   min_samples_leaf=1,\n",
       "                                                   min_samples_split=2,\n",
       "                                                   min_weight_fraction_leaf=0.0,\n",
       "                                                   n_estimators=100,\n",
       "                                                   n_jobs=None, oob_score=Fals...\n",
       "                   param_distributions={'max_depth': [None, 3, 5, 10],\n",
       "                                        'max_features': [0.5, 1, 'sqrt',\n",
       "                                                         'auto'],\n",
       "                                        'max_samples': [10000],\n",
       "                                        'min_samples_leaf': array([ 1,  3,  5,  7,  9, 11, 13, 15, 17, 19]),\n",
       "                                        'min_samples_split': array([ 2,  4,  6,  8, 10, 12, 14, 16, 18]),\n",
       "                                        'n_estimators': array([10, 20, 30, 40, 50, 60, 70, 80, 90])},\n",
       "                   pre_dispatch='2*n_jobs', random_state=None, refit=True,\n",
       "                   return_train_score=False, scoring=None, verbose=True)"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "from sklearn.model_selection import RandomizedSearchCV\n",
    "\n",
    "# Different RandomForestClassifier hyperparameters\n",
    "rf_grid = {\"n_estimators\": np.arange(10, 100, 10),\n",
    "           \"max_depth\": [None, 3, 5, 10],\n",
    "           \"min_samples_split\": np.arange(2, 20, 2),\n",
    "           \"min_samples_leaf\": np.arange(1, 20, 2),\n",
    "           \"max_features\": [0.5, 1, \"sqrt\", \"auto\"],\n",
    "           \"max_samples\": [10000]}\n",
    "\n",
    "rs_model = RandomizedSearchCV(RandomForestRegressor(),\n",
    "                              param_distributions=rf_grid,\n",
    "                              n_iter=20,\n",
    "                              cv=5,\n",
    "                              verbose=True)\n",
    "\n",
    "rs_model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'n_estimators': 90,\n",
       " 'min_samples_split': 4,\n",
       " 'min_samples_leaf': 1,\n",
       " 'max_samples': 10000,\n",
       " 'max_features': 0.5,\n",
       " 'max_depth': None}"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the best parameters from the RandomizedSearch \n",
    "rs_model.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Training MAE': 5739.3439073046875,\n",
       " 'Valid MAE': 7192.1275124731565,\n",
       " 'Training RMSLE': 0.2645974926906147,\n",
       " 'Valid RMSLE': 0.29647910600319566,\n",
       " 'Training R^2': 0.8542568601075173,\n",
       " 'Valid R^2': 0.8366783805921302}"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Evaluate the RandomizedSearch model\n",
    "show_scores(rs_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train a model with the best parameters\n",
    "\n",
    "In a model I prepared earlier, I tried 100 different combinations of hyperparameters (setting `n_iter` to 100 in `RandomizedSearchCV`) and found the best results came from the ones you see below.\n",
    "\n",
    "**Note:** This kind of search on my computer (`n_iter` = 100) took ~2-hours. So it's kind of a set and come back later experiment.\n",
    "\n",
    "We'll instantiate a new model with these discovered hyperparameters and reset the `max_samples` back to its original value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 11min 45s, sys: 5.53 s, total: 11min 50s\n",
      "Wall time: 54 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor(bootstrap=True, ccp_alpha=0.0, criterion='mse',\n",
       "                      max_depth=None, max_features=0.5, max_leaf_nodes=None,\n",
       "                      max_samples=None, min_impurity_decrease=0.0,\n",
       "                      min_impurity_split=None, min_samples_leaf=1,\n",
       "                      min_samples_split=14, min_weight_fraction_leaf=0.0,\n",
       "                      n_estimators=90, n_jobs=-1, oob_score=False,\n",
       "                      random_state=None, verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "# Most ideal hyperparameters\n",
    "ideal_model = RandomForestRegressor(n_estimators=90,\n",
    "                                    min_samples_leaf=1,\n",
    "                                    min_samples_split=14,\n",
    "                                    max_features=0.5,\n",
    "                                    n_jobs=-1,\n",
    "                                    max_samples=None)\n",
    "ideal_model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Training MAE': 2927.555314630477,\n",
       " 'Valid MAE': 5925.139309561446,\n",
       " 'Training RMSLE': 0.1433128908407748,\n",
       " 'Valid RMSLE': 0.2453295099556409,\n",
       " 'Training R^2': 0.9596919518640112,\n",
       " 'Valid R^2': 0.883030690228852}"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show_scores(ideal_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With these new hyperparameters as well as using all the samples, we can see an improvement to our models performance.\n",
    "\n",
    "You can make a faster model by altering some of the hyperparameters. Particularly by lowering `n_estimators` since each increase in `n_estimators` is basically building another small model.\n",
    "\n",
    "However, lowering of `n_estimators` or altering of other hyperparameters may lead to poorer results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4min 59s, sys: 2.76 s, total: 5min 2s\n",
      "Wall time: 25.3 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "RandomForestRegressor(bootstrap=True, ccp_alpha=0.0, criterion='mse',\n",
       "                      max_depth=None, max_features=0.5, max_leaf_nodes=None,\n",
       "                      max_samples=None, min_impurity_decrease=0.0,\n",
       "                      min_impurity_split=None, min_samples_leaf=3,\n",
       "                      min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
       "                      n_estimators=40, n_jobs=-1, oob_score=False,\n",
       "                      random_state=None, verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "# Faster model\n",
    "fast_model = RandomForestRegressor(n_estimators=40,\n",
    "                                   min_samples_leaf=3,\n",
    "                                   max_features=0.5,\n",
    "                                   n_jobs=-1)\n",
    "fast_model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Training MAE': 2538.086916839285,\n",
       " 'Valid MAE': 5921.566785123334,\n",
       " 'Training RMSLE': 0.12938511430324895,\n",
       " 'Valid RMSLE': 0.24299588986662657,\n",
       " 'Training R^2': 0.9672433268997953,\n",
       " 'Valid R^2': 0.8812702169749176}"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show_scores(fast_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Make predictions on test data\n",
    "\n",
    "Now we've got a trained model, it's time to make predictions on the test data.\n",
    "\n",
    "Remember what we've done.\n",
    "\n",
    "Our model is trained on data prior to 2011. However, the test data is from May 1 2012 to November 2012.\n",
    "\n",
    "So what we're doing is trying to use the patterns our model has learned in the training data to predict the sale price of a Bulldozer with characteristics it's never seen before but are assumed to be similar to that of those in the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SalesID</th>\n",
       "      <th>MachineID</th>\n",
       "      <th>ModelID</th>\n",
       "      <th>datasource</th>\n",
       "      <th>auctioneerID</th>\n",
       "      <th>YearMade</th>\n",
       "      <th>MachineHoursCurrentMeter</th>\n",
       "      <th>UsageBand</th>\n",
       "      <th>saledate</th>\n",
       "      <th>fiModelDesc</th>\n",
       "      <th>...</th>\n",
       "      <th>Undercarriage_Pad_Width</th>\n",
       "      <th>Stick_Length</th>\n",
       "      <th>Thumb</th>\n",
       "      <th>Pattern_Changer</th>\n",
       "      <th>Grouser_Type</th>\n",
       "      <th>Backhoe_Mounting</th>\n",
       "      <th>Blade_Type</th>\n",
       "      <th>Travel_Controls</th>\n",
       "      <th>Differential_Type</th>\n",
       "      <th>Steering_Controls</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1227829</td>\n",
       "      <td>1006309</td>\n",
       "      <td>3168</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>1999</td>\n",
       "      <td>3688.0</td>\n",
       "      <td>Low</td>\n",
       "      <td>2012-05-03</td>\n",
       "      <td>580G</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1227844</td>\n",
       "      <td>1022817</td>\n",
       "      <td>7271</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>1000</td>\n",
       "      <td>28555.0</td>\n",
       "      <td>High</td>\n",
       "      <td>2012-05-10</td>\n",
       "      <td>936</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Standard</td>\n",
       "      <td>Conventional</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1227847</td>\n",
       "      <td>1031560</td>\n",
       "      <td>22805</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>2004</td>\n",
       "      <td>6038.0</td>\n",
       "      <td>Medium</td>\n",
       "      <td>2012-05-10</td>\n",
       "      <td>EC210BLC</td>\n",
       "      <td>...</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>9' 6\"</td>\n",
       "      <td>Manual</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>Double</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1227848</td>\n",
       "      <td>56204</td>\n",
       "      <td>1269</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>2006</td>\n",
       "      <td>8940.0</td>\n",
       "      <td>High</td>\n",
       "      <td>2012-05-10</td>\n",
       "      <td>330CL</td>\n",
       "      <td>...</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>Manual</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Triple</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1227863</td>\n",
       "      <td>1053887</td>\n",
       "      <td>22312</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>2005</td>\n",
       "      <td>2286.0</td>\n",
       "      <td>Low</td>\n",
       "      <td>2012-05-10</td>\n",
       "      <td>650K</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>PAT</td>\n",
       "      <td>None or Unspecified</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 52 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   SalesID  MachineID  ModelID  datasource  auctioneerID  YearMade  \\\n",
       "0  1227829    1006309     3168         121             3      1999   \n",
       "1  1227844    1022817     7271         121             3      1000   \n",
       "2  1227847    1031560    22805         121             3      2004   \n",
       "3  1227848      56204     1269         121             3      2006   \n",
       "4  1227863    1053887    22312         121             3      2005   \n",
       "\n",
       "   MachineHoursCurrentMeter UsageBand   saledate fiModelDesc  ...  \\\n",
       "0                    3688.0       Low 2012-05-03        580G  ...   \n",
       "1                   28555.0      High 2012-05-10         936  ...   \n",
       "2                    6038.0    Medium 2012-05-10    EC210BLC  ...   \n",
       "3                    8940.0      High 2012-05-10       330CL  ...   \n",
       "4                    2286.0       Low 2012-05-10        650K  ...   \n",
       "\n",
       "  Undercarriage_Pad_Width         Stick_Length   Thumb      Pattern_Changer  \\\n",
       "0                     NaN                  NaN     NaN                  NaN   \n",
       "1                     NaN                  NaN     NaN                  NaN   \n",
       "2     None or Unspecified                9' 6\"  Manual  None or Unspecified   \n",
       "3     None or Unspecified  None or Unspecified  Manual                  Yes   \n",
       "4                     NaN                  NaN     NaN                  NaN   \n",
       "\n",
       "  Grouser_Type     Backhoe_Mounting Blade_Type      Travel_Controls  \\\n",
       "0          NaN                  NaN        NaN                  NaN   \n",
       "1          NaN                  NaN        NaN                  NaN   \n",
       "2       Double                  NaN        NaN                  NaN   \n",
       "3       Triple                  NaN        NaN                  NaN   \n",
       "4          NaN  None or Unspecified        PAT  None or Unspecified   \n",
       "\n",
       "  Differential_Type Steering_Controls  \n",
       "0               NaN               NaN  \n",
       "1          Standard      Conventional  \n",
       "2               NaN               NaN  \n",
       "3               NaN               NaN  \n",
       "4               NaN               NaN  \n",
       "\n",
       "[5 rows x 52 columns]"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test = pd.read_csv(\"../data/bluebook-for-bulldozers/Test.csv\",\n",
    "                      parse_dates=[\"saledate\"])\n",
    "df_test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "could not convert string to float: 'Low'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-65-a8c751ad5b68>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Let's see how the model goes predicting on the test data\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpredict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdf_test\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.7/site-packages/sklearn/ensemble/_forest.py\u001b[0m in \u001b[0;36mpredict\u001b[0;34m(self, X)\u001b[0m\n\u001b[1;32m    764\u001b[0m         \u001b[0mcheck_is_fitted\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    765\u001b[0m         \u001b[0;31m# Check data\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 766\u001b[0;31m         \u001b[0mX\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_validate_X_predict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    767\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    768\u001b[0m         \u001b[0;31m# Assign chunk of trees to jobs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.7/site-packages/sklearn/ensemble/_forest.py\u001b[0m in \u001b[0;36m_validate_X_predict\u001b[0;34m(self, X)\u001b[0m\n\u001b[1;32m    410\u001b[0m         \u001b[0mcheck_is_fitted\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    411\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 412\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mestimators_\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_validate_X_predict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcheck_input\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    413\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    414\u001b[0m     \u001b[0;34m@\u001b[0m\u001b[0mproperty\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.7/site-packages/sklearn/tree/_classes.py\u001b[0m in \u001b[0;36m_validate_X_predict\u001b[0;34m(self, X, check_input)\u001b[0m\n\u001b[1;32m    378\u001b[0m         \u001b[0;34m\"\"\"Validate X whenever one tries to predict, apply, predict_proba\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    379\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mcheck_input\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 380\u001b[0;31m             \u001b[0mX\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcheck_array\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mDTYPE\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maccept_sparse\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"csr\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    381\u001b[0m             if issparse(X) and (X.indices.dtype != np.intc or\n\u001b[1;32m    382\u001b[0m                                 X.indptr.dtype != np.intc):\n",
      "\u001b[0;32m~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.7/site-packages/sklearn/utils/validation.py\u001b[0m in \u001b[0;36mcheck_array\u001b[0;34m(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)\u001b[0m\n\u001b[1;32m    529\u001b[0m                     \u001b[0marray\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marray\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mastype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdtype\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcasting\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"unsafe\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcopy\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    530\u001b[0m                 \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 531\u001b[0;31m                     \u001b[0marray\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0masarray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marray\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0morder\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0morder\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdtype\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    532\u001b[0m             \u001b[0;32mexcept\u001b[0m \u001b[0mComplexWarning\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    533\u001b[0m                 raise ValueError(\"Complex data not supported\\n\"\n",
      "\u001b[0;32m~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.7/site-packages/numpy/core/_asarray.py\u001b[0m in \u001b[0;36masarray\u001b[0;34m(a, dtype, order)\u001b[0m\n\u001b[1;32m     83\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     84\u001b[0m     \"\"\"\n\u001b[0;32m---> 85\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0marray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcopy\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0morder\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0morder\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     86\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     87\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: could not convert string to float: 'Low'"
     ]
    }
   ],
   "source": [
    "# Let's see how the model goes predicting on the test data\n",
    "model.predict(df_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ahhh... the test data isn't in the same format of our other data, so we have to fix it. Let's create a function to preprocess our data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing the data\n",
    "\n",
    "Our model has been trained on data formatted in the same way as the training data.\n",
    "\n",
    "This means in order to make predictions on the test data, we need to take the same steps we used to preprocess the training data to preprocess the test data.\n",
    "\n",
    "Remember: Whatever you do to the training data, you have to do to the test data.\n",
    "\n",
    "Let's create a function for doing so (by copying the preprocessing steps we used above)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "def preprocess_data(df):\n",
    "    # Add datetime parameters for saledate\n",
    "    df[\"saleYear\"] = df.saledate.dt.year\n",
    "    df[\"saleMonth\"] = df.saledate.dt.month\n",
    "    df[\"saleDay\"] = df.saledate.dt.day\n",
    "    df[\"saleDayofweek\"] = df.saledate.dt.dayofweek\n",
    "    df[\"saleDayofyear\"] = df.saledate.dt.dayofyear\n",
    "\n",
    "    # Drop original saledate\n",
    "    df.drop(\"saledate\", axis=1, inplace=True)\n",
    "    \n",
    "    # Fill numeric rows with the median\n",
    "    for label, content in df.items():\n",
    "        if pd.api.types.is_numeric_dtype(content):\n",
    "            if pd.isnull(content).sum():\n",
    "                df[label+\"_is_missing\"] = pd.isnull(content)\n",
    "                df[label] = content.fillna(content.median())\n",
    "                \n",
    "        # Turn categorical variables into numbers\n",
    "        if not pd.api.types.is_numeric_dtype(content):\n",
    "            df[label+\"_is_missing\"] = pd.isnull(content)\n",
    "            # We add the +1 because pandas encodes missing categories as -1\n",
    "            df[label] = pd.Categorical(content).codes+1        \n",
    "    \n",
    "    return df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Question:** Where would this function break?\n",
    "\n",
    "**Hint:** What if the test data had different missing values to the training data?\n",
    "\n",
    "Now we've got a function for preprocessing data, let's preprocess the test dataset into the same format as our training dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SalesID</th>\n",
       "      <th>MachineID</th>\n",
       "      <th>ModelID</th>\n",
       "      <th>datasource</th>\n",
       "      <th>auctioneerID</th>\n",
       "      <th>YearMade</th>\n",
       "      <th>MachineHoursCurrentMeter</th>\n",
       "      <th>UsageBand</th>\n",
       "      <th>fiModelDesc</th>\n",
       "      <th>fiBaseModel</th>\n",
       "      <th>...</th>\n",
       "      <th>Undercarriage_Pad_Width_is_missing</th>\n",
       "      <th>Stick_Length_is_missing</th>\n",
       "      <th>Thumb_is_missing</th>\n",
       "      <th>Pattern_Changer_is_missing</th>\n",
       "      <th>Grouser_Type_is_missing</th>\n",
       "      <th>Backhoe_Mounting_is_missing</th>\n",
       "      <th>Blade_Type_is_missing</th>\n",
       "      <th>Travel_Controls_is_missing</th>\n",
       "      <th>Differential_Type_is_missing</th>\n",
       "      <th>Steering_Controls_is_missing</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1227829</td>\n",
       "      <td>1006309</td>\n",
       "      <td>3168</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>1999</td>\n",
       "      <td>3688.0</td>\n",
       "      <td>2</td>\n",
       "      <td>499</td>\n",
       "      <td>180</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1227844</td>\n",
       "      <td>1022817</td>\n",
       "      <td>7271</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>1000</td>\n",
       "      <td>28555.0</td>\n",
       "      <td>1</td>\n",
       "      <td>831</td>\n",
       "      <td>292</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1227847</td>\n",
       "      <td>1031560</td>\n",
       "      <td>22805</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>2004</td>\n",
       "      <td>6038.0</td>\n",
       "      <td>3</td>\n",
       "      <td>1177</td>\n",
       "      <td>404</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1227848</td>\n",
       "      <td>56204</td>\n",
       "      <td>1269</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>2006</td>\n",
       "      <td>8940.0</td>\n",
       "      <td>1</td>\n",
       "      <td>287</td>\n",
       "      <td>113</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1227863</td>\n",
       "      <td>1053887</td>\n",
       "      <td>22312</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>2005</td>\n",
       "      <td>2286.0</td>\n",
       "      <td>2</td>\n",
       "      <td>566</td>\n",
       "      <td>196</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 101 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   SalesID  MachineID  ModelID  datasource  auctioneerID  YearMade  \\\n",
       "0  1227829    1006309     3168         121             3      1999   \n",
       "1  1227844    1022817     7271         121             3      1000   \n",
       "2  1227847    1031560    22805         121             3      2004   \n",
       "3  1227848      56204     1269         121             3      2006   \n",
       "4  1227863    1053887    22312         121             3      2005   \n",
       "\n",
       "   MachineHoursCurrentMeter  UsageBand  fiModelDesc  fiBaseModel  ...  \\\n",
       "0                    3688.0          2          499          180  ...   \n",
       "1                   28555.0          1          831          292  ...   \n",
       "2                    6038.0          3         1177          404  ...   \n",
       "3                    8940.0          1          287          113  ...   \n",
       "4                    2286.0          2          566          196  ...   \n",
       "\n",
       "   Undercarriage_Pad_Width_is_missing  Stick_Length_is_missing  \\\n",
       "0                                True                     True   \n",
       "1                                True                     True   \n",
       "2                               False                    False   \n",
       "3                               False                    False   \n",
       "4                                True                     True   \n",
       "\n",
       "   Thumb_is_missing  Pattern_Changer_is_missing  Grouser_Type_is_missing  \\\n",
       "0              True                        True                     True   \n",
       "1              True                        True                     True   \n",
       "2             False                       False                    False   \n",
       "3             False                       False                    False   \n",
       "4              True                        True                     True   \n",
       "\n",
       "   Backhoe_Mounting_is_missing  Blade_Type_is_missing  \\\n",
       "0                         True                   True   \n",
       "1                         True                   True   \n",
       "2                         True                   True   \n",
       "3                         True                   True   \n",
       "4                        False                  False   \n",
       "\n",
       "   Travel_Controls_is_missing  Differential_Type_is_missing  \\\n",
       "0                        True                          True   \n",
       "1                        True                         False   \n",
       "2                        True                          True   \n",
       "3                        True                          True   \n",
       "4                       False                          True   \n",
       "\n",
       "   Steering_Controls_is_missing  \n",
       "0                          True  \n",
       "1                         False  \n",
       "2                          True  \n",
       "3                          True  \n",
       "4                          True  \n",
       "\n",
       "[5 rows x 101 columns]"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_test = preprocess_data(df_test)\n",
    "df_test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SalesID</th>\n",
       "      <th>MachineID</th>\n",
       "      <th>ModelID</th>\n",
       "      <th>datasource</th>\n",
       "      <th>auctioneerID</th>\n",
       "      <th>YearMade</th>\n",
       "      <th>MachineHoursCurrentMeter</th>\n",
       "      <th>UsageBand</th>\n",
       "      <th>fiModelDesc</th>\n",
       "      <th>fiBaseModel</th>\n",
       "      <th>...</th>\n",
       "      <th>Undercarriage_Pad_Width_is_missing</th>\n",
       "      <th>Stick_Length_is_missing</th>\n",
       "      <th>Thumb_is_missing</th>\n",
       "      <th>Pattern_Changer_is_missing</th>\n",
       "      <th>Grouser_Type_is_missing</th>\n",
       "      <th>Backhoe_Mounting_is_missing</th>\n",
       "      <th>Blade_Type_is_missing</th>\n",
       "      <th>Travel_Controls_is_missing</th>\n",
       "      <th>Differential_Type_is_missing</th>\n",
       "      <th>Steering_Controls_is_missing</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1646770</td>\n",
       "      <td>1126363</td>\n",
       "      <td>8434</td>\n",
       "      <td>132</td>\n",
       "      <td>18.0</td>\n",
       "      <td>1974</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4593</td>\n",
       "      <td>1744</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1821514</td>\n",
       "      <td>1194089</td>\n",
       "      <td>10150</td>\n",
       "      <td>132</td>\n",
       "      <td>99.0</td>\n",
       "      <td>1980</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1820</td>\n",
       "      <td>559</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1505138</td>\n",
       "      <td>1473654</td>\n",
       "      <td>4139</td>\n",
       "      <td>132</td>\n",
       "      <td>99.0</td>\n",
       "      <td>1978</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2348</td>\n",
       "      <td>713</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1671174</td>\n",
       "      <td>1327630</td>\n",
       "      <td>8591</td>\n",
       "      <td>132</td>\n",
       "      <td>99.0</td>\n",
       "      <td>1980</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1819</td>\n",
       "      <td>558</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1329056</td>\n",
       "      <td>1336053</td>\n",
       "      <td>4089</td>\n",
       "      <td>132</td>\n",
       "      <td>99.0</td>\n",
       "      <td>1984</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2119</td>\n",
       "      <td>683</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 102 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   SalesID  MachineID  ModelID  datasource  auctioneerID  YearMade  \\\n",
       "0  1646770    1126363     8434         132          18.0      1974   \n",
       "1  1821514    1194089    10150         132          99.0      1980   \n",
       "2  1505138    1473654     4139         132          99.0      1978   \n",
       "3  1671174    1327630     8591         132          99.0      1980   \n",
       "4  1329056    1336053     4089         132          99.0      1984   \n",
       "\n",
       "   MachineHoursCurrentMeter  UsageBand  fiModelDesc  fiBaseModel  ...  \\\n",
       "0                       0.0          0         4593         1744  ...   \n",
       "1                       0.0          0         1820          559  ...   \n",
       "2                       0.0          0         2348          713  ...   \n",
       "3                       0.0          0         1819          558  ...   \n",
       "4                       0.0          0         2119          683  ...   \n",
       "\n",
       "   Undercarriage_Pad_Width_is_missing  Stick_Length_is_missing  \\\n",
       "0                                True                     True   \n",
       "1                                True                     True   \n",
       "2                                True                     True   \n",
       "3                                True                     True   \n",
       "4                                True                     True   \n",
       "\n",
       "   Thumb_is_missing  Pattern_Changer_is_missing  Grouser_Type_is_missing  \\\n",
       "0              True                        True                     True   \n",
       "1              True                        True                     True   \n",
       "2              True                        True                     True   \n",
       "3              True                        True                     True   \n",
       "4              True                        True                     True   \n",
       "\n",
       "   Backhoe_Mounting_is_missing  Blade_Type_is_missing  \\\n",
       "0                        False                  False   \n",
       "1                         True                   True   \n",
       "2                        False                  False   \n",
       "3                         True                   True   \n",
       "4                        False                  False   \n",
       "\n",
       "   Travel_Controls_is_missing  Differential_Type_is_missing  \\\n",
       "0                       False                          True   \n",
       "1                        True                         False   \n",
       "2                       False                          True   \n",
       "3                        True                         False   \n",
       "4                       False                          True   \n",
       "\n",
       "   Steering_Controls_is_missing  \n",
       "0                          True  \n",
       "1                         False  \n",
       "2                          True  \n",
       "3                         False  \n",
       "4                          True  \n",
       "\n",
       "[5 rows x 102 columns]"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Number of features of the model must match the input. Model n_features is 102 and input n_features is 101 ",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-69-2d4f5b7d1aca>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Make predictions on the test dataset using the best model\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mtest_preds\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mideal_model\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpredict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdf_test\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.7/site-packages/sklearn/ensemble/_forest.py\u001b[0m in \u001b[0;36mpredict\u001b[0;34m(self, X)\u001b[0m\n\u001b[1;32m    764\u001b[0m         \u001b[0mcheck_is_fitted\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    765\u001b[0m         \u001b[0;31m# Check data\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 766\u001b[0;31m         \u001b[0mX\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_validate_X_predict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    767\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    768\u001b[0m         \u001b[0;31m# Assign chunk of trees to jobs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.7/site-packages/sklearn/ensemble/_forest.py\u001b[0m in \u001b[0;36m_validate_X_predict\u001b[0;34m(self, X)\u001b[0m\n\u001b[1;32m    410\u001b[0m         \u001b[0mcheck_is_fitted\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    411\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 412\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mestimators_\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_validate_X_predict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcheck_input\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    413\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    414\u001b[0m     \u001b[0;34m@\u001b[0m\u001b[0mproperty\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.7/site-packages/sklearn/tree/_classes.py\u001b[0m in \u001b[0;36m_validate_X_predict\u001b[0;34m(self, X, check_input)\u001b[0m\n\u001b[1;32m    389\u001b[0m                              \u001b[0;34m\"match the input. Model n_features is %s and \"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    390\u001b[0m                              \u001b[0;34m\"input n_features is %s \"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 391\u001b[0;31m                              % (self.n_features_, n_features))\n\u001b[0m\u001b[1;32m    392\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    393\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mX\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: Number of features of the model must match the input. Model n_features is 102 and input n_features is 101 "
     ]
    }
   ],
   "source": [
    "# Make predictions on the test dataset using the best model\n",
    "test_preds = ideal_model.predict(df_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've found an error and it's because our test dataset (after preprocessing) has 101 columns where as, our training dataset (`X_train`) has 102 columns (after preprocessing).\n",
    "\n",
    "Let's find the difference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'auctioneerID_is_missing'}"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can find how the columns differ using sets\n",
    "set(X_train.columns) - set(df_test.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, it's because the test dataset wasn't missing any `auctioneerID` fields.\n",
    "\n",
    "To fix it, we'll add a column to the test dataset called `auctioneerID_is_missing` and fill it with `False`, since none of the `auctioneerID` fields are missing in the test dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SalesID</th>\n",
       "      <th>MachineID</th>\n",
       "      <th>ModelID</th>\n",
       "      <th>datasource</th>\n",
       "      <th>auctioneerID</th>\n",
       "      <th>YearMade</th>\n",
       "      <th>MachineHoursCurrentMeter</th>\n",
       "      <th>UsageBand</th>\n",
       "      <th>fiModelDesc</th>\n",
       "      <th>fiBaseModel</th>\n",
       "      <th>...</th>\n",
       "      <th>Stick_Length_is_missing</th>\n",
       "      <th>Thumb_is_missing</th>\n",
       "      <th>Pattern_Changer_is_missing</th>\n",
       "      <th>Grouser_Type_is_missing</th>\n",
       "      <th>Backhoe_Mounting_is_missing</th>\n",
       "      <th>Blade_Type_is_missing</th>\n",
       "      <th>Travel_Controls_is_missing</th>\n",
       "      <th>Differential_Type_is_missing</th>\n",
       "      <th>Steering_Controls_is_missing</th>\n",
       "      <th>auctioneerID_is_missing</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1227829</td>\n",
       "      <td>1006309</td>\n",
       "      <td>3168</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>1999</td>\n",
       "      <td>3688.0</td>\n",
       "      <td>2</td>\n",
       "      <td>499</td>\n",
       "      <td>180</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1227844</td>\n",
       "      <td>1022817</td>\n",
       "      <td>7271</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>1000</td>\n",
       "      <td>28555.0</td>\n",
       "      <td>1</td>\n",
       "      <td>831</td>\n",
       "      <td>292</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1227847</td>\n",
       "      <td>1031560</td>\n",
       "      <td>22805</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>2004</td>\n",
       "      <td>6038.0</td>\n",
       "      <td>3</td>\n",
       "      <td>1177</td>\n",
       "      <td>404</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1227848</td>\n",
       "      <td>56204</td>\n",
       "      <td>1269</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>2006</td>\n",
       "      <td>8940.0</td>\n",
       "      <td>1</td>\n",
       "      <td>287</td>\n",
       "      <td>113</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1227863</td>\n",
       "      <td>1053887</td>\n",
       "      <td>22312</td>\n",
       "      <td>121</td>\n",
       "      <td>3</td>\n",
       "      <td>2005</td>\n",
       "      <td>2286.0</td>\n",
       "      <td>2</td>\n",
       "      <td>566</td>\n",
       "      <td>196</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 102 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   SalesID  MachineID  ModelID  datasource  auctioneerID  YearMade  \\\n",
       "0  1227829    1006309     3168         121             3      1999   \n",
       "1  1227844    1022817     7271         121             3      1000   \n",
       "2  1227847    1031560    22805         121             3      2004   \n",
       "3  1227848      56204     1269         121             3      2006   \n",
       "4  1227863    1053887    22312         121             3      2005   \n",
       "\n",
       "   MachineHoursCurrentMeter  UsageBand  fiModelDesc  fiBaseModel  ...  \\\n",
       "0                    3688.0          2          499          180  ...   \n",
       "1                   28555.0          1          831          292  ...   \n",
       "2                    6038.0          3         1177          404  ...   \n",
       "3                    8940.0          1          287          113  ...   \n",
       "4                    2286.0          2          566          196  ...   \n",
       "\n",
       "   Stick_Length_is_missing  Thumb_is_missing  Pattern_Changer_is_missing  \\\n",
       "0                     True              True                        True   \n",
       "1                     True              True                        True   \n",
       "2                    False             False                       False   \n",
       "3                    False             False                       False   \n",
       "4                     True              True                        True   \n",
       "\n",
       "   Grouser_Type_is_missing  Backhoe_Mounting_is_missing  \\\n",
       "0                     True                         True   \n",
       "1                     True                         True   \n",
       "2                    False                         True   \n",
       "3                    False                         True   \n",
       "4                     True                        False   \n",
       "\n",
       "   Blade_Type_is_missing  Travel_Controls_is_missing  \\\n",
       "0                   True                        True   \n",
       "1                   True                        True   \n",
       "2                   True                        True   \n",
       "3                   True                        True   \n",
       "4                  False                       False   \n",
       "\n",
       "   Differential_Type_is_missing  Steering_Controls_is_missing  \\\n",
       "0                          True                          True   \n",
       "1                         False                         False   \n",
       "2                          True                          True   \n",
       "3                          True                          True   \n",
       "4                          True                          True   \n",
       "\n",
       "   auctioneerID_is_missing  \n",
       "0                    False  \n",
       "1                    False  \n",
       "2                    False  \n",
       "3                    False  \n",
       "4                    False  \n",
       "\n",
       "[5 rows x 102 columns]"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Match test dataset columns to training dataset\n",
    "df_test[\"auctioneerID_is_missing\"] = False\n",
    "df_test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now the test dataset matches the training dataset, we should be able to make predictions on it using our trained model. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make predictions on the test dataset using the best model\n",
    "test_preds = ideal_model.predict(df_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When looking at the [Kaggle submission requirements](https://www.kaggle.com/c/bluebook-for-bulldozers/overview/evaluation), we see that if we wanted to make a submission, the data is required to be in a certain format. Namely, a DataFrame containing the `SalesID` and the predicted `SalePrice` of the bulldozer.\n",
    "\n",
    "Let's make it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SalesID</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1227829</td>\n",
       "      <td>20256.936017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1227844</td>\n",
       "      <td>18324.603777</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1227847</td>\n",
       "      <td>49532.610965</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1227848</td>\n",
       "      <td>60273.620071</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1227863</td>\n",
       "      <td>48296.569897</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12452</th>\n",
       "      <td>6643171</td>\n",
       "      <td>43937.338316</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12453</th>\n",
       "      <td>6643173</td>\n",
       "      <td>16042.221329</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12454</th>\n",
       "      <td>6643184</td>\n",
       "      <td>16800.163465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12455</th>\n",
       "      <td>6643186</td>\n",
       "      <td>21127.577610</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12456</th>\n",
       "      <td>6643196</td>\n",
       "      <td>30649.491898</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>12457 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       SalesID     SalePrice\n",
       "0      1227829  20256.936017\n",
       "1      1227844  18324.603777\n",
       "2      1227847  49532.610965\n",
       "3      1227848  60273.620071\n",
       "4      1227863  48296.569897\n",
       "...        ...           ...\n",
       "12452  6643171  43937.338316\n",
       "12453  6643173  16042.221329\n",
       "12454  6643184  16800.163465\n",
       "12455  6643186  21127.577610\n",
       "12456  6643196  30649.491898\n",
       "\n",
       "[12457 rows x 2 columns]"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create DataFrame compatible with Kaggle submission requirements\n",
    "df_preds = pd.DataFrame()\n",
    "df_preds[\"SalesID\"] = df_test[\"SalesID\"]\n",
    "df_preds[\"SalePrice\"] = test_preds\n",
    "df_preds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Export to csv...\n",
    "#df_preds.to_csv(\"../data/bluebook-for-bulldozers/predictions.csv\",\n",
    "#                index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Importance\n",
    "\n",
    "Since we've built a model which is able to make predictions. The people you share these predictions with (or yourself) might be curious of what parts of the data led to these predictions.\n",
    "\n",
    "This is where **feature importance** comes in. Feature importance seeks to figure out which different attributes of the data were most important when it comes to predicting the **target variable**.\n",
    "\n",
    "In our case, after our model learned the patterns in the data, which bulldozer sale attributes were most important for predicting its overall sale price?\n",
    "\n",
    "Beware: the default feature importances for random forests can lead to non-ideal results.\n",
    "\n",
    "To find which features were most important of a machine learning model, a good idea is to search something like \"\\[MODEL NAME\\] feature importance\".\n",
    "\n",
    "Doing this for our `RandomForestRegressor` leads us to find the `feature_importances_` attribute.\n",
    "\n",
    "Let's check it out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([3.24060988e-02, 1.97050082e-02, 4.26297938e-02, 1.76409520e-03,\n",
       "       3.34160134e-03, 1.97189725e-01, 3.16784963e-03, 9.57954039e-04,\n",
       "       4.56424316e-02, 4.80405531e-02, 6.49578425e-02, 4.93754858e-03,\n",
       "       2.26399032e-02, 1.53444481e-01, 4.57065333e-02, 5.98403092e-03,\n",
       "       2.56308361e-03, 4.01381774e-03, 3.73720264e-03, 7.25000437e-02,\n",
       "       3.85679909e-04, 7.46340803e-05, 8.81091295e-04, 1.63507568e-04,\n",
       "       9.41071463e-04, 4.22299681e-05, 2.79399795e-04, 9.50383154e-03,\n",
       "       3.13751864e-04, 1.87662838e-03, 5.44079956e-03, 1.69253188e-03,\n",
       "       3.73017212e-03, 8.06102548e-04, 5.16191428e-04, 6.66009391e-03,\n",
       "       1.10644550e-03, 1.40975949e-02, 6.49509580e-05, 2.09174492e-03,\n",
       "       6.39440957e-04, 1.02247996e-03, 2.48824717e-03, 5.54912269e-04,\n",
       "       4.93625386e-04, 3.43711353e-04, 2.69791452e-04, 1.63544295e-03,\n",
       "       8.41405245e-04, 2.45289575e-04, 2.50133059e-04, 7.36682703e-02,\n",
       "       3.78398821e-03, 5.67653994e-03, 2.88845154e-03, 9.92082048e-03,\n",
       "       2.60251024e-04, 1.47095100e-03, 3.40282660e-04, 0.00000000e+00,\n",
       "       0.00000000e+00, 2.08359428e-03, 1.32896488e-03, 6.02185904e-03,\n",
       "       1.83797099e-02, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,\n",
       "       0.00000000e+00, 5.85576081e-05, 4.56351176e-06, 2.95827383e-04,\n",
       "       4.82402483e-06, 1.45307118e-04, 3.76983361e-06, 2.65323131e-04,\n",
       "       2.74289339e-05, 1.15171522e-03, 3.18815689e-03, 7.11585423e-05,\n",
       "       9.90535735e-04, 2.02651887e-03, 7.76576663e-04, 1.56064761e-03,\n",
       "       7.47381895e-04, 2.37677480e-03, 6.29380414e-03, 2.90634512e-04,\n",
       "       1.28293535e-02, 2.48053769e-03, 2.15264222e-03, 1.80378249e-04,\n",
       "       1.33180032e-04, 7.25519411e-05, 1.10587348e-04, 5.52825934e-05,\n",
       "       3.84221205e-05, 4.54627546e-04, 2.23739898e-04, 9.96524931e-05,\n",
       "       1.72560429e-04, 1.10763404e-04])"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find feature importance of our best model\n",
    "ideal_model.feature_importances_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install Seaborn package in current environment (if you don't have it)\n",
    "# import sys\n",
    "# !conda install --yes --prefix {sys.prefix} seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "# Helper function for plotting feature importance\n",
    "def plot_features(columns, importances, n=20):\n",
    "    df = (pd.DataFrame({\"features\": columns,\n",
    "                        \"feature_importance\": importances})\n",
    "          .sort_values(\"feature_importance\", ascending=False)\n",
    "          .reset_index(drop=True))\n",
    "    \n",
    "    sns.barplot(x=\"feature_importance\",\n",
    "                y=\"features\",\n",
    "                data=df[:n],\n",
    "                orient=\"h\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAADrCAYAAABXYUzjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAHZUlEQVR4nO3dP4yXdx0H8OdnoDVw/I6DOyhqIrGQuBoZHZzUWrWhJXZQB+O/kDp0curQoZOTg01jNXEwDjW0pGpFnRwcIa42UBMHpcDBccdBpJA8Dm5N4/Vz9/097x88r9d4ueP3nt759Onzzm/S930HwPA+kg4AMFYKGCBEAQOEKGCAEAUMEKKAAUJ2VX55eXm5P3r06IyiADycLly4sNr3/cr7f14q4E/snXbnvvN8u1QAD4CV09/c0d9PJpN/ftDPPYIACFHAACEKGCBkywKeTCbfn0wm5yeTyfnrmxtDZAIYhS0LuO/7V/u+P9H3/YmDC9MhMgGMgkcQACEKGCBEAQOElIYYu1YO7PiFZAD+xwUMEFK6gO9du9y9+8pLs8oCBD12+oV0hNFxAQOEKGCAEAUMEFKcIt8eIhPAKBSnyHuHyAQwCh5BAIQoYIAQBQwQUhpi7F454mVtgEZcwAAhpQv4P1cvdX9/+alZZSHo08+9mY4Ao+MCBghRwAAhChggpDRFXtt8b4hMAKNQmiIvLTwyRCaAUfAIAiBEAQOElN4D/uihY94XBWjEBQwQooABQkqPIG6tXuz+8vMnZ5WFgM9/7610BBgtFzBAiAIGCFHAACGlKfL6LVNkgFZKU+TFfabIAK14BAEQooABQkrvAe9bPu69UYBGXMAAIaULeG31Ynfml1+aVRYaOPXtP6YjAB+SCxggRAEDhChggBAFDBBSmiJv+FZkgGZKU+Spb0UGaMYjCIAQBQwQUhpiLC0f96I/QCMuYICQ0gV87frF7me/+uKssjyUfvCtP6UjAHPKBQwQooABQhQwQIgCBggpTZE3fSsyQDOlKfKCb0UGaMYjCIAQBQwQUhpirBw8blgA0IgLGCCkdAH/e+1i9+JvTJG38uLX/VcCsDUXMECIAgYIUcAAIQoYIKQ0Rb6zYYoM0EppirxnaooM0IpHEAAhChggpDTE+NjScSMDgEZcwAAhpQv44s13uifefGZWWR4a5556PR0BeAC4gAFCFDBAiAIGCCkt4d7buDtEJoBRKC3hHpk+OkQmgFHwCAIgRAEDhChggJDSEOP4/seNDAAacQEDhBSnyJe7L599aVZZHkh/OPlCOgLwgHIBA4QoYIAQBQwQUpwi3x4iE8AoFKfIe4fIBDAKHkEAhChggBAFDBBSnCIfMTwAaMQFDBBSnCJf655845VZZRncW0+fTkcARswFDBCigAFCFDBASG2KvL45RCaAUahNkRcXhsgEMAoeQQCEKGCAkOISbsW7swCNuIABQhQwQEjpEcSltRvdV878elZZyn5/6hvpCADb5gIGCFHAACEKGCCk+K3IG0NkAhiF4rciT4fIBDAKHkEAhChggJDSe8DHlg549xagERcwQIgCBggpTpHXu6+d+d2ssmzpt6e+GvtsgNZcwAAhChggRAEDhBSnyOtDZAIYheIUeXGITACj4BEEQIgCBggpTpEXvYsL0IgLGCCkdAG/s7bZnXz9r7PK8oHOPvO5QT8PYCguYIAQBQwQooABQhQwQEhpinx34+YQmQBGoTRFfnS6f4hMAKPgEQRAiAIGCCkNMR5fWjCMAGjEBQwQUrqA/3HzbvfsG5eah3jt6WPN/02AeecCBghRwAAhChggRAEDhNSmyOs3hsgEMAq1KfLigSEyAYyCRxAAIQoYIKQ0xPjU/keNJgAacQEDhJQu4Ks373Uvn73S7MOfO3m42b8F8KBxAQOEKGCAEAUMEKKAAUJKU+TNDVNkgFZKU+SFqSkyQCseQQCEKGCAkNIQ49D+3cYTAI24gAFCShfw+tr97txrq9v+sCeeXd723wI8bFzAACEKGCBEAQOElJZwGxvXh8gEMAqlJdx0enCITACj4BEEQIgCBghRwAAhpSHG4tIuYwqARlzAACGlC/jO6v3ub7+4uq0P+sx3D23r7wAeVi5ggBAFDBCigAFCSlPktVumyACtlKbIS/tMkQFa8QgCIEQBA4QoYICQ0hBjz/IugwqARlzAACGlC/jeu/e6yz/+15a/d+RHH992IICxcAEDhChggBAFDBBSmiJfv22KDNBKaYp8cK8pMkArHkEAhChggJDSe8C7H9vtHV+ARlzAACEKGCCkNkW+cqe78pMLW/7e4ec/u+1AAGPhAgYIUcAAIQoYIKQ0Rb5xe22ITACjUJoiH9i7NEQmgFHwCAIgRAEDhNSmyIf3eMcXoBEXMECIAgYIKT2CuH91o7v60z//39859MMv7CgQwFi4gAFCFDBAiAIGCKl9K/Lm+hCZAEah9q3IC4tDZAIYBY8gAEIUMEBI6T3gXYem3vMFaGTS9/2H/+XJ5FbXdW/PLs6OLXddt5oOsYV5zyjfzs17xnnP13Xzn7Ga75N936+8/4elC7jrurf7vj9R/JvBTCaT8/Ocr+vmP6N8OzfvGec9X9fNf8ZW+TwDBghRwAAh1QJ+dSYp2pn3fF03/xnl27l5zzjv+bpu/jM2yVf6n3AAtOMRBECIAgYIUcAAIQoYIEQBA4T8F8ytAl4vjisSAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_features(X_train.columns, ideal_model.feature_importances_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(ideal_model.feature_importances_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "216605"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.ProductSize.isna().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Medium            64342\n",
       "Large / Medium    51297\n",
       "Small             27057\n",
       "Mini              25721\n",
       "Large             21396\n",
       "Compact            6280\n",
       "Name: ProductSize, dtype: int64"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.ProductSize.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "None or Unspecified    77111\n",
       "Yes                     3985\n",
       "Name: Turbocharged, dtype: int64"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.Turbocharged.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "None or Unspecified    85074\n",
       "Manual                  9678\n",
       "Hydraulic               7580\n",
       "Name: Thumb, dtype: int64"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.Thumb.value_counts()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}