Solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Improving a model with Grid Search\n",
    "\n",
    "In this mini-lab, we'll fit a decision tree model to some sample data. This initial model will overfit heavily. Then we'll use Grid Search to find better parameters for this model, to reduce the overfitting.\n",
    "\n",
    "First, some imports."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Reading and plotting the data\n",
    "Now, a function that will help us read the csv file, and plot the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztnX2QHGd54H/P2lp9tOZibC+xWXtjVtok2NT5sFUGzJIzMSRmF3C0MVknSjA5pVQkUJfN3pW0WtcRgu2yo6tQ2UhJiLXxxcopZC8gwAkDWOBQ3rmLnUgqGWMMSN4jZd26YmMS7+z6A4ye+2N6pNlRz0zPTH/386vq2pnp3n6feWfmfbqfT1FVDMMwjPzRE7cAhmEYRjyYAjAMw8gppgAMwzByiikAwzCMnGIKwDAMI6eYAjAMw8gppgAMwzByiikAwzCMnNK1AhCRy0Xk70XkSRF5QkR+2+MYEZE/EpGTIvJ1Ebmm23ENwzCM7jg/gHO8CvwXVT0mIgXgqIgcVtVv1hzzbmDI3d4M/Kn7tykXX3yxXnHFFQGIaBiGkQ+OHj36PVXt83Ns1wpAVZ8BnnEfl0XkSaAfqFUANwMHtFJ34hERuUBELnX/tyFXXHEFR44c6VZEwzCM3CAi/+z32EB9ACJyBfAm4NG6Xf3A0zXPT7mvGYZhGDERmAIQkY3AZ4AJVV2q3+3xL55V6ERkh4gcEZEjzz33XFDiGYZhGHUEogBEZA2Vxf+gqh7yOOQUcHnN88uARa9zqeq9qrpFVbf09fkyYxmGYRgdEEQUkAB/Djypqp9ocNgDwAfcaKC3AC+0sv8bhmEY4RJEFNDbgF8DHheR4+5r08AAgKp+EigCI8BJ4EXg1wMY1zAMw+iCIKKASnjb+GuPUeDD3Y5lGIZhBIdlAhuGYeQUUwCGYRg5xRSAYRhGTjEFYBiGkVNMARiGYeSUIMJADSMSyuUyc3NznDixwNDQIOPj4xQKhbjFMozUYncARioolUr0929iYqLInj0OExNF+vs3USqV4hbNMFKL3QEYiadcLjMyMka5fBB4FwArKwCHGRkZY3FxgY0bN8YpomGkErsDMBLP3Nwcp08PU138z/IuTp8eZm5uLg6xDCP1mAIwEs+JEwusrFzruW9l5RpOnlyIWCLDyAZmAsoYWXSUDg0N4jhF1+yzGsc5xubNo9ELZRgZQCplepLJli1b1DqC+adUKjEyMsbp08OsrFyL4xylp6dEsXiI4eHhuMXrmHK5TH//plU+gAqHKRS2mQ/AMGoQkaOqusXPsXYHkBGy7CgtFAoUi4dqlNs1OM6xM8otre/LMOLGfAAZIeuO0uHhYRYXF5iZGWVq6iVmZkZZXFxI9Z2NYcSN3QFkhDw4Sjdu3Mj27dvjFsMwMoPdAWSEiqP0qOe+iqN0MGKJDMNIOqYAMsL4+Dg9PSXgcN2ew/T0lBgfH49DLMMwEoyZgDKCOUoNw2iXQBSAiNwHvAd4VlXf6LH/BuDzwP91Xzqkqh8PYmzjLFVH6dzcHCdPLrB58yjj4wds8TcMw5Og7gD+AtgHHGhyzLyqvieg8YwGmKPUMAy/BOIDUNWHge8HcS7DMAwjGqJ0Ar9VRB4TkS+KyFURjmsYhmF4EJUT+BjwE6q6LCIjwOeAIa8DRWQHsANgYGAgIvEMwzDyRyR3AKq6pKrL7uMisEZELm5w7L2qukVVt/T19UUhnmEYRi6JRAGIyCUiIu7j69xxn49ibMMwDMOboMJAPwXcAFwsIqeA3wXWAKjqJ4FbgN8UkVeBl4BbNcllSA3DMHJAIApAVX+5xf59VMJEDSOVZLHPgmFYKQjDaIE1pDeyipWCMIwmZLnPgmHYHYBhNCHrfRaMfGMKwDCakIc+C0Z+MROQYTSh24b05jw2kow1hTd8kdeFrJuG9KVSqaY897U4ztEz5bmtlaURFu00hTcFYLQk7wvZ6ve/us9Co/ffjeIwjG5oRwGYCchoikXBdNZnwY/zOKtlu/N6t5hGTAEYTcnzQlZLu30W8uo8Pvduscjk5HRu7hbThikAoyl5Xci6pVvncRqxu8X0YWGgRlMqC9lRz32VhWwwYonSwfj4OD09JeBw3Z7D9PSUGB8fj0OsULGcifRhdwBGU8bHx5mcnKaykK12ZlYWsmZdQPNLoVCgWDzU0Hnc7ZVw1c6+cOIEg0NDibCzJ/1u0XwT52IKwGhK2AtZlunEeeyHUqnE2MgIw6dPc+3KCkXHYXpykkPFYqx29iSbvcw34Y2FgRq+WF5erlnIKldPtvhHT7lcZlN/PwfL5br7MdhWKLCwuBjb55LU0NekyhUWFgZqBE67UTBGOMzNzTF8+rSHlR2GT5+ONSorqXeLFsnWGFMAhpEiFk6c4FovGwtwzcoKCydPRizRasIye3VD0n0TcWIKwDC6JErn4uDQEEXHwcvQfsxxGN28OZRx2yFpd4tJ9k3ETSA+ABG5D3gP8KyqvtFjvwAzwAjwIvBBVT3W6rzmAzCSTtRlMpLsA0gq5gNoTFB3AH9BpeVjo5jAdwND7vZm4E/dv4aRWuJIfCoUChwqFs9EAV2zssIxx6HU08OhYjFTC1lQJNU3kQSC6gn8sIhc0eSQm4EDbiP4R0TkAhG5VFWfCWJ8w4iDuJyLw8PDLCwuVvIATp5kdPNmDrSIysp7DHzVN3H//ffzd39XBGB09KNcffXVMUsWL1H5APqBp2uen3JfMwVgpJY4nYvt2NktBr7C8ePH2b37987Mw/z8Q0xPfzx381BLVApAPF7zdD6IyA5gB8DAwECYMhlGV6TBuWj1eSrYPHgTVS2gU8DlNc8vAxa9DlTVe1V1i6pu6evri0Q4w+iENNT7sfo8FWwevIlKATwAfEAqvAV4wez/RtqpOhcLhW04zhhwJ44zRqGwLTHORYuBr2Dz4E0gJiAR+RRwA3CxiJwCfhdYA6CqnwSKVEJAT1IJA/31IMY1jLhJYuJTLWkwU0WBzYM3VgvIMDJM3mLgG5GnebBaQIZhABYDX8XmwRu7AzCMHGDVXCvkYR7auQMwBWCklriaoiSxGYthVDEFYGSe+qYoR2vKIYSZ1BPXuHFjSi89mAJIIXlP1W+HuAqi5bUQW16VXlppRwFYU/gEUCqV6O/fxMREkT17HCYmivT3b6JUKsUtWiLx0xQlS+PGSblcZmxkhIPlModWVrgdOLSywkH39eXl5bhFNLrAFEDM1Kaor6wcAm5nZeUQ5fJBRkbG7AfmQVxNUZLejCUM8qj08oQpgJjpJkW9XC4zOzvLrl3TzM7OUi6XQ5U1KQwODXHUcTz3HXMcBkNqihLXuHGSR6WXJ0wBxEynKep5NhuNj49T6unxqMADpZ6e0GrwxDVunORR6eUJUwAxU0lRP+q5r5KiPnjO63k3G1WbomwrFBhzHO4ExhyHbe7rYTli4xo3TvKo9PKERQHFTCcp6rOzs0xMFN3FfzWOM8bMzGiierKGRTWpZ+HkSQY3b44sqSeuceOiNgqovgOZRQElDysFkSI6SVG3yoYV4mo+nrSm52HjpwOZhTGnE1MACaDdipJW2TDfxJGU1UzpWcex9JIZE1Car0DalT2tlQ3T/BklhaQlZaX1u5hl2jEBoaqJ3a699lr1w/z8vBYKfeo4WxXuVMfZqoVCn87Pz/v6/zjpVPbV/3dH4t9zmj+jpLC0tKR9hYI+CKo124OgfYWClsvlyGXav3+/+5nqOZvjbNXZ2dlI5VlaWtL9+/frzp27df/+/bq0tBTp+EkAOKI+19jYF/lmmx8FsLS0pIVCn8KDdV/AB7VQ6IvlR+GXbmUvl8s6OzurU1PTOjs7m9j3msTPqLpQ7N65MzULxf79+3Wr45y70oJudZzIF1tV1Z07dyvc6akA4A6dmpqOTBa7yKjQjgJIvQ/ATyJVUh123cqeFmdkN+8zDHt3vRml6DhMT04mPqoliUlZUfqjmpkQrel7Z6Q+DyDNETFplr0dukl229TfT3FiAmfPHooTE2zq7+8q2S3NtW2SmJQ1Pj5OT08JPDIFenpKgeUJtEp8tKbvnRGIAhCRm0Tk2yJyUkSmPPZ/UESeE5Hj7vYbQYwLnSVSJYU0y94OnSa7hbFQp7m2TRKTsqphzIXCNhxnDLgTxxmjUNgWWKctP4mPebmYCpquFYCInAf8MfBu4Ergl0XkSo9D51T1P7jbbLfjVonqCiQM0ix7O3TyPsNaqJNoRvFLUjORq2HMMzOjTE29xMzMKIuLC4GZ0/xc3eflYipogvABXAecVNUFABH5a+Bm4JsBnLslae71mWbZ26GT9xnWQj04NETRcfAyWh9zHEYTVNvGy//hJykrDsL0R/m5up+enmJycprKRcbqcNTKRcaBUGRLO0EogH7g6Zrnp4A3exz3iyLyM8B3gN9R1ac9jumIdhOpkkSaZW+Hdt9nWAv1+Pg405OTHstExYxyoMFdV9TJV60c1Wlw/geFH0dzXi6mAsdvuFCjDXg/MFvz/NeAvXXHXASsdR9/CHioyfl2AEeAIwMDAyEFShlJJ8yY9/n5ee0rFHSr4+gdbghlX6HQMFyw9vg7fRzfLUmM94+TdsKI0xIaHSZEmQcAvBX4cs3z3cDuJsefB7zg59x+E8GMbNLuQt0O1YViemqq6UIRx2KcxHj/uElb4mOctKMAgjAB/RMwJCKvB/4fcCvwK7UHiMilqvqM+/R9wJMBjGsknG7NJmHau/3arP04o4M2x6TZUR0WeTGVRk3XCkBVXxWRjwBfpnJ1f5+qPiEiH6eiiR4A/rOIvA94Ffg+8MFuxzWSTVDJVnEnu8WxGKfJUR0lcX8XskggmcCqWgSKda99tObxbiqmISMH1Mbwn7lyXlnhMDA2MsLC4mJqrtziWIw7dVQbRrukPhPYSB5pSLby2085juSrpMb7G9kj9bWAjOSRdBt2O/Xrq4txo45YYS3GSY33N7KFKQCjKZ04cpNsw+6kaFhci7HZvI2wyUxDGCN4Om0+Ui6X2dTfv9oHQMVssq1QiNUH0Kyf8oYN7+WWWy6k/5JLIuu0ZWSDOLq0NcJ6Ahtd040jNy6ziR8alxUocfrFB3n+U8pP/vCHsZaITtJiYrQmreXFgfQ3hDHCIYhkJL/JVlHi3cFqSdeRjMzbqLOOje5IYtY2bSSCWRSQ4UkQjtyqDfuuu+9m+/btiXBgelcmnePt/DD2qKU09yrIK2mIeGuGKQDDkyQ2HwkCr/r1a8/fw3/kZc/jo4xaSupi4jdkNo8kPeKtFaYADE+S2HwkKOrr14//yls5kgBll8TFpFUnrryT+gslv7aiOLZ2fADVJt87d+5OTZPvpBNmMbYksbS0pBdu2OBpx71wwwbfdtxuG83v3btX39Tbq7tB94MuxVwErp0qnHkl7T6A2Bf5ZptfBbC6UuCdVikwQJLoyA2apaUlXb/+Al3Hen0XFWX3Lhxdx3pdv/4CX++5W+ft/Py89m3cqD8Hlf8H7QOdj3Ex8XaYVzbH2ZrLqqReJO1CqR0FkPow0E4Sewz/tJOMlNbwxbm5OXp63sEKBzjMHF/hJMpmYByn5wMtK352W/vozP8vL5+TN/FuYN3GjXw2hvBZ67PrjzRnbadeAfjpF2rZlOGT5ljoswvdRmA7tamRfha6bktGN/v/G9au5aZ77ollDv104soz1QueEycWGBoaTM0FTy2pdwLbVUr8pD18sduG4t06b5v9/5tfeYXFU6ea/n9YeIfMwtk+u+kNBOiWrDjHU68Auv3xGt2T1PBFv3S70HUbCZLUSBKvkFnHGaNQ2JbrPru1ZudKSZHbWVk5RLl8kJGRscRf8NSSegVgVynxE2b4YjUGfXrXrlBi0Ku38b/w3nexdu0YGza8F6+Frpkc3YbMJjnktj5kdmZmlMXFhcSb9cLEj9k5LaTeB1C9Sjlb3vcaHOcYPT2lXF+lRElY1T/D9ivUn//djsPXfvQVtm57De94x9mWg63k8Kp99OjatTysyod+8zcr4XZN8Fs7KS4nu1UlXU2mzM5+w4WabcBNwLeBk8CUx/61wJy7/1HgCj/nbScPoBquODU1HXm4Yt5zELqJhW40d2HGVy8tLenevXvVWbNGf6cu3r7+/O3IUS6XddeuXbp2zXrt7X2Twn9rKyS5Wcit1QhKDkkPjyXKPAAqfYCfAgaBXuAx4Mq6Y34L+KT7+FZgzs+501AMznIQKnQSC91s7oIoRtdMzvesXXtOvL3X+duRI6zEqSQmG+WZpCfItaMAgjABXQecVNUFABH5a+Bm4Js1x9wMfMx9/Glgn4iIK2xqsRyEs7QbC91q7nb8xq8G7lfwjNenYmsfAxaoBILWnr8d/0ZYIcndhpkawZIls3MQCqAfeLrm+SngzY2OUdVXReQF4CLgewGMHxuWg7CadmzFrebuueefZyFgv0LThZSKjXJ73fkHh4b4woYN8OKLLeUIyzacxBpBeafqHJ+bm+PkyQU2bz7rM0oTQSgA8Xit/srezzGVA0V2ADsABgYGupMsZDLlDIqYVnN30UX/xhfdyJj6q/WHRfjZl15ieteutpyhTRdSKncA1cibA27kzcDAAF9+8UVPOWqPg/ASp5LcYjPPZME5HkQY6Cng8prnlwGLjY4RkfOBHwO+73UyVb1XVbeo6pa+vr4AxAsPy0HonFZzd9VVb+BQsci2QoExx+FOYMxx+KX169HTp3loagpnzx6KExNs6u/3lYDTLN7+EeCLvb1scyNyqqGfv3rLLdwFbKNiJrqTij3zZuB/fvrTq674wgpJTnKYaJKwstUd4NdZ0GijchexALyes07gq+qO+TCrncD/y8+5k+4ETrozKGnUVsvcu3evbtx4Ucu5q42M2bdvn/Zt3NixM7SZM/Xf9fbqvn37Vp2j1gFcBp0FnXb/3uQeX89qx/YdgQUFJK3gWNKwYIyzEHU1UGAE+A6VaKDb3dc+DrzPfbwO+BsqYaD/CAz6OW/SFYBqeD/4rOEVxnjhhg26fv0FvucuiMigdhbS3Tt36p1esX6gd4CuXbPe8//CCknOQ2XWTrALsdW0owACSQRT1SJQrHvtozWPXwbeH8RYSSMrzqAwaVYtc9vGHn73nhs5dWqx5dwF4QxtJ1qpme39YRxe+eEOz2ivVrbhThO6smBzDgMLxuic1GcCJwH7YTanafSNKuvWrePuu+8683qjBTIoZ6jfz2t8fJzpyUlPB/A8PcDHOX36u20tMGmumppULBijc0wBGKHTzpV7swWy2YJcH5ETBNUSDaM33sibfwA/ww94GId5eniZIrCxrQWm274Bjc6Zxh4MQWJlq7vAr60oji0NPgCjNX5t934yXuNwhu7du1d7e9+kwpTCrEK5o9T/oLObrTxEBfMBrIa8tYQ0ko3fUgZ+F8ionaFBLTCtnMrTU1PnjNuox7CVh1iNBWOcpR0FYCYg4xyCNiv4rXbp11QUtc8lqNT/dnwYrXwFQZeHSLspyYIxOsSvpohjszuA6AnTrNDqyj2sAnBB0W14p9+r9upxnwPdD7rb/fu5muPavZtohpmSsgVmAjI6IW6zQtPxN27UvXv3pr7kth8fxv79+/Xt69ZpH5VqpbVVS9++bp3Ozs4Gpizj/syN4GlHAaS+I5gRHHG3dqyaiurLP9y6YQPLPzqfqamHUt1/Fc7mIYzOzPDS1BSjMzMsLC6uCgF98hvf4Osvv8xBoNJwsPL3IPD1l1/mW9/8ZmDlIeL+zI14MR+AcYYkVJ2sT9S68bLLODz1MV5a/iuyUnK7lQ/je//6r1yPV1oTXA889/zzvv0qrUjCZ27Eh90BGGdISnPy6gJ51913s3btWlTfThb6r/ql78ILeVuDfdcD337iCWZnZ7n66qtb3k20IimfuREPpgCMMySx6mQeszx/+qqr+Md16zz3/W/gkiNHzlRBPX78+BlluX379rbvhpL4mRvRYQogZpJUwraRDb62RHLU5LHk9vj4OP+wZo3nonwU+Evg0MoKB93M4uXl5Y7HSuJnbkSHVJzGyWTLli165MiRuMUIjVKpVBNbfi2Oc/RMbHmcdWGWl5fP2OAHN29mvElrx7Apl8v0929a1TqywmEKhW2p9AH4oTYP4JqVFf4PcISKM/hqKt3LFoAv9fay/ROf4MMf/nBX4yXpMze6Q0SOquoWX8eaAoiHZgtbb+8t/MEf3MVtt92WqmScdmgn8Wi1olydhJXlAmrVRfm+/ft57aOP8pfAcSqNaYaBa4F/AOZ7e/nCV7+a6bkw/GMKIAXMzs4yMVFkZeWQx9730Nu7yNq1pzK5yNVnuR6tiV5p9F6ri2Ely3MwV1eos7OzFCcmuH9lhU1UwkHri+FtKxQ6KiZnZA9TAClg165p9uxxqER513Mn8BJwQ+bMHOVymU39/asrYmKLWDOqc/ar5TLfpWIGqmfMcRidmbGy5EZbCsCcwDHRzLkJx4BBshjqaIlH7VN11N67Zg3e8VAWs290himAmGjWQBxKQCX8LmuhjpZ41BnDw8Pc84lP8MjatZ77LWbf6ISuFICIXCgih0XkhPv3NQ2O+5GIHHe3B7oZMytUK0wWCttYu/a9VMw+Y8A2Kjf5FTNI1kIdu0k8qobMTu/aFXvIbBzcdtttPNrbazH7RnD4LRrktQF7gCn38RTw+w2OW+7k/HkoBlcul3Xfvn26Zo2j8Durmo1ksaFFp8XHrGJlhTga4gRBtbdB2ov5pQHaKAbXlRNYRL4N3KCqz4jIpcDXVPWnPI5bVtW2PXtZdgLXk6dQx/oY92MtooA6cRynvb59M9IWs5/UfJesElkUkIj8m6peUPP8X1X1HDOQiLxKJYT5VeAeVf2cn/PnSQFAvkId21nEqmGQhzx8B17RL52EmRrhkNdEvjhpRwG0rAYqIl8BLvHY5RW/2IgBVV0UkUHgIRF5XFWfajDeDmAHwMDAQBtDpJ+oO13FSTvvtR3HcRiN12vJ8p1FGMzNzXH69DDNivnl5TufRFo6gVX1nar6Ro/t88C/uKYf3L/PNjjHovt3Afga8KYm492rqltUdUtfX18Hb8nIGu04jsMMMy2VSmzq76c4MYGzZ8+Zgmxp7EsQFXks5pcmuu0H8ABwG3CP+/fz9Qe4kUEvquorInIx8DYqzmMjBqpXsCdOLDA0NNj1FWwUV8Tj4+NMT05ymHMzYEs9PRyoiX4JK8w0jDuLMOcuKXcqlXyXolcbZDfCbTRymYwa/HqLvTbgIuCrwAn374Xu61uAWffx9cDjwGPu3+1+z5+HKKAomZ+f10KhTx1nq8Kd6jhbtVDo6ziCJMrIHL/RL2H1FQ76vGHOXZIippaWlrRQ6FN4sG7ashfhlhSwnsBGPUH/EOPoJduqqXyYcgXZhD3MuUtij9/VFx53dH3hYTSnHQVgmcA5wY8zrt3zRV3SobZTWKPmJ2HVtw+yc1aYc5fEUhvDw8MsLi4wMzPK1NRLzMyMsri4YBFZCcB6AueEoJ1xSS7pUN9XeHTzZg50GVLbjh+iFWHOXVI/lzxFuKUJUwA5IWhn3ODQEEXHweuExxyH0ZDr0rRycga94ATVhB3Cnbu4P5dmBB2AYASAX1tRHJv5AIIjCz6AKnE6Of34IVrRbO4u2rhR9+7dq7t37uyoZEISfQCqwQcgGI3BnMCGF0E74+KoS5PUBa5dvObugvXr9cING7pWbEmrF2SRQNHSjgKwhjA5I+hyE1HXpWm3LESSqZ271112Gb83NcXB5eVAGuUkqV5Qs+53jjPGzMxoaj6zWpJq0gq0FISRLYK2jUft3Euqk7MTaududnaWYdWm0TvtzHOSnK7NAxCu4jOfOcR3vvNUohZRaL7An1vgrsjk5HTqCtxZGKiRKoIMx0wSWVJs9TTuflcC/pCHHuphzx6HiYki/f2bElFao1Qq0d+/iYmJ4jmylctlRkbGKJcPunc1t7Oycohy+SAjI2MsLy/HLb5vTAEYiae2EczLL7/MwyKZa4qSVcUGjbrflYGbgUO88srfkqRFtNUCf//99weaUxMnZgIyEk19aeeHHAdV5ZfWr+cdPT1dhWMmCT95Bkmp79Mu1e53tf0uensP8YMfbCGJVUJbJU1+4QtfZGXles//TVuBO1MARmJpVoBt28aN3HjPPSyeOhVIolfctMozOH78+CpFWHQcpicnU9PjoJoNXA1AOH78Ur70pWQuoq2SJkX+Acc5mokCd6YAjMTStKyBKuvWreOuu++OQ7RQaJTBrKrndkQLsMdBVNQ7vefnk1kltFXS5OjoCKXS74HH/VpPT4nx8QMRSRoAfuNF49gsDyDfBFmALc2EVeE0TpKcG+BHtiQXuKONPAC7AzASS5LLGkRJFiOEvPwCtX2w47yj8SNbvUlr8+ZRxscPnJE7qTkC9VgimJFYOmkGn0WylPxWT5L7YHcq27k5AkfPKI8o/DWRNYUPG1MARm0UUL1jNGnOz7CidEwRpodyuUx//ybK5YPU+wcKhW0sLi6E/lmZAjAyRZLKGjSiPlz1aACKqtaMAD/if/zJnzCsmnhFmGeSUPYislIQIvJ+4GPAG4DrVNVztRaRm4AZ4DwqrSLv6WbcrBC3nTDu8f2SpLIGXoTRL9jLjCCyjp/8yHZeEslE6GsWCbrvRuj49RZ7bVQW/p8CvgZsaXDMecBTwCDQS6U38JV+zp/lKKC4y+PGPX6WCDpKJ4gImaWlJd2/f7/u3Lm7o7LSRmfs37/f/U2d+3VwnK2RRGwRdTnoFgrgrcCXa57vBnb7OW9WFUDcIXBxj581gg5X7XYRyZNyT5qiS8Jvqx0FEEUtoH7g6Zrnp9zXckvQ/XnTNn4t1To/u3ZNMzs7S7lcjmzsoAi6jk83ZoQsFSprRbOCbXFRDSEtFLbhOGPAnTjOGIXCttjDW71oqQBE5Csi8g2P7WafY4jHaw09zyKyQ0SOiMiR5557zucQ6SJuO2Hc41dJ4g+4E8bHxyn19ARWoK5x9cxqluxgw/9NknIPkyQrumqOwMzMKFNTLzEzM8ri4kIinfUtncCq+s4uxzgFXF7z/DJgscl49wL3QiUKqMuxE0nQ/XnTNj6s/gFXF6uKPIcZGRmLJFwuKILsFwwVhTI5OU0npQaSotzDxo9EZcJ3AAAKPUlEQVSiizNwIOmBC1WiMAH9EzAkIq8XkV7gVuCBCMZNLN7lceHsDzzcksZxjw/Zu1Kt1vEZnZnhpakpRmdmWFhc7OiqrxszQjd3D2kiL4oubLoNA90K7AX6gC+IyHFV/XkReR2VcM8RVX1VRD4CfJlKRNB9qvpE15KnmLjT4OMeH7L5Aw7yqq9VqYFGdHP3kCaScBebBSwRLEbiToOPc/woE2bSku8QFKtzCFYr9yTaoTshCRm3ScUygY3EE9UPOO66LHER98VFFCRB0SWxSY8pACMVhP0DtqvE7BOnoguj/EcQmAIwUkOYP+Ak1GUxskmSC/RFVgvIMLolzHC5LDqajWTQtFvd6dOxh6H6xRSAkSiCdNhapIgRFllp0hNFHoBh+CLozOAk5DsY2STo8h9xYT4AwxdhRzuE5bBNQqRImshbyGynZMUHYAogBJIYGtYNUUQ7hOmwzUNIZBDkNWS2U5Larc6cwDFSv1gWHYfpycnYvxSdEkazEy/CdNimpS5LnGSpNlNUVMt/VLvVpbFJjymAAIlqsYySqKIdzGEbL0kvrpZU0n5xYU7gAPGzWKaNqKIdzGEbL1GFzFb7P0zv2pXa/g9ZwhRAgGQlNKyWqKId0tZII2tEUUW0VCqxqb+f4sQEzp49FCcm2NTfn7r+D5nCb+uwOLa0tYQMujdsElhaWtK+QkEfrHs/D4L2FQqBt7grl8s6OzurU1PTOjs7a+0pIyLsVoZRf4/yDFH3BA5rS5sCCOtLHnff0/n5ee0rFHSr4+gdrjLrKxQy2WM2z6zuJXxHoL2Es3hxlFTaUQDmBA6QoDtDgVdoXpHJyelIQ/OyEO1gtKbTHgR+yKJ5NAuYAgiYIBfLJIXmpT3awfBHWJ/z4NAQRcfBK8zrmOMwmpLM2axhiWAJxqpZGlkhyZmzWcMSwTKCVbM0skIY5lGje7rtCfx+4GPAG4DrVNXzcl1EvguUgR8Br/rVTnnHkqOMLGG+pOTRlQlIRN4AnAb+DPivLRTAFlX9Xjvnz7sJyDpaGYbRLpGZgFT1SXfAbk5jNKCaHNWomqUt/oZhdENUPgAFHhQRBf5MVe+NaNzUE2ZonmEY+aalAhCRrwCXeOy6XVU/73Oct6nqooi8FjgsIt9S1YcbjLcD2AEwMDDg8/TZxkIwDcMIg5YKQFXf2e0gqrro/n1WRD4LXAd4KgD37uBeqPgAuh3bMOLCmqsYSSf0YnAi4ohIofoY+DngG2GPaxhxEnR7S8MIg27DQLcCe4E+4AsiclxVf15EXgfMquoI8OPAZ11H8fnAX6nql7qU2zASS5IyuA2jGV3dAajqZ1X1MlVdq6o/rqo/776+6C7+qOqCql7tblep6l1BCG4YScVPcxXDSALWD8AwAsYyuI20YArAMAImiuYqhhEEpgAMI2CsvaWRFqwYnGEEjGVwG2nBFIBhhECYGdyWX2AEhfUDMIwUcW6HuKNn7iyi6hBnJBvrB2AYGcTyC4ygMSewYaQEyy8wgsYUgGGkBMsvMILGFIBhpATLLzCCxhSAYaQEyy8wgsacwIaREiy/wAgaUwCGkSKsQ5wRJKYADCNlWIc4IyjMB2AYhpFTTAEYhmHkFFMAhmEYOaUrBSAi/11EviUiXxeRz4rIBQ2Ou0lEvi0iJ0VkqpsxDcMwjGDo9g7gMPBGVf33wHeA3fUHiMh5wB8D7wauBH5ZRK7sclzDMAyjS7rtCfygqr7qPn0EuMzjsOuAk25v4B8Afw3c3M24hmEYRvcE6QP4T8AXPV7vB56ueX7Kfc0wDMOIkZZ5ACLyFeASj123q+rn3WNuB14FDnqdwuO1hk0IRGQHsANgYGCglXiGYRhGh7RUAKr6zmb7ReQ24D3AjerdXeYUcHnN88uAxSbj3QvcC5WGMK3kMwzDMDqj2yigm4BdwPtU9cUGh/0TMCQirxeRXuBW4IFuxjUMwzC6p6uWkCJyElgLPO++9IiqfkhEXgfMquqIe9wI8IfAecB9qnqXz/M/B/xzxwIahmHkj59Q1T4/Bya6J7BhGIYRHpYJbBiGkVNMARiGYeQUUwCGYRg5xRSAYRhGTjEFYBiGkVMSrQCSXm1URN4vIk+IyGkR2dLkuO+KyOMiclxEjkQlX5syxlaxVUQuFJHDInLC/fuaBsf9yJ3D4yISSS5Jq3kRkbUiMufuf1RErohCrjbk+6CIPFczb78RpXyuDPeJyLMi8o0G+0VE/sh9D18XkWsSKOMNIvJCzTx+NGL5LheRvxeRJ93f8297HNP+PKpqYjfg54Dz3ce/D/y+xzHnAU8Bg0Av8BhwZUTyvQH4KeBrwJYmx30XuDimOWwpY5xz6I6/B5hyH095fc7uvuWI567lvAC/BXzSfXwrMJcw+T4I7Ivju1cjw88A1wDfaLB/hEodMQHeAjyaQBlvAP4uxjm8FLjGfVygUn25/rNuex4TfQegCa82qqpPquq3oxirU3zKGHfF1puB+93H9wO/EOHYzfAzL7Wyfxq4UUS86l/FJV/sqOrDwPebHHIzcEArPAJcICKXRiNdBR8yxoqqPqOqx9zHZeBJzi2q2fY8JloB1JHmaqMKPCgiR91id0kj7jn8cVV9BipfdOC1DY5bJyJHROQREYlCSfiZlzPHuBcrLwAXRSDbqrFdGn1uv+iaBD4tIpd77I+buL9/fnmriDwmIl8UkaviEsI1M74JeLRuV9vz2LIYXNhEXW00DPl88DZVXRSR1wKHReRb7hVHUmQMdQ6huYxtnGbAncdB4CEReVxVnwpGQk/8zEvoc9cEP2P/LfApVX1FRD5E5W7lZ0OXrD3inEO/HKNSYmHZLW3zOWAoaiFEZCPwGWBCVZfqd3v8S9N5jF0BaMTVRoOWz+c5Ft2/z4rIZ6ncugemAAKQMdQ5hOYyisi/iMilqvqMe8v6bINzVOdxQUS+RuUqKEwF4GdeqsecEpHzgR8jOlNCS/lU9fmap/up+NKSRujfv26pXWxVtSgifyIiF6vq96KSQUTWUFn8D6rqIY9D2p7HRJuAJAPVRkXEEZFC9TEVx7ZnpEGMxD2HDwC3uY9vA865axGR14jIWvfxxcDbgG+GLJefeamV/RbgoQYXKrHIV2cDfh8V23HSeAD4gBvF8hbghapJMCmIyCVV346IXEdl7Xy++X8FOr4Afw48qaqfaHBY+/MYl1fbp+f7JBWb1nF3q0ZbvA4o1nm/v0PlavD2COXbSkXrvgL8C/DlevmoRGg85m5PRCmfXxnjnEN37IuArwIn3L8Xuq9voVJVFuB64HF3Hh8Htkck2znzAnycykUJwDrgb9zv6j8CgxHPXSv57na/d48Bfw/8dJTyuTJ8CngG+KH7XdwOfAj4kLtfqPQNf8r9bBtG1MUo40dq5vER4PqI5RumYs75es16ONLtPFo1UMMwjJySaBOQYRiGER6mAAzDMHKKKQDDMIycYgrAMAwjp5gCMAzDyCmmAAzDMHKKKQDDMIycYgrAMAwjp/x/xbT2gfYGCtQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fb0766ff160>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def load_pts(csv_name):\n",
    "    data = np.asarray(pd.read_csv(csv_name, header=None))\n",
    "    X = data[:,0:2]\n",
    "    y = data[:,2]\n",
    "\n",
    "    plt.scatter(X[np.argwhere(y==0).flatten(),0], X[np.argwhere(y==0).flatten(),1],s = 50, color = 'blue', edgecolor = 'k')\n",
    "    plt.scatter(X[np.argwhere(y==1).flatten(),0], X[np.argwhere(y==1).flatten(),1],s = 50, color = 'red', edgecolor = 'k')\n",
    "    \n",
    "    plt.xlim(-2.05,2.05)\n",
    "    plt.ylim(-2.05,2.05)\n",
    "    plt.grid(False)\n",
    "    plt.tick_params(\n",
    "    axis='x',\n",
    "    which='both',\n",
    "    bottom='off',\n",
    "    top='off')\n",
    "\n",
    "    return X,y\n",
    "\n",
    "X, y = load_pts('data.csv')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function will help us plot the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_model(X, y, clf):\n",
    "    plt.scatter(X[np.argwhere(y==0).flatten(),0],X[np.argwhere(y==0).flatten(),1],s = 50, color = 'blue', edgecolor = 'k')\n",
    "    plt.scatter(X[np.argwhere(y==1).flatten(),0],X[np.argwhere(y==1).flatten(),1],s = 50, color = 'red', edgecolor = 'k')\n",
    "\n",
    "    plt.xlim(-2.05,2.05)\n",
    "    plt.ylim(-2.05,2.05)\n",
    "    plt.grid(False)\n",
    "    plt.tick_params(\n",
    "    axis='x',\n",
    "    which='both',\n",
    "    bottom='off',\n",
    "    top='off')\n",
    "\n",
    "    r = np.linspace(-2.1,2.1,300)\n",
    "    s,t = np.meshgrid(r,r)\n",
    "    s = np.reshape(s,(np.size(s),1))\n",
    "    t = np.reshape(t,(np.size(t),1))\n",
    "    h = np.concatenate((s,t),1)\n",
    "\n",
    "    z = clf.predict(h)\n",
    "\n",
    "    s.shape = (np.size(r),np.size(r))\n",
    "    t.shape = (np.size(r),np.size(r))\n",
    "    z.shape = (np.size(r),np.size(r))\n",
    "\n",
    "    plt.contourf(s,t,z,colors = ['blue','red'],alpha = 0.2,levels = range(-1,2))\n",
    "    if len(np.unique(z)) > 1:\n",
    "        plt.contour(s,t,z,colors = 'k', linewidths = 2)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Splitting our data into training and testing sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import f1_score, make_scorer\n",
    "\n",
    "#Fixing a random seed\n",
    "import random\n",
    "random.seed(42)\n",
    "\n",
    "# Split the data into training and testing sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Fitting a Decision Tree model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "# Define the model (with default hyperparameters)\n",
    "clf = DecisionTreeClassifier(random_state=42)\n",
    "\n",
    "# Fit the model\n",
    "clf.fit(X_train, y_train)\n",
    "\n",
    "# Make predictions using the unoptimized and model\n",
    "train_predictions = clf.predict(X_train)\n",
    "test_predictions = clf.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's plot the model, and find the testing f1_score, to see how we did."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztnXt8VOWZ+L9PEsgk0QoWCkJItS0KtNqgUamyW7e7SY1SQ7fa2osVV0tti2GtsdqVXj6WtbWksURQq6xV/Fnb1UpMrbLQ6rYFxSJmVBRU7CoJWMELqJNMru/vj5kJk+TM/cycc2ae7+eTT2bOOXPeJyfJ+7zvcxVjDIqiKErhUeS0AIqiKIozqAJQFEUpUFQBKIqiFCiqABRFUQoUVQCKoigFiioARVGUAkUVgKIoSoGiCkBRFKVAyVgBiMgMEXlURHaIyHMistTiGhGRVhHZJSLPiMiJmY6rKIqiZEaJDfcYAK4wxjwlIocD20RkozHm+ahr6oGZ4a9TgZvD3+MyYcIkM23a0TaIqCiKUhjs2LHtDWPM5GSuzVgBGGNeA14Lv35XRHYA04FoBdAArDWhuhNbRGSCiBwV/mxMpk07mrvuejJTERVFUQqGmhp5NdlrbfUBiMjRwFzgiVGnpgOdUe+7wscURVEUh7BNAYjIYcBvgX83xrwz+rTFRyyr0InIYhF5UkSefPvt/XaJpyiKoozCFgUgIuMITf53G2Put7ikC5gR9b4S2Gt1L2PMrcaYGmNMzcSJSZmxFEVRlDSwIwpIgP8CdhhjWmJc1g58NRwNNA84mMj+ryiKomQXO6KATgcuAJ4VEX/42H8AVQDGmFuAh4CzgF1AN3CRDeMqiqIoGWBHFNAmrG380dcY4FuZjqUoiqLYh2YCK4qiFCiqABRFUQoUVQCKoigFiioARVGUAkUVgKIoSoFiRxioouSEQCDAxo0b2L17D1VV06mtraOiosJpsRTFs6gCUDyB399BY+OVDA1VEwzOxufbTEvLalpbV1BdPddp8RTFk6gCUFxPIBCgsfFKuruXA/MACAYBttDYeCXr17dTXl7upIiK4knUB6C4no0bNzA0VE1k8j/EPIaGqtmwYYMTYimK51EFoLie3bv3EAzOtjwXDM6iq2tPjiVSlPxATUB5Rj46SquqpuPzbQ6bfUbi8+2ksnJ+7oVSlDxAdwB5hN/fQX19A83Nm1m7tozm5s3U1zfg93c4LVpG1NbWUVTkB7aMOrOFoiI/dXV1ToilKJ5HdwB5Qj47SisqKmhtXREVBTQLn28nRUV+WltXePbnUhSnUQWQJyTjKF24cKETotlCdfVc1q9vZ8OGDXR17aGycj51ddfq5K8oGaAKIE8oBEdpeXm5p5WYorgN9QHkCSFH6Q7LcyFH6fQcS6QoittRBZAnqKNUUZRUURNQnqCOUkVRUsUWBSAitwMLgH3GmI9ZnD8DeAD4v/Ch+40x19oxtnIIdZQqipIKdu0A7gBWAWvjXPMXY8wCm8ZTYqCOUkVRksUWH4Ax5s/AW3bcS1EURckNuXQCf0JEnhaRh0XkozkcV1EURbEgV07gp4APGmPeE5GzgDZgptWFIrIYWAwwdWpVjsRTFEUpPHKyAzDGvGOMeS/8+iFgnIhMinHtrcaYGmNMzcSJk3MhnqIoSkGSEwUgIlNFRMKvTwmP+2YuxlYURVGssSsM9B7gDGCSiHQBPwDGARhjbgHOBb4hIgNAD3C+McbYMbaiKIqSHrYoAGPMFxOcX0UoTFRRPEk+9llQFM0EVpQEaEN6JV9RBaAoccjnPguKosXgFCUO2pBeyWdUAShKHAqhz4JSuKgJSFHikGlDenUeK25GdwBKUgQCAdra1tHauoq2tnUEAgGnRcoJmfRZ8Ps7qK9voLl5M2vXltHcvJn6+gb8/o6syqwoySJuDsefM6fG3HXXk06LUfCMjYLZMdxnoBCiYEb+/CP7LMT6+QOBAPX1DSOcxyG2UF6+TJ3HStaoqZFtxpiaZK5VE5ASF42CSa/PQjLO43wt261mL++gCkCJSyFPZNGk2mehUJ3HmjPhLVQBKHEp1IksUzJ1HnsR3S16D3UCK3EJTWQ7LM+FJrLpOZbIG2TiPPYqmjPhPXQHoMSltraOlpbVhCaykc7M0ESmrZ2tqKiooLV1RUzncaYr4YidvWt3J5VVM1xhZ3f7blF9E2NRBaDEJdsTWT6TjvM4Gfz+Dpoal1I9NMTsYA+bfGWsarmB5taVjtrZ3Wz2Ut+ENRoGqiRFd3d31EQ2nbq6Op38HSAQCNBQX8/y7sCo/RgsK6+gff16x34vbg19datc2ULDQBXbSTUKRskOGzduoHpoyMLKDtVDQ45GZbl1t6iRbLFRBaAoHqJrdyezgz2W52YFe9jT1ZljiUaSLbNXJrjdN+EkqgAUJUNy6VysrJrBJl8ZWCiBnb4y5lfOyMq4qeC23aKbfRNOY0sYqIjcLiL7RGR7jPMiIq0isktEnhGRE+0YV1GcJtf1fmpr6/AXFVkEl4K/qCgvw0szpRBDcpPFrh3AHYRaPq6Ncb4emBn+OhW4OfxdUTyLE4lPFRUVNLeuHI4CmhXsYaevDH9REc2tK/PKmWkXbvVNuAG7egL/WUSOjnNJA7A23Ah+i4hMEJGjjDGv2TG+ojiBU87F6uq5tK9fz4YNG9jT1cn8yhlcmyAqq9Bj4CO+iQcffJC//GUzAPPnf42ZM491WDJnyZUPYDoQ7Z3qCh9TBaB4Fiedi6nY2TUGPsSLL77AqlW3Dj+Hjo6trF59W8E9h2hypQDE4phlAoKILAYWA0ydWpVNmRQlI7zgXNT6PCH0OViTq1pAXUB0eEIlsNfqQmPMrcaYGmNMzcSJk3MinKKkgxeci1qfJ4Q+B2tytQNoB5aIyK8JOX8Pqv1f8TpecC5qDHwIfQ7W2KIAROQe4Axgkoh0AT8AxgEYY24BHgLOAnYB3cBFdoyrKE7jxsSnaLxgpsoF+hys0VpAipLHFFodnFgU0nPQWkBKWjz66Dr+/OffZXWMk0/+FGed9ZWY519++XnuuWclg4P9WZUDoLTUx0UXfZcpUw65p15/vYtf/vIn9PZ2p33fiROn8LWvfY+yskMTyvbtT/Db3/4iI3nTZe7cw3j88X/FmCMx5ghEDiLyFnPnnsKKFUsckckJ9DmMRXcACgCf+9wcXn3VuvGL3UyePI2HHx5rc73ppmu4/fbrciJDNN///u2cc85FPPjgWn74wwttu+99973A0Ucfy/e+91Uefvgu2+6rKAlIegegHcEUgJxN/gD79++lu/u9Mcd/85vVOZMhmjvu+PGI73Zx3303AbBhw69tva+i2IWagJQx/PHmm7Ny33/+xjeGXw8NDY05H70bXfO973HMtGkx7/XSrl38YtUqPjw0xAf7+3l13DheLiri60uWMPMjH0koy4YtW7j+zjsBGBwcCo9/SKYrL7iAM087LeVxv7NyJdt27gzfd2DM5x9oaeGwsrKE8rmNnmCQbdu28cbrrzNpyhROOukkynw+p8VSLIj+P0uEKgCX4JZU/SIRPnXyyVm597iSEvoHxk6KVnzyxBP5yAzrypaBQIDrrriCm3p7D7nz+vtDTVFuvjmppijvBgJxz5/6sY+NeQ7JjDuzqmpYAVjxTyedxOEeK8HQ4fdz3RVXUD00xMeDQXb4fNxw//2saG1lbnW10+IpGaAmIBeQ64qSXmfDxo0Jm6Lk07hOEggEuLKxkeXd3TQHg1wMNAeDLO/u5srGRrq703eWK86jCsBholPUg8Fm4GKCwWa6u5eHj+s/2Gj27N7NbKuAbmBWMMierq68GtdJClHpFRKqABwmkxT1QCBAW9s6WltX0da2jkACs0a+ML2qih0x7M87fT6mV1bm1bhOUohKr5BQBeAw6aaoF7LZqK621pGmKE6N6ySFqPQKCVUADhNKUbcOwQylqE8fc7zQzUYVFRWsaG1lWXk5TT4fa4Amn49l5eWsaG3NWkanU+M6SSEqvUJCo4Acpra2jpaW1YT+pUamqIcqSl475jNONSJxE3Orq6OaonQxv7IyYVMUL4/rFBGld2VjY7gDWZCdPh/+oqK8VXqFhCoAh0mnoqRWNgzhVPNxtzU9T5bBwUFO+spXePqll9L6/B8jL8I+gRMvucQewVzEtMmTeen++yn3YK5GOqgCcAGpVpTUyoaFTSAQYMPGjezZvZvpVVXU1dYmlTPy3dWr0578C4W9+/fTcMUVbLzpJqdFyQl5owDckkiVDmNlr427tU7HbOQ2fv/737FgwULP/I7cQoffP2yOmR0MstnnY3VLS1JJWZ1///uI93Y0XBoaGuLgwYPAYYQrwIfpB97jiCOOoKjI/a7Gd955ezhz+/W33nJYmtyRFwrAyz1P05HdC41IRtPh9zMQlQXc2rqF1at/6YnfkVuITsoaVvvBIFuAKxsbk8qAjnDeed/iqqtWZSxTW9s6mps3h4MRRuLzNXHZZfNzai5LdyH4+9/fxQ9+8NUcSOguPK8AvNzrMxPZ3d6IJJpAIMDSxiYMxcAgAL29/wk879jvqCfKfhYIvOeJHIpkkrJy7Ztwkz/KywtBp/C8AvByREymsnvFGblh48bwz7lp1JnEP2e69u54dPj9XHftITNZ8MABGurrGX9Yb0b3zTZuTMrKpT8q3ureywtBJ3G/cS4BblqBpIqXZU+Fzt1d9KTxc3b4/TTU17O5uZmytWvZ3NxMQ309HX5/2rJEzChf6esbPvZ+DMu7A+zf93ra980FbkzKqq2to6jIDxaZAiF/lD15AokSH7Xpe3rYogBE5EwReUFEdonI1RbnF4nIfhHxh79six9LJ5HKLXhZ9lSYUVVJWYo/Z3e42JjdRcgiZpTjRh2fB7i9uLEbk7Ii/qjy8mX4fE3AGny+JsrLl9nmj0om8bFQFlN2k7ECEJFiYDVQD8wBvigicywu/Y0xpjr8tSbTcSPkagWSDbwseyrU1daGf87R3edi/5ybNm/OShGyeGaU8YztUeAm3JqJHPFHNTXNZ9GiXpqa5rN+fbttdvdkVveFspiyGzt8AKcAu4wxfwMQkV8DDcDzNtw7IV6MiIngZdlToaKigpWtzZx0ySXDKqC09BqKi3fE/Dlf37s3K/bu6VVVbPb5+AeLe/e5zCJq5f9wayZyNv1RyazuL7pokedDo53ADgUwHeiMet8FnGpx3edE5B+BF4HLjTGdFtekhZciYkbjZdlTYW51NSVRDWEaGz/BZz5zfcyfc8q0aSF7t8VEvdPnY36a9u662lpWt7TwgVHHtwDW6iZENpzR8UgU7+8F579dJONoLpTFlN3YoQDE4tjovf7vgHuMMb0icilwJ/Apy5uJLAYWA0ydWpW0EF6JiLHCy7Kny9lnL4j7Tzn/9NO5/447LNZzIXv3tWmaxyJmlAu/+U0IO4LfRFhWXsHkw6awb9+rYz6TSfJVOtgZ758PJJv4WCiLKTuxQwF0AdG9+yqBvdEXGGPejHp7G3B9rJsZY24FbgWYM6dmtCJRCoTysF07G0XI5lZXc80PfsD511wDgG/CBNrb13PBBWNt1k5Mxm6M93eSVFb3hbiYygQ7FMBWYKaIHAPsAc4HvhR9gYgcZYx5Lfz2HMDaW6PkFaPNJqmSTXu3r7R0+HVFxWEx7+nEZOzGeH+n0dV9dshYARhjBkRkCfA/QDFwuzHmORG5FnjSGNMONIrIOcAA8BawKNNxFXdjZTYZSLIhfDROr+icmIwjjmq7/R9ex+m/hXzElkxgY8xDwEOjjn0/6vV3ge/aMZbifmKZTVYSWgF4CScm44ij2m7/h6KMxl1xb0peEMtsYhUt4BTJ1gJyIvnKrfH+Sv7h+VpAivuIZzZxAx1+P9dee93w+wMHgtTXN3CYRS0gpzpiuTXeX8kvVAEocUkn/j2e2cRpIpVJe/u+TCggDWAS3d3L6e62jEx2bDJWm7eSbVQBKDFJN/49lg3bDTG9hyqTWlUDKgV6AHjir3/lzHnzhpWdTsZKPHKdKGgXqgAUSzKJf49lNhl0wY4gdmXSDiKTP8DO9nYaHn44a8le8fDqZFKo5DpR0E7UCaxYkkz8ezwiZpP5TU30LlrE/KYmSkqcX29YVyYdYjxLmRy1R7mwvz/jyqPpkI0S2Er2iF4o2Vm1NleoAlAssSP+PWI2+daSJa4xnxyqTPpC1NFu5jLA6P1MppVHU8Xrk0khkulCyWlUASiWuLH5iB1EKpOWjr876ug7nIh1N7BcZt66dTIJBAK0ta2jtXUVbW3rPNE+M1d4PWtbFYBiiRubj9jF3OpqfvCDa4bfl5WVssNXZnltLpWdGyeTRJ24Ch2vL5ScN8raRLx+oUrqOBX/niuiawFNnDiRp94ylI66ZgvwFCSdeRsIBOiKmqRTLX0xecoU2ktK6BkYYDpQB0T+gp0oAaF9dhPj9aztvFAAfn9HVKXA2fh8m2lpWU1r6wrbuhIVIoWUjNRrxvFO1Ib4NkLvxST3LxKJBNkftYJ/8IF2zj7760l//rZVqzhuYIAyYDOhNnsrgF6cmUyS6cTlFt+OU3h9oeR5BaCrlOySSvy7V8MXu7u7EakhVMx2FwAvcg4TWUqZfD/hRBftvF3JIffyvKF+mhqXkigDYvjzPT1jVpGNwPiyMpodmEy0z25yeHmh5HkFoKsUd+DlWOiBgYHwRPe7qKOnAOX0BGfRmWCii3berow6PhUoHhriT0Pxew3Hc/6eNG4cp112mSPPMJlOXIVMZMHTubuLGVWVnlnwRON5J7CuUpzH6+GLJSUlMRuKl/l2MiNBQ/H4ztseEu0A4n3+Y/397N+3L+7ns0VtbV04ZHZsKECoE5e77dvZpMPvp76+gZ81b+LOtWX8rHkT9fUNnsvX8LwCCK1SrP95Q6uU+P+8Sua4NXwxWcrLy8MTXc+oM8lNdPEjQcpIVAfVrZEkkU5c5eXL8PmagDX4fE2Uly8r6D67kXpSge7l9ASbgYvpCTYT6F7O0sYm1y94ovG8AtBVivNkM3wxEAiwrq2NVa2trGtry0oMend3N2f846nA/uFj48etpaJ8GStbmykvL48rR6yQ2b8Tct4WFcX/N3NzyG2kE1dT03wWLeqlqWk+69e3F3RwxaF6UrHNzl7B8z6AVPqFKtkhW01TcuVXCB44wAfXr6eMQ3uAc86ZxQ+X3kx5eXlCOaIjQbqCQQjb/B+jiC+fey533/143PGTjSRxysmuhfBGErueFEn5jNyELQpARM4k5P8qBtYYY34y6nwpsBY4CXgT+IIx5hU7xgbn+4UWeg5CJrHQ0c9uYKB/+Hh32H+QjWbsgUCAv2zaNPz+SAwXAw8xxKvhY9XV1cMr/2TkiESC/MNXvwqvvALAILP47//ezeBg4nyARJEkXnay5xuhelKb6LHY9IZ8Rt5xjmesAESkmFDIci3QBWwVkXZjzPNRl10MvG2M+YiInA9cD3wh07GjcWqVojkI6cdCj3520Q0jN23enJVm7JGJtCxqt/IaMNp15/f7+VJ9fUpN4Y0x7N4dvfo7nmCwGfg5MJhQtlh/w5lUZlXsp662lhtaVoHFkidkdr7WIclSx44dwCnALmPM3wBE5NdAAxCtABqAH4Zf3wesEhExxrihRHzaaA7CIVKNhe7uHvvs4NtAaBfQ1dlpu18heiJ9HXgwfPxIoAk4LOrat998E0jNv7Fh40ZgAtG+hBBFJKMAYpGKElKyT6Se1NLGJoaGqukJzqIsbHaO+Iy8gh0KYDrQGfW+Czg11jXGmAEROQi8H3jDhvEdQ3MQRpLKLuzRRx+J8exCvPvuu6HIGBv9CtET6QNRx33ATOCZqGMT3/9+IOTf2JSkHJ27uxgYOoKxCiAz3FgjqNCZW109bHbu7NrDjBybne3CDgVgFeM2emWfzDWhC0UWA4sBpk6tykyyLKM5COnT2bk35rMDOPzwI/hrODJmtF+hQ4STe3tZ1dqakjM07kQKbIt6Xx22q0+dOpUtYXNLIv/GjKpKSooOQvy8r5TJlpNdyYx8cI7bEQbaBcyIel8J7I11jYiUAEcAb1ndzBhzqzGmxhhTM3HiZBvEyx6ag5A+M2ZMi/nsQuens6K1lWXl5TT5fKwBmnw+ri4txRjD1htvTLlhSrx4++1AT9Q6pXT8eAKBAN/7znf4JrCMkJloDSFD1RXAj3760xErvrraWuCAxd0z0whuDhN1E1q2OnXsUABbgZkicoyIjAfOB9pHXdMOXBh+fS7wiNft/6A5CJnQ29uHSAdjn12I+fPnj+kqdkpjIyVFRVwXDKaVcRxrIg0CHSXjOHzikSOOR0xGXyb0BzyfUGG2fwROLCmhc/fuEddXVFRwUk204/+5cAJV+vb/yH2tlOGy8nJPFBzLBVq2Oj0yNgGFbfpLgP8hFAZ6uzHmORG5FnjSGNMO/Bdwl4jsIrTyPz/Tcd2A5iAkT4ffP6I88lO/+AXjpJih0qsRqSEYnEV0Jm5ZWag+f/Q2e11bG9XGpO0MjY5Wqujvh/6Qw/kthNtuuZkf/eh83n77NQDOveoqRASM4ecW9xoaGGDVT3/KF1esCF0XZjCq7s9xxw1y3nnz+fGPVyYVChoPLxccyzYajJE+tuQBGGMeAh4adez7Ua+DwHl2jOU2nM5B8AKR6JtiDgV6Lu8Nsp1ilpUVsfiyU9i3bz+/+lUJfX39Me9jhzM0MpH+6MYbab/3XgCOmjaN6uq5VFZ+hFdffXH42sgmNd76fcgYiLGZPffcC1i4cCE//vFIF1i6CV35YHPOBhqMkT6eLwXhBiL/mEuWfIuFCxfq5D+KiClldCTAPKDaGMaPH8+SJd+ipGTciM+MLrtgV82c8vJy5p16KFBNJPRv0NLSzrx5n+aww45I/oezQKSIc8/9Bp/97NfGnHv62We16bvNaDBG+ni+FITifhJVy9zT1Tnm+LY1azitr29Exmu2uy8VFxezatX6Ecf8/g6WXvoNPjYAc+mngzI6KKKPlcBcYA2LFvWyZMm3khpj2ZVX8pPouv8ZJnR5tQeDnWjZ6vTRHYCSdRJVy5xeGQkiO2RK+W5f3xgnr4jk3BlaXT2Xb15+OdtKPswtLOIJmuhjPaHJP/VorxNsrJra4ffrbgINxsgE3QEoWSeych9tKR+9co/VQ3e0kzfXztAFCxawevVtDAycTKap/8f19loet/JhxFvda3mIQ2gwRvqoAlDGYLdZIRJ9c9IllwwfW1bqY3vxeJpbVw7/gw4Oxo6Xj54gc+0MtXOCeaG0FCyUwOiErkTF3+wuDxEpyte1u5PKqhmeK2iowRjpoQpAGYEBGurrba86Obe6mpKSEvrDq/xPNDZy3WdGOsyLi4sYjBFy43TGq10TzDMxspujd0KR1f2y7m7eJtSp+PRgkAUcWt3bWR7C7++gqXFp+HfewyZfGatabqC5daWnChpqlFTqqAJQRmCMyYlZ4eyzF4y5V0lJCX19Y6/dAvjD5R9aW1c51n/Vjglm+YoVLPvOd+JWTd2wcSNHDwywHKgGZgObCVUsPXpggA0bNthWHiIQCNDUuJTl3YGo33kPW4CmxqUFZUoqRFQBKGOYxciCBrOAWYODrGtv5+yzz077vkMJk78PBYr+cPx4Tunr48XSUp4G+gaLuKH1MXp6j6Os9FFW/KyVn1y/nBOOPz4tWQ46VCbg48cfn9CH8cquXbzU18f1jN0pXNXXxyv/9398bfFiWyKiNm7coJVGCxhVAMoYJlod7O1ldXMzNDfnRIa7+/q4OzxuWABC6+DNoZfAJxsbcyKL3STaSRx4910+jlVaE3wcePvgwbR7MIyma3cns4OjeyGHiBWiq+QPGgaqADBhwqScjeXzVVBRcfiY45/85Dk5kyGaz3zmwsQX5ZAJ73sfH49x7gTg1ZdfZl1bG8fOnDmiVtL8piba169PyVdTWTWDHb4yy3MjQ3SVfER3AAoAGzfu46mnNvH1r3+aqQwRvX7sBv5OEcccc8xw1my6nHDCJ/iP/7hlRP2cCD/60V185Svf5qabvsdrr4WaM7755hscPFhEqG3LaN5iwoQhjjwyPeU1fnwpl112Paee+s9pfT5bHP3hD/OX8eOxcog8Dbx/xw42NzcPO+czMdHU1taxquWGrCXXKe5GFYDDuKWfsIhQVuZj4cJFPPy73+MDJg/2sb94PG8DCz9zNkcdNc2WsX75y+vinj/hhHmccEJoOnr++efZtOlVBgfrx1xXXPwwH//40cyZE7uvQCK2b3+c7dvjN21PlYkTp/CZzyxi3LhxiS+2YDjjua9vzKS8g1Bl0nKbnPMVFRU0t64cjgKaFexhp68Mf1HRiBBdJT9RBeAgbuonfOGF83juuSeG378S/opUQmtruzmn8ozlsTFHBgfhT396jD/9yQFxEvDjH3+d9vZXOOqo1JsaWdn3nyHUY3UFoVDddYTCQ6f09fHggw/y+c9/Pm1Zq6vnRjmmO5lfOUMrjRYIqgAcIl4J20sv/Xcuv/ybLFiwIGe7gejJX8kcYwz33HMD3/72DWl9Prr8c3tbGxO3b6cdeIFQg+1IeOikgQFWt7Qw89hjM8rT0Bj6wkQVgEPEK2E7MFDNDTe0s3r1bY7sBm675pqs3n93Zyf33nMPM4aGOGpwkNeKi+ksKuK8L36RqhnWTse+vj6ee+453nr7AEdOnMBHP/pRxo8fn1U50+H6O+9kVzgJq78/dmnrZIhMygbYvGsXJhjkSuDQkiHEloGBgiv/oNiDKgCHiFfCFo5nYKCXgYGTct7QokiESz772azdPxAI0FBfzy/7+w9NYoODbBkcZNl993G1xyexP27dOqwA7CLiE/gFoZW/xuwrdqFhoA4Rr58w7ASmE93QIl9IpoaNMpKIT+D+khJiLRlSLf+gKKAKwDHilbANJf2Hwu/yraGFnTVsCom51dVcdvnlPBsjsiiVhjiKEiEjBSAiR4rIRhF5KfzdMolURAZFxB/+Gt0wviCJVJgsL1/GuHH/DqHq9sAyQrEeITNIqvXm3U4mXb0CgQDr2trGdAorFBYsWMD2ceOslwxFRVr3XkmZTHcAVwN/NMbMBP4Yfm9FjzGmOvzlTLqnC4lUmLz88tPP5rqqAAAVP0lEQVQoKbkDmEYoyjvi9M2/hhZ1tbX4wxUxo0k0iWnzk0OmoFw2xLGLQCBAW9s6WltX0da2ruCUt1vJ1AncAJwRfn0n8L/AVRnes6AoLy/n85//PMceOzOcE7A3rxtapFPDJp3mJ/naKjE6PDRXDXEyxU35LspIMlUAU4wxrwEYY14TkQ/EuM4nIk8CA8BPjDFtGY6bdxRSQ4tUJ7FUm58kaqbidbwUsx8v3yXXEW7KWBIqABH5AzDV4lQqweJVxpi9IvIh4BERedYY83KM8RYDiwGmTk09i9LLeOkfO1NS+VlTcRxnu1Vivu4sskW8fJdIhFuh/M27kYQ+AGPMvxhjPmbx9QDwuogcBRD+vi/GPfaGv/+NkJko5r7PGHOrMabGGFMzceLkNH4kJd9IxXGczTBT9UOkTrx8l3yLcPMimZqA2oELgZ+Evz8w+oJwZFC3MaZXRCYBpwM/zXBcJU3sLj6XixXxcHE0EleszFaYaTZ2Ftl8dlb3doJQvstmq8Zl4Qi3+bkXShkm0yignwC1IvISUBt+j4jUiMia8DWzgSdF5GngUUI+gOczHFdJA7+/g/r6BpqbN7N2bRnNzZupr2/A7+9I6365WhGnEv2SSZhpPOzeWWTz2cW699sHDiT+sM3Ey3fJtwg3L5LRDsAY8yYwppi6MeZJ4JLw68eA9Pr2KbaRyBmXzv2yaWsfTbKO41R2C6lg584im88u3r0bnnwyrXtmQiTf5VAUUP5GuHkRrQVUICRyxoXSOJIn1cgcO0jGcWxXq8TR2NWEHbL77OLdeyIxnHRZppAi3LyGKoACIZEzLlXcXNIhG7Hy6e0shsYc2bBlC398/HFKg0HWW3xiIBhk45Yt+D4QK6I6Phvj3DswNFaeXOGlCLfX3niD9Y+N7T+Rj6gCKBASOeNizOUxsXNFnA6JHKh2TzjJ7iyKi4uHP3PvvTfzhS8sHXGfc686lCf581iD/eEP3PSHP2Qkb8x7hykuzv2/vlu631kR/TzeOHCA+sZGB6XJHWKMcVqGmMyZU2Puuiv3dst8JBAIUF/fMMIHEGIL5eXL6O4OmYCKRBjcujWp+zXU14+0NRNaES8rL89qbfrRiV47oibibCd6dXd3D+8spldWUjdqZ/H0iy9S/aUvDb+vrv4HKis/xIMP3plVuVJl3bqXmDHjIzkbb2w28I5hP4AbsoHfe+8dPvWp9zM0NOC0KHawzRhTk8yFqgAKiJH/hCOdcZdcciKQvAKAkRPx6BVxtiZiJxVPstzwq1/x7ZYWAGbPPpm77vorL7zQQVvbGowJmWH273+Dx/6yiUnGMMEMcEBK2AcIMBmYaAZ4W0p4Q4TT/mE+kycn3/je6t6R+8ycOYeLLvoupaXWkVLZINHiwy3ZwENDQ9x9dwt79ljmqHqG++67JWkFoCagAsJuZ5wTdWmccD6nSllp6Zhjxx03l6uuWj3i2KHdRCeTPzCFW2+8keU9UYrNwBYDy7a+Tvv6O1N6rtH3nl45Y8xOJZd4JRu4qKiICy5oSvp6t5q07rvvlqSvVQVQYNhtG8+1c8/NzudUiX52bW3rqDbGNsXmJqdr/ACED/PII4+ye3eXqyZRiD/B50uBO20Io3iKbCV6OU3X7k5mB3ssz80K9rCnqzPHEtlH7O53HcCv2LpVbElMtJN4SZPROTXBYDNwMcFgM93dy8PHu50WP2lUASiuJ7oRTF9vLx0iedcUpbJqBjt8ZZbndvrKmF45I8cS2Yd1NnAAuAJYQX//z3HTJJpogn/wwQcTmrS8gpqAFFczOuJnq8+HAa4uLaVGxLZEL6epra1jVcsNcfMMIiaJrt2dVFbNcJW5JB5W2cAlJY8yMDAbN/oFEvksNm16jGDwBMvPeq3AnSoAxbXEK2uwrKyMUy67jP379nmiKUoiKioqaG5dSVPj0nBUVQ87fWX4i4pobl3Jiy++MHxudrCHTb4yVrXcQHPrSk/YnEcHILzwwiQef9ydk2iipEmRZ/H5duRFgTtVAIpriRvxYwzjx4/nW0uWOCFaVqiunhsVVdXJ/MoZXFtXhzEmHPoaiFKEPWwBmhqXuiL0NRlGO707OtxZJTRR0uT8+afj998KFvu1UIG7a3MkaeaoD0BxLfkU8ZMskUnyW0suY+HChZSXl7Nx44as9ThwCjdXCU0k24IFC8KF7Jbh8zUBa/D5migvX+a5Ane6A1Bci9PlJtxCPkYIublKaDKyJcqpcWuOwGhUASiuJVulnb1GZdUMNvnKwEIJ7PSVMd+jEUJurhKajGyxci28lCOgCkBxLdkq7ZwtIgXq/veRR2y9bzIRQl7FTQlro0lHtkR9N9xS9iKCKgDF1ThRbiIdosNVx0WZrLq7A2nfM9qMcM6557Ls3nupNmZMhJDbnkUh45WyFxEyUgAich7wQ0JtH08JdwKzuu5MYCVQDKwxxvwkk3HzBafthNHjuxk3rxJhbLjqLcD/C5/rfPVVuru7U56krapniozng19YSK8wHCGkk7+7SBRC6rYcgUx3ANuBfwV+EesCESkGVhPqGdwFbBWR9kLvC+y0nXD0+BHcWxvWvcQKVwUoh5RXffHMCPfem1z1TKcXF4VKohBSt+UIZNoTeAeAiMS77BRglzHmb+Frfw00AAWrAJy2E1qNH27hjDEmrRXraLp7elj1m9/w9nvvZSasB9i2dSslwSDfDb+P9gCUMZRylE6mZgSnFxe5xG2Krra2jpaW1XglRyAXPoDpQPR/QBdwag7GdS1O2wljjx8i0/E3+/3Mv+SStD/vVR62OBakOOU6PpmYEZxeXOQSNyo6N4e3WpFQAYjIH4CpFqeuMcY8kMQYVtuDmJYGEVkMLAaYOrUqidt7D6fthPHGB+jMcPyrV63K6PP5xND4D6ec1JSJGcHpxUWucLOic3N462gSKgBjzL9kOEYXEL0EqgT2xhnvVuBWCHUEy3BsV+K0nTDe+AAzKqdndP+BwcHh12ec8Vk+/enzx1wTDAa57rqf0t9/PnBs1JkXGTfu11xzzVWUWjRWcSuvvPI31t5+B0cbw/SBPv5e4qNz3CR+dmPq4aqZmBGcXlzkCrcrOrcHLkTIhQloKzBTRI4B9gDnA1+K/5H8xmk7YezxQ9iZhn/yyZ+itvbzY463ta2juPhM+vuXjTlXXHyAgYHDWbDA/f9A0Xz5y43DdXzmZdCFKxMzgtOLi1xRKIou22QaBvpZ4EZCbUx/LyJ+Y8ynRWQaoXDPs4wxAyKyBPgfQmGgtxtjnstYcg/jtJ3QavwIIpKTrWo+/gPbuepL14zgxOLCGMM777xNLvuLT5r0PkpLn6C3d+zzLi3t4MgjT+XAgTdzJo9XyTQKaB2wzuL4XuCsqPcPAQ9lMla+4bSdcPT4d9wROh43nstGcrlSdVukSLKko1ByvbjYtu1PXHrpP+V08h/JTWOO9PZCS8tNtLQ4II7HEOd+cYmZM6fG3HWXZW6ZYjM1NaGpv0iEwa1bM7rXJy66iC3PPgvAlVfeyBe+MLZkcyAQoL6+YVQoKsAWysuTi3VPBquEqshkmG8hkdFEmsKHFhfTs9YU/pOfnEAgcND2+yoZsc0YU5PMha4uBTEw0M8bb/zdaTGULJCLlaqbI0WyTa6ckIOD/cOvj62qorzMuq1lthgaGuKdd96hv7+fcePG8b73vY+iotxUue/p7mZPVxdlxuADgkCPCNMrKylz8O/K/8ILSV/ragXw0kvPcOaZRzkthpIlsm0Gc3ukSL7xx5tvpnLKFKfFyAmBQICG+nruMGbk/tUYlr35Ju133+3Y4kJqklr8Ay5XAErumXf88TkdL5sr1Xx0NCvuIG63unCTHi8sLlytACp8Po7/yEecFqMwEOGb557LBWef7agYdjpsCyUkUsk9+dKtztUKYNYxx/B4JDxFyXvsTu13Ot9CyV/ypVudqxWA4h4izU727N7N9Koq6mprbQ2lzIbD1ul8Cy/i1ZDZXJMv3epUAWSBbE+WuSa62cnsYJDNPh+rW1pY0drK3OpqW8bIlsPW6XwLL+HG4mpuxWvd6mKhCsBmcjFZ5pLRzU4ACAbZAlzZ2Ej7+vW2/LFn02HrlbosTlLIIbPp4pVudfFQBWAjuZosc0muoh3UYessGjKbHl5fXOQmY6JASGay9Bq5inaora2jqMhPyIoaTcRh6w2bqlfJVchsIBBgXVsbq1pbWdfWRiCQfs9kJXN0B2Aj+RIaFk2uoh3UYessudiB5Zt5NB/QHYCNTK+qYofPZ3lup8/HdI+EhkVTV1uLv6jIYl0einawc2Uecdg2Nc1n0aJemprms359uzogc0C2d2DR5tHmYJCLgeZgkOXd3VzZ2Eh3d3dG91fSw9U7gG07dqSU1uwWfmZ1MBjkZ8uXw/LluRYnacaPG8djt9/OSbMPmQJyHe3gdZuqV8n2DixfMmfzDVcrACW39PX3c9q//Ru9jz8+4ng+RDsoiclmyGw+mkfzAVcrgKKiYg477H1Oi5EyxkB/fx+Dg0MUFxcxbtx4JI1C+0ND8N577wDljPxVDQDdHH74+9K6rxXvvPN26M4DA5bndWVeGGTr95wvmbP5hqsVwHHHVVPI/QDa2tbR3LyZYLB5zDmfr4mlS+fb9s8a6QegKNkgXzJn8w1XK4BCR6tZKvlCvmTO5huZ9gQ+D/ghMBs4xRhjuVwXkVeAd4FBYCDZbjWFjiZHKfmE+pLcR6Y7gO3AvwK/SOLafzLGvJHheAWFVrNU8g31JbmLTJvC7wAQuzyRygg0OUpRlGySKx+AATaIiAF+YYy5NUfjeh6tZqkoSrZIqABE5A/AVItT1xhjHkhynNONMXtF5APARhHZaYz5c4zxFgOLAaZOrUry9vmNbpsVRckGCRWAMeZfMh3EGLM3/H2fiKwDTgEsFUB4d3ArwJw5NSbTsRXFKbS5iuJ2sl4LSEQqROTwyGugjpDzWFHyFr+/g/r6BpqbN7N2bRnNzZupr2/A7+9wWjRFGSbTMNDPAjcCk4Hfi4jfGPNpEZkGrDHGnAVMAdaFHcUlwK+MMeszlFtRXIs2V1G8QqZRQOuAdRbH9wJnhV//Dfh4JuMoipfQ5iqKV9By0IpiM5rBrXgFVQCKYjOhDO4dludCGdzTcyyRolijCkBRbEbbWypeQYvBKYrNaAa34hVUAShKFshmBrfmFyh2oQpAUbJENjK4/f6OqJ3FbHy+zbS0rKa1dYX2TlZSRhWAongEzS9Q7EadwIriEZLJL1CUVFAFoCgeQfMLFLtRBaAoHkHzCxS7UQWgKB5B8wsUu1EnsKJ4BM0vUOxGFYCieAjtEKfYiSoARfEY2iFOsQv1ASiKohQoqgAURVEKFFUAiqIoBUpGCkBEVojIThF5RkTWiciEGNedKSIviMguEbk6kzEVRVEUe8h0B7AR+Jgx5gTgReC7oy8QkWJgNVAPzAG+KCJzMhxXURRFyZCMFIAxZoMxZiD8dgtQaXHZKcAuY8zfjDF9wK+BhkzGVRRFUTLHTh/AvwEPWxyfDnRGve8KH1MURVEcJGEegIj8AZhqceoaY8wD4WuuAQaAu61uYXHMxBlvMbAYYOrUqkTiKYqiKGmSUAEYY/4l3nkRuRBYAPyzMcZqYu8CZkS9rwT2xhnvVuBWgDlzamIqCkVRFCUzMo0COhO4CjjHGNMd47KtwEwROUZExgPnA+2ZjKsoiqJkjlgv2pP8sMguoBR4M3xoizHmUhGZBqwxxpwVvu4s4OdAMXC7MeY/k7z/fuDVtAVUFEUpPD5ojJmczIUZKQBFURTFu2gmsKIoSoGiCkBRFKVAUQWgKIpSoKgCUBRFKVBUASiKohQorlYAbq82KiLnichzIjIkIjVxrntFRJ4VEb+IPJkr+VKU0bGKrSJypIhsFJGXwt8nxrhuMPwM/SKSk1ySRM9FREpF5Dfh80+IyNG5kCsF+RaJyP6o53ZJLuULy3C7iOwTke0xzouItIZ/hmdE5EQXyniGiByMeo7fz7F8M0TkURHZEf5/XmpxTerP0Rjj2i+gDigJv74euN7immLgZeBDwHjgaWBOjuSbDRwH/C9QE+e6V4BJDj3DhDI6+QzD4/8UuDr8+mqr33P43Hs5fnYJnwvwTeCW8Ovzgd+4TL5FwCon/vaiZPhH4ERge4zzZxGqIybAPOAJF8p4BvCgg8/wKODE8OvDCVVfHv27Tvk5unoHYFxebdQYs8MY80IuxkqXJGV0umJrA3Bn+PWdgFsa3ibzXKJlvw/4ZxGxqn/llHyOY4z5M/BWnEsagLUmxBZggogclRvpQiQho6MYY14zxjwVfv0usIOxRTVTfo6uVgCj8HK1UQNsEJFt4WJ3bsPpZzjFGPMahP7QgQ/EuM4nIk+KyBYRyYWSSOa5DF8TXqwcBN6fA9lGjB0m1u/tc2GTwH0iMsPivNM4/feXLJ8QkadF5GER+ahTQoTNjHOBJ0adSvk5JiwGl21yXW00G/IlwenGmL0i8gFgo4jsDK843CJjVp8hxJcxhdtUhZ/jh4BHRORZY8zL9khoSTLPJevPLg7JjP074B5jTK+IXEpot/KprEuWGk4+w2R5ilCJhffCpW3agJm5FkJEDgN+C/y7Mead0actPhL3OTquAEyOq43aLV+S99gb/r5PRNYR2rrbpgBskDGrzxDiyygir4vIUcaY18Jb1n0x7hF5jn8Tkf8ltArKpgJI5rlErukSkRLgCHJnSkgonzHmzai3txHypbmNrP/9ZUr0ZGuMeUhEbhKRScaYN3Ilg4iMIzT5322Mud/ikpSfo6tNQJIH1UZFpEJEDo+8JuTYtow0cBCnn2E7cGH49YXAmF2LiEwUkdLw60nA6cDzWZYrmecSLfu5wCMxFiqOyDfKBnwOIdux22gHvhqOYpkHHIyYBN2CiEyN+HZE5BRCc+eb8T9l6/gC/BewwxjTEuOy1J+jU17tJD3fuwjZtPzhr0i0xTTgoVHe7xcJrQavyaF8nyWkdXuB14H/GS0foQiNp8Nfz+VSvmRldPIZhsd+P/BH4KXw9yPDx2sIVZUFOA14NvwcnwUuzpFsY54LcC2hRQmAD7g3/Lf6V+BDOX52ieT7cfjv7mngUWBWLuULy3AP8BrQH/5bvBi4FLg0fF4I9Q1/Ofy7jRlR56CMS6Ke4xbgtBzLN5+QOeeZqPnwrEyfo1YDVRRFKVBcbQJSFEVRsocqAEVRlAJFFYCiKEqBogpAURSlQFEFoCiKUqCoAlAURSlQVAEoiqIUKKoAFEVRCpT/D+9njH6uAi5kAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fb07d1a8c50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Training F1 Score is 1.0\n",
      "The Testing F1 Score is 0.7\n"
     ]
    }
   ],
   "source": [
    "plot_model(X, y, clf)\n",
    "print('The Training F1 Score is', f1_score(train_predictions, y_train))\n",
    "print('The Testing F1 Score is', f1_score(test_predictions, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Woah! Some heavy overfitting there. Not just from looking at the graph, but also from looking at the difference between the high training score (1.0) and the low testing score (0.7).Let's see if we can find better hyperparameters for this model to do better. We'll use grid search for this.\n",
    "\n",
    "### 4. (SOLUTION) Use grid search to improve this model.\n",
    "\n",
    "In here, we'll do the following steps:\n",
    "1. First define some parameters to perform grid search on. We suggest to play with `max_depth`, `min_samples_leaf`, and `min_samples_split`.\n",
    "2. Make a scorer for the model using `f1_score`.\n",
    "3. Perform grid search on the classifier, using the parameters and the scorer.\n",
    "4. Fit the data to the new classifier.\n",
    "5. Plot the model and find the f1_score.\n",
    "6. If the model is not much better, try changing the ranges for the parameters and fit it again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The training F1 Score is 0.814814814815\n",
      "The testing F1 Score is 0.8\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztnX10XGd54H+P4o8ZCUoM+XBiSQSoqe2mIKeqCUTlhBapUepFBgKku2eL07BuljgyAaVNwUt7sm6X4EHGE7kBk7CJKQs0FCuumzpSSDjBBlM7aEI+pCROmkiyU5yEJIWRRo7wu3/MHXss3/m8d+Z+Pb9zdDS69+q+z1xp3ud9n08xxqAoiqJEjwavBVAURVG8QRWAoihKRFEFoCiKElFUASiKokQUVQCKoigRRRWAoihKRFEFoCiKElFUASiKokQUxwpARFpE5AERGRWRx0Rkg801IiJJETkkIj8TkYucjqsoiqI4Y54L95gFPmOM+amIvB54SESGjTGP513TDSy1vt4F3Gp9L8qZZ55lzj//AhdEVBRFiQajow+9aIw5u5xrHSsAY8zzwPPW61+KyCiwBMhXAD3ADpOtO7FfRM4UkfOs3y3I+edfwDe+cdCpiIqiKJGhvV2eK/daV30AInIBsBL4yZxTS4CJvJ8nrWOKoiiKR7imAETkdcA/AZ8yxvzn3NM2v2JbhU5E1onIQRE5+PLLL7glnqIoijIHVxSAiMwnO/l/0xjzPZtLJoGWvJ+bgSN29zLGbDfGtBtj2hctKsuMpSiKolSBG1FAAtwOjBpj+gtctgv4Uysa6GLg1VL2f0VRFKW2uBEFdAnw34FHRCRlHfss0ApgjPkKcA9wOXAImAKucmFcRVEUxQFuRAHtxd7Gn3+NAa51OpaiKIriHpoJrCiKElFUASiKokQUVQCKoigRRRWAoihKRFEFoCiKElHcCANVlLqQTqcZHh5ifPwwra1L6OzsoqmpyWuxFCWwqAJQAkEqNUJv7w0cP95GJrOcWGwf/f3bSCY309a20mvxFCWQqAJQfE86naa39wampjYBFwOQyQDsp7f3Bvbs2UVjY6OXIipKIFEfgOJ7hoeHOH68jdzkf5KLOX68jaGhIS/EUpTAowpA8T3j44fJZJbbnstkljE5ebjOEilKOFATUMgIo6O0tXUJsdg+y+xzKrHYGM3NHfUXSlFCgO4AQkQqNUJ3dw+JxD527IiTSOyju7uHVGrEa9Ec0dnZRUNDCtg/58x+GhpSdHV1eSGWogQe3QGEhDA7SpuamkgmN+dFAS0jFhujoSFFMrk5sO9LUbxGFUBIKMdRumbNGi9Ec4W2tpXs2bOLoaEhJicP09zcQVfXTTr5K4oDVAGEhCg4ShsbGwOtxBTFb6gPICRkHaWjtueyjtIldZZIURS/owogJKijVFGUSlETUEhQR6miKJXiigIQka8Dq4GjxpgLbc5fCtwN/Lt16HvGmJvcGFs5iTpKFUWpBLd2AHcAA8COItf80Biz2qXxlAKoo1RRlHJxxQdgjHkQ+IUb91IURVHqQz2dwO8WkYdF5F9F5LfrOK6iKIpiQ72cwD8F3myM+ZWIXA4MAkvtLhSRdcA6gMWLW+sknqIoSvSoyw7AGPOfxphfWa/vAeaLyFkFrt1ujGk3xrQvWnR2PcRTFEWJJHVRACKyWETEer3KGveleoytKIqi2ONWGOi3gEuBs0RkEvhrYD6AMeYrwBXA/xSRWWAauNIYY9wYW1EURakOVxSAMeZPSpwfIBsmqiiBJIx9FhRFM4EVpQTakF4JK6oAFKUIYe6zoChaDE5RiqAN6ZUwowpAUYoQhT4LSnRRE5CiFMFpQ3p1Hit+RncASlmk02kGB3eSTA4wOLiTdDrttUh1wUmfhVRqhO7uHhKJfezYESeR2Ed3dw+p1EhNZVaUchE/h+OvWNFuvvGNg16LEXlOj4IZPdFnIApRMKe+/1P7LBR6/+l0mu7unlOcx1n209i4UZ3HSs1ob5eHjDHt5VyrJiClKBoFU12fhXKcx2Et261mr+CgCkApSpQnsnwq7bMQVeex5kwEC1UASlGiOpE5xanzOIjobjF4qBNYKUp2Ihu1PZedyJbUWaJg4MR5HFQ0ZyJ46A5AKUpnZxf9/dvITmSnOjOzE5m2drajqamJZHJzQeex05Vwzs4+OT5Bc2uLL+zsft8tqm/idFQBKEWp9UQWZqpxHpdDKjVCX+8G2o4fZ3lmmr2xOAP9W0gkt3pqZ/ez2Ut9E/ZoGKhSFlNTU3kT2RK6urp08veAdDpNT3c3m6bSc/ZjsLGxiV179nj2d/Fr6Ktf5aoVGgaquE6lUTBKbRgeHqLt+HEbKzu0HT/uaVSWX3eLGslWGFUAihIgJscnWJ6Ztj23LDPN4cmJOkt0KrUyeznB774JL1EFoCgOqadzsbm1hb2xONgogbFYnI7mlpqMWwl+2y362TfhNa6EgYrI10XkqIg8WuC8iEhSRA6JyM9E5CI3xlUUr6l3vZ/Ozi5SDQ02waWQamgIZXipU6IYklsubu0A7iDb8nFHgfPdwFLr613ArdZ3RQksXiQ+NTU1kUhuPREFtCwzzVgsTqqhgURya6icmW7hV9+EH3CrJ/CDInJBkUt6gB1WI/j9InKmiJxnjHnejfEVxQu8ci62ta1k1549DA0NcXhygo7mFm4qEZUV9Rj4nG9i9+7d/PCH+wDo6PgfLF36do8l85Z6+QCWAPneqUnrmCoAJbB46VysxM6uMfBZnnzyCQYGtp94DiMjB9i27WuRew751EsBiM0x2wQEEVkHrANYvLi1ljIpiiOC4FzU+jxZ9DnYU69aQJNAfnhCM3DE7kJjzHZjTLsxpn3RorPrIpyiVEMQnItanyeLPgd76rUD2AWsF5Fvk3X+vqr2fyXoBMG5qDHwWfQ52OOKAhCRbwGXAmeJyCTw18B8AGPMV4B7gMuBQ8AUcJUb4yqK1/gx8SmfIJip6oE+B3u0FpCihJio1cEpRJSeg9YCUqrigQd28uCD/+y1GIrLrFz5On784w9hzBsxppkFC36LefMe842Zqh4EwVznBboDUAD48IdX8Nxz9o1flPDxD/+QYtmyd3otRt2JQlVb3QEoFaOTf7TYvfv/smzZl70Wo+74rU6R16gCUE7j+7fe6rUIJXnq0CG+OjDA244f582vvcZz8+fzdEMDf75+PUt/8zdDN64b/MXWrTw0NgbAr389W9Hv+rEDmeIcVQA+wS+p+g0i/MHv/V7dx62EdDrN333mM/z9zMxJd95rr2Wbotx6a82aong1rlssbW09oQAqwa8dyBTnaFN4H1DvipJBZ2h4uGRTlDCN6yXpdJq+3g1smkqTyExzNZDITLNpKnt8amrKaxEVB6gC8Jj8FPVMJgFcTSaTYGpqk3VcP2BzOTw+znK7gG5gWSbD4cnJUI3rJeV0IFOCiyoAj3GSop5Opxkc3EkyOcDg4E7S6XRNZfULS1pbGY3FbM+NxWIsaW4O1bhe4vcOZIozVAF4TLUp6lE2G3V1dnrSFMWrcb2kubWF0Vjc9txYLM4SH3QgU6pHFYDHZFPU7UMwsynqS047HnWzUVNTE5uTSTY2NtIXi3Eb0BeLsbGxkc3JZM0csV6N6yXagSzcaBSQx3R2dtHfv43sR+rUFPVsRcmbTvsdrxqR+ImVbW15TVEm6WhuLtkUJcjjeoV2IAs3qgA8ppoUda1smMWrpJ6oJROV04HML2HMSmWoAvABlVaU1MqG0SadTjM0PMzh8XGWtLbS1dlZ88m2mNLTjmPBJTQKIMgrkNNl7yy6ta7GbOQHchPXxPgkLa3NdZm4wsZIKsUNvb1WUlaGfbEY2/r72ZxMsrKtre7yaKetYBMKBRDkFUg1sgexsuFIKsWG3j6OH29jOrOceGwvW/oH2JpMeDJxBZF0Os0Nvb1smpo6qfYzGfYDN/T2epKJ7Dd/VJAXgl4QeAUQ5BWIE9n93ogkn3Q6zYbePtJ573Paep8bevs8+Rt5YUZxSjmZyPX2TfjJHxXkhaBXBF4B+G0FUglOZQ+KM3JoeLjq91mLidpvZpRy8WMmcj39UcVW90FeCHpJ4PMA/LQCqZQgy14JE+OTTBd4n9OZZUwUeJ8jqRQ93d3sSySI79jBvkSCnu5uRlKpqmXJN6MkMhmrtk2GTVNT3NDb6+scCj9mInd2dtHQkAKbTIGsP8qdPIFSiY/a9L06XFEAInKZiDwhIodE5Eab82tF5AURSVlfn3BjXKgukcovBFn2SmhpbSZe4H3GY2O0FEh2q8VEHeSCbn7MRM75oxobNxKL9QG3EYv10di40TV/VDmJj1FZTLmNYwUgImcA24BuYAXwJyKywubS7xhj2qyv25yOm6NeK5BaEGTZK6Grs7Pi91mridqPZpRy8Wsmcs4f1dfXwdq1M/T1dbBnzy7X7O7lrO6jsphyGzd8AKuAQ8aYZwBE5NtAD/C4C/cuSRAjYnIEWfZKaGpqYmsykRcFtIy49T63JhO277NWE/WS1lb2xWLYGa3HYjE6fFTQzc7/4ddM5Fr6o8pZ3V911dpAhkZ7jRsKYAmQXxJwEniXzXUfFpH3Ak8C1xtjXCsjGKSImLkEWfZKWNnWduJ9TkwepqXE+6zVRN3V2cm2/n6baSJrRrmpwK6r3lFDpRzVQXD+u0U5juaoLKbcxg0FIDbH5naa/2fgW8aYGRG5BrgT+APbm4msA9YBLF7cWrYQQYmIsSPIsldCJe+z2om6FDkzSm5yXZbJMBaLkWpoKGhGqXfUkB/j/b2k3MTHqCym3MQNBTAJ5NeEbQaO5F9gjHkp78evATcXupkxZjuwHWDFiva5ikSJCNVM1OVSiRnFi8nYj/H+XlLJ6j4qiym3cEMBHACWishbgMPAlcB/zb9ARM4zxjxv/fgBwN5bo4QKp2aTWtq7y50ovJiMg+yorhW6uq8NjhWAMWZWRNYD9wJnAF83xjwmIjcBB40xu4BeEfkAMAv8AljrdFzF37hlNvF6RefFZBwkR3U98fp/IYy4kgdgjLnHGPN2Y8zbjDF/ax37vDX5Y4z5K2PMbxtj3mmMeZ8xZsyNcRV/EuRkq7l4kXzlx3h/JZwEPhNY8R9BSLZKp9PsHBwkmRxg5+BgwX7KXkzGfo33V8JH4GsBKf7D7zbsSiqT1tIZXQy/xvsr4UIVgFKUahy5frZhV1OZ1KvJWG3eSq1RBaAUpFpHbq1i+N2gWGXSXx+/kJtvvpmz3/Sm05SdTsZKMXKVSifHJ2hubQlMHwJVAIotTuLfvTKblEPhyqQpTGY/r9wLrbOznpaIDmKvgiiTSo3Q17vBWihNszcWZ6B/C4nkVt/3IVAFoNjiNP7drzbsbGXSvZbZJ0eahfTSzywXz1qHPMq8DWqvgqiSTqfp693Apql03kJpmv1AX+8G32dtaxSQYosbjtyc2eTa9etZs2aNLz4I9pVJh1nJrOdRS2EKn40Kw8NDvo94K4YqAMUWPzYfcYNcZdKmxo3Erfr1C864k4s4Znt9PaOW/Bo+m06nGRzcSTI5wODgzoIhs1FkcnyC5Zlp23PLMtMcnnSt5mVNUAWg2BLmZKRcZdJc/fquy97B4z5Qdn4Mny3ViSvqNLe2MBqL254bi8VZ0txie84vhMYHUKxfqFI5fnbkukF+VE86nea/fP/7tlFLP4Wyo5acOm/PPvdcds2bx/TsLEuALiD3216Ez2qf3dJ0dnYx0L/FlxFv5RAKBZBKjeRVClxOLLaP/v5tJJObfe+F9zN+deTWghkzj0+zkJUIK8kwQowRDGLK+4g4dd6OpFJ8bWCA35qdJQ7sI9tmbzMwgzeTSTmduKIeGtvU1EQiufVEFNCyzDRjsTiphgYSya2+/6wEXgHoKqW2VBL/HtTwxaHhYUTayXAT+xniJ0xiaAa6iMvnS050TktGn/j96enTVpG9wIJ4nIQHuy7ts1sebW0r8xZKE3Q0twRmoRR4BaCrFH8Q5PDFk7kBjcCaU7oZTWeWMVFionMaMlvs9393/nzec911njzDcjpxRZkwmJ0D7wTWVYr3BD18MZsbYN+iIh4bo6VEQ3Gnzttiv3/ha6/xwtGjRX+/VnR2dtmEzMLJTlz+tm/XkrA4xwOvALKrFPsPb3aVUvzDqzjHr+GL5WKfGwDlTnROQ2b9GnKb68TV2LiRmBUyG4v10di4MdJ9dvPNzplMAriaTCbB1NQm67i/Fzz5BF4B6CrFe2oZvpgr2zyQTBYt2+zk/kPDw1z63otZMP8GYrFPAbcRj/XR1LiRrckEjY2NReVwGjLr55DbXCeuXMhsX18He/bsinRwRTlm56AQeB9AJf1CldpQq+qftfYrzL3/e2IxHjr+b7zvst+gvf1ky8FSctiFzD46fz4jxvDhK67AmOKtrcsNufXKya6F8E4lTGZnVxSAiFwGbCXbEvI2Y8wX5pxfCOwAfhd4CfiYMeZZN8YG7/uFhsEZ5AQn1T9zk9rE+CQtrc0nJrVaNmNPp9Ps3r2bW7Zs4UOzs/w5Vry9df+ND/6Az372xhMr/3LkyIXM3n777dz+zW9jzAW8Nvv7/OM/HuK73+2x7TWQT6mQ2yA72cNGmJzjjhWAiJxBNmS5E5gEDojILmPM43mXXQ28bIz5TRG5ErgZ+JjTsfPxapWiOQjVJ40Va8zy7LPP1qQZe24ivfC117hqdpZRoIdsvP1Km/tXEuFjjOGuu+7m2OyXKLfXQD6F/odrqQyVyuns7KK/fxvYLHmyZuebPJKsctzYAawCDhljngEQkW+T/UzlK4Ae4G+s198FBkRETKm9sc/RHISTVJo0VqoxywfXXO66X8F2IiX7Mb4B2EU2EDT//pX4N4r1GnASkuw0zFRxlzCZnd1QAEuA/IpHk8C7Cl1jjJkVkVeBNwEvujC+Z2gOwqlUsgsrNVm+8uqrHHbZr1B0IgWGgDVz7r+ktZW9ZcpRuNdAefkEhfBjjaCo47XZ2S3cUABic2zuyr6ca7IXiqwD1gEsXtzqTLIaEyZnUL0pNVm+4Q2/Yp8VGTN3tT4iwu/NzDCQTFbkDC06kQKHOd1vsXjxYvZb5pZS/g37XgNZsvkE1dmG/dxiM8qEwTnuRhjoJJBf8q4ZOFLoGhGZB7wB+IXdzYwx240x7caY9kWLznZBvNqhOQjVUyr56q1vvYDNySQbGxvpi8W4DeiLxbhx4UKMMRy45RbiO3awL5Ggp7ubkVSq5JjF4u0fAX40bx4bGxtP+C3S6TT/6y/+gk8CG4FsJDx8GvgM8L+/+MVTVnxO8wkK4ecwUT+hZasrxw0FcABYKiJvEZEFwJVkzan57AI+br2+Arg/6PZ/0ByESsmPpT82M4PICMWeXc6v0NHXx8zatazq7WVeQwN/l8lUlXFcdCKdN4+eT3+aXXv2nIiqyZmM/hvZf+AOsoXZ3gtcNG8eE+Pjp9zHrtfA3HyCasg52ecqw3xlFXXCkplbbxybgCyb/nrgXrJhoF83xjwmIjcBB40xu4DbgW+IyCGyK/8rnY7rB8LkDKo1c8MYD8RizAdiC29EpJ3pzDLi1rPLnyzzt9k7BwdpM6ZqZ2ixaKUv24RT5puMslWCTvLi7Cxb+rey9O1vP+X3cr0GhoaGmJg8TItLtuEoVWatFA3GqB5X8gCMMfcA98w59vm81xngI26M5TfC4gyqJcXCGDfG46y7bhU/P/pCycnSDWdoJRNpMdv7CDGOzX7INryzlG242oSuMNica4EGY1RP4DOB/YB+MItTNPrGGBYsWMB16689cbzQBOmWM7Tcv1exBLcRGoBrOH78SEUTjCZ0uY8GY1SPKgCl5lSyci82QTrJOK6GnMnoU9dcw4WzsJJZq1FMAzMkgcaKwjtrkdAV1B4MbhKmzNx6E/hicIr/KbfaZamy0iJSd2foyrY2Pnn99Tw07218hbXsp48Z9pDNHCivXHQOt6umjqRS9HR3sy+RqDgiKkxoMEb16A5AqTnlrtzLzXittzN09erVbNv2NV6bbT/tHVSS+l+pD6PY6l7LQ5xEgzGqRxWAchpumxXKrRVU7gRZb59LLrzzZN0i+4ilUlTiwyjlK3C7PESuoOHk+ATNrS2BK2iowRjVoQpAOQUD9HR3u+6kLCf6xs8Zr26Ed5a7E8qt7jdOTfEy2QzlSzIZVnNyde9meYhUauREU/PlmWn2xuIM9G8hkdwaqIKGGoxROaoAlFMwxtTMrFDqA1p0grTKPySTA6eUja4nTieYcndCQ8PDXDA7yyaynoblwD4gBVwwO8vQ0JBryjKdTtPXu4FNU+m8v/k0+4G+3g2RMiVFEXUCK6fhVWvHQhmvn43FmD5+BrfccoA7d8T5UmIv3d09gXR2zs1u7ujrOyX7GODZQ4d46tgxNgHZhoPZ75uAp44d49l//3fXykMMDw8Fup2n4gzdASinYFe1D+pXdXKuqWjVOefwk1u2k8nYl40OYpZnqZ3EK7/8Je/EXhG/E3j51Ver7sEwl8nxCZZnpm3PLctMc3hywvacEg5UASinUKhAUz1t8HPLPxgTrSzPM3/jNyhUB/cdwN6nn2bn4CBdnZ2OI6KaW1vYG4uDjRIYi8XpaG6x+S0lLKgJSDkNP1WdrFWNfT9zwdvexmMLFtieexh40+joiZj/J558kjVr1nDt+vWsWbOm4t1QZ2eXVhqNMLoD8Bi/9RMWETbG447MCm5Sqxr7fuaEM/zYsdOc4aNYnctccs43NTWRSG49EQW0LDPNWCxOqqGBRHJr4MxrSmWoAvAQP/YTFvBV1cmuzk629A8Qhv6r5WJn3/8Z2R6rm8ma6XaSDQ8999gxdu/ezUc/+tGqx2trW5n3N5+go7lFK41GBFUAHlGshO0113yK66//JKtXr/ZkN1CPeOpyk83cSsIKGvnO8F2Dgyx69FF2AU+QbbCdCw89a3aWbf39p5WlrhSNoY8mqgA8olgJ29nZNrZs2cW2bV/zdDdQKyqtiFmrGvt+JzcpG2DfoUOYTIYbyIaDnrIXmp2NXPkHxR1UAXhEsRK28DvMzs4wO/u7oWtoUW0NmyivUHM+ga+SXfm7Vf5BUTQKyCOK9ROGMWAJ+aGOYcHtiphRIOcT+N68eRRaMtQrT0MJF6oAPKJYCdts0n82/C5sDS3crGETJVa2tXHd9dfzyPz5tufzy2orSrk4UgAi8kYRGRaRp6zviwpc92sRSVlfcxvGR5JcCdvGxo3Mn/8pyBY+ADaSjfXImkGyDS3KqzcfBMrtDWBHflP5nYODpNPpWonpS1avXs2j8+drzL7iGk53ADcC3zfGLAW+b/1sx7Qxps36+oDDMUNDroTt9de/h3nz7gDOJxvlnXP6hq+hRbU1bLT5SeFaSbVsiOMW6XSawcGdJJMDDA7ujJzy9itOncA9wKXW6zuBHwB/6fCekaKxsZGPfvSjvP3tS62cgCOhbmhRTQ2bahzHYW2VWElTe7/gx3wXJYtTBXCuMeZ5AGPM8yJyToHrYiJyEJgFvmCMGXQ4buiIUkOLSiexSpufhL3xepAioorlu4Qtwi2IlFQAInIfsNjm1OcqGKfVGHNERN4K3C8ijxhjni4w3jpgHcDixYVKYoWTIH2wnVLJe63EcVzrVolh3VnUimL5LmEt5hckSvoAjDHvN8ZcaPN1N/BzETkPwPp+tMA9jljfnyFrJiq47zPGbDfGtBtj2hctOruKt6SEjUocx7UMM1U/ROUUy3cJW4RbEHFqAtoFfBz4gvX97rkXWJFBU8aYGRE5C7gE+KLDcZUqcbv4XD1WxOW2UoTahZnWYmdRy2dnd28vyOa77LNrXGZFuIWvmF+QcBoF9AWgU0SeAjqtnxGRdhG5zbpmOXBQRB4GHiDrA3jc4bhKFaRSI3R395BI7GPHjjiJxD66u3tIpUaqul+9VsSVRL84CTMthts7i1o+u0L3fvmVVxzfu1KK5buELcItiDjaARhjXgL+0Ob4QeAT1usfAb/jZBzFOaWccdXcr5a29rmU6ziuZLdQCW7uLGr57Irdu+fgwaru6YRcvsvJKKDwRrgFEa0FFBFKOeOyaRzlU2lkjhuU4zh2q1XiXNxqwg61fXbF7r2IAk66GhOlCLegoQogIpRyxlWKn0s61CJWvpqdxePPPMP4f/zHaceHf/xjFmYy7LEZZzaTYXj/fmLnFIqoLk6xe6ePH6/qnm4QpQi3IKEKICKUcsYVmMsL4uaKuBpKOVDdnnAq2VnMHDtG6x//MUdffrnoPb9c6MR99/H3993nSN6C97Y444z6f/T91v1O0WJwkaGUM65Sqi3p4AZehWPmdhYdfX3MrF1LR18fu/bsOS257DtDQyUnf6/52MfW13U8twMQFHfQHUBEKOWM+8QnLqr4frWwtZei3s7nuZSzs5iamTnxet68BaxZ82enXfPCCy/yox/u5SxjONPM8orM4yjZlpxnA4vMLC/LPF4U4T2/38HZZ59Vtox2987dZ+nSFVx11V+xcKF9pFQt0Gxg/6IKIEK47Yzzoi6NF85nJyxd+k5uvPFW23NTU1Mn+vCefc65bL/lFjZN5yk2A/sNbDzwc3btubOi55p/7yXNLXR5WC8orNnAYTBpqQKIGG7bxuvt3POz87lS8p/d4OBO2oxxTbH5yelaPADhbdx//wOMj0/6bhItNsGHpcCd+gCUQFGrRC+vmRyfYHlm2vbcssw0hycn6iyRexTufjcC/D8OHBDf+QWK+SzyTVqZTAK4mkwmwdTUJuv4lNfil40qAMX35DeCOTYzw4hI6JqiNLe2MBqL254bi8VZ0txSZ4ncwz4AIQ18BtjMa699GT9NoqUm+N27d5c0aQUFNQEpvmZuaecDsRgGuHHhQtpF6uZ8rjWdnV0M9G8pmmeQM0lMjk/Q3NriK3NJMewCEObNe4DZ2eX40S9Qymexd++PyGTeYfu7QStwpwpA8S3FIn42xuOsuu46Xjh6NBBNUUrR1NREIrmVvt4NVlTVNGOxOKmGBhLJrTz55BMnzi3PTLM3FmegfwuJ5NZA2JznBiA88cRZ/PjH/pxESyVNijxCLDYaigJ3qgAU31I04scYFixYwLXr6xvPXkva2lbmRVVN0NHcwk1dXRj0umpTAAAOaklEQVRj6OnuZtNUOk8RTrMf6OvdUPPQV7eY6/QeGfFnldBSSZMdHZeQSm0Hm/1atsDdTXWS1DnqA1B8S5gifsolN0leu/461qxZQ2NjI8PDQzXrceAVfq4SWkq21atXW4XsNhKL9QG3EYv10di4MXAF7nQHoPgWr8tN+IUwRgj5uUpoObKVyqkJSo6AKgDFt9SqtHPQaG5tYW8sDjZKYCwWpyOgEUJ+rhJajmyFci2ClCOgCkDxLV6Vm6iWXIG6H9x/v6v3LSdCKKj4KWFtLtXIFrSyF6oAFF/jRbmJasgPV52fZ7KamkpXfc98M8IHrriCjXfdRZsxp0UI+e1ZRJmglb1wpABE5CPA35Bt+7jK6gRmd91lwFbgDOA2Y8wXnIwbFry2E+aP72f8vEqE08NVvwL8g3Vu4rnnmJqaqniSPt2MMIrIAt78sTXMCCcihHTy9xelQkj9liPgdAfwKPAh4KuFLhCRM4BtZHsGTwIHRGRX1PsCe20nnDt+DlPzkcNHoXBVgEaoeNVXzIxw110byzIjeL24iCqlQkj9liPgKAzUGDNqjHmixGWrgEPGmGeMMceAbwM9TsYNOl7XErEbP4cxJlC1TPxAsXDVOMcrjtIpx4xQjCjV3k+n0wwO7iSZHGBwcCfpdPUmNzfwc3irHfXIA1gC5H8CJq1jkcXpB7x242epZ1x5rs5PMjnAzsFBzz/A1VCsQN00DRXX8XFiRvB6cVFP/KjociGkQckRKGkCEpH7gMU2pz5njLm7jDHE5lhBS4OIrAPWASxe3FrG7YOH13bCYuMDTNTJTjmSSrGht4/jx9uYziwnHtvLlv4BtiYTp3XZ8jOFwlUBpqDiVZ8TM0LQnJDV4udoGz+Ht86lpAIwxrzf4RiTQP4SqBk4UmS87cB2gBUr2kNpkvbaTlhsfICW5tpv0NLpNBt6+0jnfYCnrQ/wht4+34XLFWNuuOp/5D3Ylje/ueL30dnZRX//NqopNeD14qJe+F3R+T1wIUc9TEAHgKUi8hYRWQBcCeyqw7i+xWs7YeHxs9TDTjk0PByakrpwar/g2YtPvqfGxsodr07MCIVr7+cWF+GwvkZF0dUaRwpARD4oIpPAu4F/EZF7rePni8g9AMaYWWA9cC8wCvyjMeYxZ2IHG6/thHbj5xCRuqy8J8YnmS7wAZ7OLKubGcpNcqu+S9/3Psf3ypkR+vo6WLt2hr6+Dvbs2VUyQszrxUW9iIqiqzWOwkCNMTuBnTbHjwCX5/18D3CPk7HChtd2wrnj33FH9ridw6YWtLQ2E4/ttcw+pxKPjdHiohksl6E7MT5JS2szXZ2dgQiJrMaM4OcaO27ixEymnEQzgT3Eazth/vh33FHfsspdnZ1s6R+g1h/gsDiaK8HrxUU98IuiC2qTnhyqABRPaGpqYmsykTc5LyNufYC3JhOufIDD5GiuFK8XF/XAa0WXSo0EukkPqAJQPGRlW9uJD/DE5GFaXP4Al+NoDvskGXa8UnTpdJq+3g2Bb9KjCkDxlFp+gMPoaFb8QTlNeoKwuFAFoPgKNx229XQ0K9EiLE16tCWk4htGUim6u3v4UmIvd+6I86XEXrq7exhJpaq6X1dnZyRCIpX609zawmgsbntuLBavuPyHV+gOQCmL3Mr88Pg4S1pbXQ+lrIXDth6O5rChVUTLIyxNelQB1ICgh4bNJb/ZyfJMhn2xGNv6+9mcTLoWSlkrh22tHc1hwusS5UGiqamJRHLriSigoDbpUQXgMmEIDctnbrMTADIZ9gM39Pa6Fu1QS4dtFEIineLn4mp+pa1tZV63uolANulRBeAiYQkNy6dQsxO3ox3UYestfi+u5leCvrhQJ7CLlBMaFjSKNTtZlslweHLSlXHUYest9SqulmvgMpBM+qKBS9TRHYCLhCU0LJ8lra3si8Wwqx09FovR0dzsyjjqsPWWepQoD5t5NAzoDsBFwhIalk9XZyephgabdXk22sHNlXnOYTu3AmZYa/b4iVpXEc03jyYy01wNJDLTbJrKHg9Tp7IgoTsAF6lVaJiXoXlzm50sy2QYi8VINTSwOZl0fWUedJtqUKl1cbWwZM6GDVUALlKL0DA/hOblmp1kox0m6WhuDly0g1KaWhZXC6N5NAyoAnAZN0PD/BSapyvzaFCrv3Nzawt7Y3GwUQJjsTgdATSPhgH1AdSA3Ifo2vXXsWbNmqon6XJC8xQlCHR2dtXNl6SUj+4AfIz2PVXCQlgyZ8OGIwUgIh8B/gZYDqwyxhwscN2zwC+BXwOzxph2J+NGhXqE5ilKvQhD5mzYcLoDeBT4EPDVMq59nzHmRYfjRQrte6qEDfUl+QunTeFHAUTq1Uo8Wvil76miKOGkXj4AAwyJiAG+aozZXqdxA4/XfU8VRQkvJRWAiNwHLLY59TljzN1ljnOJMeaIiJwDDIvImDHmwQLjrQPWASxe3Frm7cONbpsVRakFJRWAMeb9Tgcxxhyxvh8VkZ3AKsBWAVi7g+0AK1a0G6djK4qXvPLKL0gmB7S5iuJLap4HICJNIvL63Gugi6zzWFFCz/PP/4odO+IkEvvo7u4hlRrxWiRFOYEjBSAiHxSRSeDdwL+IyL3W8fNF5B7rsnOBvSLyMPBvwL8YY/Y4GVdR/MyxmZm8n1qBq8lkEkxNbbIyu7XwmeIPnEYB7QR22hw/AlxuvX4GeKeTcRQlSDw+OlrgjDZXUfyFloJQFJd5+RcvFzynGdyKn1AFoCgus+iNiwqey2ZwL6mjNIpSGFUAiuIyK5bb12/S9paK39BicIriMgsWLsz7aRy4TTO4FV+iCkBRash5572OP/qjGVczuL3sEKeEC1UAilJDzjzzjaxff61r9/NDhzglPKgCUJSA4KcOcUo4UCewogQE7RCnuI0qAEUJCNohTnEbVQCKEhCyHeLss4w1v0CpBlUAihIQOju7aGhIgU1rdc0vUKpBncCKEhC0Q5ziNqoAFCVAaIc4xU1UAShKwNAOcYpbqA9AURQloqgCUBRFiSiqABRFUSKK05aQm0VkTER+JiI7ReTMAtddJiJPiMghEbnRyZiKoiiKOzjdAQwDFxpj3gE8CfzV3AtE5AxgG9ANrAD+RERWOBxXURRFcYgjBWCMGTLGzFo/7geabS5bBRwyxjxjjDkGfBvocTKuoiiK4hw3fQB/BvyrzfElwETez5PWMUVRFMVDSuYBiMh9wGKbU58zxtxtXfM5YBb4pt0tbI6ZIuOtA9YBLF7cWko8RVEUpUpKKgBjzPuLnReRjwOrgT80xthN7JNAS97PzcCRIuNtB7YDrFjRXlBRKIqiKM5wGgV0GfCXwAeMMVMFLjsALBWRt4jIAuBKYJeTcRVFURTniP2ivcxfFjkELAResg7tN8ZcIyLnA7cZYy63rrsc+DJwBvB1Y8zflnn/F4DnqhZQURQlerzZGHN2ORc6UgCKoihKcNFMYEVRlIiiCkBRFCWiqAJQFEWJKKoAFEVRIooqAEVRlIjiawXg92qjIvIREXlMRI6LSHuR654VkUdEJCUiB+slX4UyelaxVUTeKCLDIvKU9X1Rget+bT3DlIjUJZek1HMRkYUi8h3r/E9E5IJ6yFWBfGtF5IW85/aJespnyfB1ETkqIo8WOC8ikrTew89E5CIfynipiLya9xw/X2f5WkTkAREZtT7PG2yuqfw5GmN8+wV0AfOs1zcDN9tccwbwNPBWYAHwMLCiTvItB34L+AHQXuS6Z4GzPHqGJWX08hla438RuNF6faPd39k696s6P7uSzwX4JPAV6/WVwHd8Jt9aYMCL/708Gd4LXAQ8WuD85WTriAlwMfATH8p4KbDbw2d4HnCR9fr1ZKsvz/1bV/wcfb0DMD6vNmqMGTXGPFGPsaqlTBm9rtjaA9xpvb4T8EvD23KeS77s3wX+UETs6l95JZ/nGGMeBH5R5JIeYIfJsh84U0TOq490WcqQ0VOMMc8bY35qvf4lMMrpRTUrfo6+VgBzCHK1UQMMichDVrE7v+H1MzzXGPM8ZP/RgXMKXBcTkYMisl9E6qEkynkuJ66xFiuvAm+qg2ynjG1R6O/2Ycsk8F0RabE57zVe//+Vy7tF5GER+VcR+W2vhLDMjCuBn8w5VfFzLFkMrtbUu9poLeQrg0uMMUdE5BxgWETGrBWHX2Ss6TOE4jJWcJtW6zm+FbhfRB4xxjztjoS2lPNcav7silDO2P8MfMsYMyMi15DdrfxBzSWrDC+fYbn8lGyJhV9ZpW0GgaX1FkJEXgf8E/ApY8x/zj1t8ytFn6PnCsDUudqo2/KVeY8j1vejIrKT7NbdNQXggow1fYZQXEYR+bmInGeMed7ash4tcI/cc3xGRH5AdhVUSwVQznPJXTMpIvOAN1A/U0JJ+YwxL+X9+DWyvjS/UfP/P6fkT7bGmHtE5O9F5CxjzIv1kkFE5pOd/L9pjPmezSUVP0dfm4AkBNVGRaRJRF6fe03WsW0baeAhXj/DXcDHrdcfB07btYjIIhFZaL0+C7gEeLzGcpXzXPJlvwK4v8BCxRP55tiAP0DWduw3dgF/akWxXAy8mjMJ+gURWZzz7YjIKrJz50vFf8vV8QW4HRg1xvQXuKzy5+iVV7tMz/chsjatlPWVi7Y4H7hnjvf7SbKrwc/VUb4PktW6M8DPgXvnykc2QuNh6+uxespXroxePkNr7DcB3weesr6/0TreTraqLMB7gEes5/gIcHWdZDvtuQA3kV2UAMSAu6z/1X8D3lrnZ1dKvv9j/d89DDwALKunfJYM3wKeB16z/hevBq4BrrHOC9m+4U9bf9uCEXUeyrg+7znuB95TZ/k6yJpzfpY3H17u9DlqNVBFUZSI4msTkKIoilI7VAEoiqJEFFUAiqIoEUUVgKIoSkRRBaAoihJRVAEoiqJEFFUAiqIoEUUVgKIoSkT5/8+skkpj8DHIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fb074257fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=4,\n",
       "            max_features=None, max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=2, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, presort=False, random_state=42,\n",
       "            splitter='best')"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import make_scorer\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "clf = DecisionTreeClassifier(random_state=42)\n",
    "\n",
    "# TODO: Create the parameters list you wish to tune.\n",
    "parameters = {'max_depth':[2,4,6,8,10],'min_samples_leaf':[2,4,6,8,10], 'min_samples_split':[2,4,6,8,10]}\n",
    "\n",
    "# TODO: Make an fbeta_score scoring object.\n",
    "scorer = make_scorer(f1_score)\n",
    "\n",
    "# TODO: Perform grid search on the classifier using 'scorer' as the scoring method.\n",
    "grid_obj = GridSearchCV(clf, parameters, scoring=scorer)\n",
    "\n",
    "# TODO: Fit the grid search object to the training data and find the optimal parameters.\n",
    "grid_fit = grid_obj.fit(X_train, y_train)\n",
    "\n",
    "# Get the estimator.\n",
    "best_clf = grid_fit.best_estimator_\n",
    "\n",
    "# Fit the new model.\n",
    "best_clf.fit(X_train, y_train)\n",
    "\n",
    "# Make predictions using the new model.\n",
    "best_train_predictions = best_clf.predict(X_train)\n",
    "best_test_predictions = best_clf.predict(X_test)\n",
    "\n",
    "# Calculate the f1_score of the new model.\n",
    "print('The training F1 Score is', f1_score(best_train_predictions, y_train))\n",
    "print('The testing F1 Score is', f1_score(best_test_predictions, y_test))\n",
    "\n",
    "# Plot the new model.\n",
    "plot_model(X, y, best_clf)\n",
    "\n",
    "# Let's also explore what parameters ended up being used in the new model.\n",
    "best_clf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Conclusion\n",
    "Note that by using GridSearch we improved the F1 Score from 0.7 to 0.8 (and we lost some training score, but this is ok). Also, if you look at the plot, the second model has a much simpler boundary, which implies that it's less likely to overfit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}