Digit_Recognizer_SunnyBhuee.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://www.linkedin.com/in/sunnybhuee/\"><b>LinkedIn/SunnyBhuee</b></a>\n",
    "\n",
    "<a href=\"https://github.com/SunnyBhuee\"><b>github/SunnyBhuee</b></a>\n",
    "<HR>\n",
    " # Digit Recognizer Project\n",
    " <HR>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**MNIST** is the classic dataset of handwritten images has served as the basis for benchmarking classification algorithms. \n",
    "\n",
    "**Problem:** Our goal is to correctly identify digits from a dataset of tens of thousands of handwritten images. \n",
    "\n",
    "**Data:** The data files ***train.csv*** and ***test.csv*** contain gray-scale images of hand-drawn digits, from zero through nine. Each image is 28 pixels in height and 28 pixels in width, for a total of 784 pixels in total. Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and 255, inclusive.\n",
    "\n",
    "**Approach:** We use the train dataset to test two approaches for this dataset i.e. ***K-Nearest Neighbors Classifier*** and ***Decision Tree Classifier***. Then the better of the two approaches is used to classify the images in the test dataset.\n",
    "\n",
    "**Data Source** Kaggle.com\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".\\input\\test.csv\n",
      ".\\input\\train.csv\n",
      "['.\\\\input\\\\train.csv', '.\\\\input\\\\test.csv']\n"
     ]
    }
   ],
   "source": [
    "from PIL import Image\n",
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
    "from sklearn.model_selection import train_test_split\n",
    "import csv\n",
    "import random\n",
    "import matplotlib.pyplot as plt\n",
    "from datetime import datetime\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "\n",
    "files = []\n",
    "import os\n",
    "for dirname, _, filenames in os.walk('.\\\\input'):\n",
    "    for filename in filenames:\n",
    "        print(os.path.join(dirname, filename))\n",
    "        files.append(os.path.join(dirname, filename))\n",
    "\n",
    "files = sorted(files, reverse=True)\n",
    "\n",
    "print(files)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def shape_printer(numpy_arr):\n",
    "    assert type(numpy_arr) is str, 'Ensure Input is a String name of the Numpy Array'\n",
    "    print('Shape of {} is {}'.format(numpy_arr,eval(numpy_arr).shape))\n",
    "    \n",
    "def img_visualizer(np_array):\n",
    "    assert type(np_array) is np.ndarray, 'Ensure Input is a Numpy Array'\n",
    "    return Image.fromarray((np_array.reshape((28,28))* 255).astype(np.uint8), \"L\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of X is (42000, 784)\n",
      "Shape of y is (42000,)\n",
      "Shape of test_data is (28000, 784)\n"
     ]
    }
   ],
   "source": [
    "X = pd.read_csv(files[0]).iloc[:,1:].values\n",
    "y = pd.read_csv(files[0]).iloc[:,0].values\n",
    "test_data = pd.read_csv(files[1])\n",
    "\n",
    "shape_printer('X')\n",
    "shape_printer('y')\n",
    "shape_printer('test_data')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Label of image at Index 25649 is 1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAAcklEQVR4nGNgGHBwm+cBEo8JVXLJ\n9+u4dYoxbcctySTxFbexDCbceCRRzUHl/v+PR5KRkRY6HzBw4pacyWDHgBOosT/ErZNRQR6n5O3b\nKI5FlVzLqIXbSi+mazjlXvMwowogG3vmqzFuUx+Iz8MtSS8AAEKNFrEEXxufAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<PIL.Image.Image image mode=L size=28x28 at 0x26B88C583C8>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Visualizing and checking label of a random image\n",
    "index_of_img = random.randrange(X.shape[0])\n",
    "print('Label of image at Index {} is {}'.format(index_of_img,y[index_of_img]))\n",
    "img_visualizer(X[index_of_img])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Train and Test sets\n",
    "X_tr, X_test, y_tr, y_test = train_test_split(X, y, \n",
    "                                              test_size=0.2, \n",
    "                                              random_state=123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of X_tr is (33600, 784)\n",
      "Shape of y_tr is (33600,)\n",
      "Shape of X_test is (8400, 784)\n",
      "Shape of y_test is (8400,)\n"
     ]
    }
   ],
   "source": [
    "shape_printer('X_tr')\n",
    "shape_printer('y_tr')\n",
    "shape_printer('X_test')\n",
    "shape_printer('y_test')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------------------------------------------------------------------\n",
      "Running k = 1\n",
      "Time taken for k = 1: from 2021-01-10 18:11:45.018447 to 2021-01-10 18:17:21.685051 = 0:05:36.666604\n",
      "----------------------------------------------------------------------------------------------------\n",
      "Running k = 2\n",
      "Time taken for k = 2: from 2021-01-10 18:17:21.685051 to 2021-01-10 18:23:01.701862 = 0:05:40.016811\n",
      "----------------------------------------------------------------------------------------------------\n",
      "Running k = 3\n",
      "Time taken for k = 3: from 2021-01-10 18:23:01.702861 to 2021-01-10 18:28:44.524539 = 0:05:42.821678\n"
     ]
    }
   ],
   "source": [
    "\n",
    "\n",
    "accuracy_list = []\n",
    "k_list = []\n",
    "for k in range(1,4):\n",
    "    startTime = datetime.now()\n",
    "    print('-'*100)\n",
    "    print('Running k = {}'.format(k))\n",
    "    knn = KNeighborsClassifier(n_neighbors=k).fit(X_tr, y_tr)\n",
    "    accuracy_list.append(knn.score(X_test, y_test))\n",
    "    k_list.append(k)\n",
    "    print('Time taken for k = {}: from {} to {} = {}'.format(k,\n",
    "                                                             startTime,\n",
    "                                                             datetime.now(),                                                              \n",
    "                                                             datetime.now() - startTime))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl4FFXW+PHvIWENaxbWQEJkDRCy\n4jaKoo6oo6iIgqAw4qAo4+4og6O+zPi64K64oKAw7Coi4woijo6Omg0IBAIhYQkBkrAEkhCy3d8f\nXflNT95gOiTp6k6fz/PwUF1Vt+tUUfTpulV9rhhjUEoppVrYHYBSSinPoAlBKaUUoAlBKaWURROC\nUkopQBOCUkopiyYEpZRSgCYEpZRSFk0ISimlAE0ISimlLP52B1AfwcHBJjw83O4wlFLKqyQnJxcY\nY0LqWs+rEkJ4eDhJSUl2h6GUUl5FRPa4sp52GSmllAI0ISillLJoQlBKKQVoQlBKKWXRhKCUUgrQ\nhKCUUsqiCUEppRSgCUEppTza4aJTzP5HOifLKpt8W5oQlFLKQ1VWGe5ZnsqSn/ew50hxk2/Pq36p\nrJRSvuTlr3fwQ+ZhnhsbxaDuHZt8e3qFoJRSHmjD9jxe+yaTG+NDuTGht1u26VJCEJHRIpIhIpki\n8mgty8NEZL2IbBaRb0Uk1GlZHxFZKyLbRCRdRMKt+d+LyEbrT66IrG6snVJKKW+270gJ963YSGSP\njsweM9Rt260zIYiIHzAXuAKIBCaISGSN1Z4HFhljooDZwNNOyxYBc4wxg4ERQB6AMeYCY0y0MSYa\n+DewqqE7o5RS3u5URSV3L02hyhjenBRLm5Z+btu2K1cII4BMY0yWMaYMWA6MqbFOJLDemt5QvdxK\nHP7GmHUAxpgiY0yJc0MR6QCMAvQKQSnl82b/I53NOYW8MG44YUEBbt22KwmhF7DP6XWONc/ZJmCs\nNX0d0EFEgoABwDERWSUiqSIyx7ricHYdsN4Yc7z+4SulVPPxcWoOS37eyx0jI/jtkO5u374rCUFq\nmWdqvH4IGCkiqcBIYD9QgeMppgus5QlABDClRtsJwLLTblxkmogkiUhSfn6+C+EqpZT32X7wODNX\npXF230Ae/u1AW2JwJSHkAM63uEOBXOcVjDG5xpjrjTExwCxrXqHVNtXqbqrA0S0UW93OuooYAXx2\nuo0bY+YZY+KNMfEhIXUO+KOUUl7nRGk50xen0KFNS167OQZ/P3seAHVlq4lAfxHpKyKtgPHAGucV\nRCRYRKrfayawwKltFxGp/iQfBaQ7NR0HfGqMKT3THVBKKW9mjOFPH25m75ESXp8QQ9cObWyLpc6E\nYH2znwF8BWwDVhpjtorIbBG5xlrtIiBDRHYA3YCnrLaVOLqL1otIGo7up3ec3n48v9JdpJRSzd38\nf2XzxZaDPDJ6IGdHBNkaixhT83aA54qPjzc6prJSqrlI2n2E8fN+4pLBXXlrUhwitd2ybTgRSTbG\nxNe1nv5SWSmlbFBQdIq7l6YQ2qUtc8YNb7JkUB+aEJRSys0qqwz3LEvlWEk5b0yMo2OblnaHBGhx\nO6WUcrsX12Xw467DzLkhisieTV+0zlV6haCUUm60ftsh5m7YxfiE3oyLd0/ROldpQlBKKTfZd6SE\n+1dsZEjPjjx5zRC7w/k/fCIhlFdWUVhSbncYSikfVlpeyfQlyQC8OTHOrUXrXNXsE4Ixhjv/nszU\nhYmcqmj6IeiUUqo2//OPdLbsP86LN0bTJ6id3eHUqtknBBHhutheJO05yiMfbsabfnehlGoePkzO\nYdkve5l+0VlcGtnN7nBOyyeeMvpdVE92FxTz/Nod9A1uz72X9rc7JKWUj9h24DizPk7j3IggHrxs\ngN3h/CqfSAgAd1/cj6yCYl76egfhwe0YE12zgrdSSjWu46XlTF+cTKe2LXl1gn1F61zl2dE1IhHh\n6euHMSI8kIc/3EzyniN2h6SUasaMMfzpg83sO3qSuRNjCenQ2u6Q6uQzCQGgtb8fb98SR89ObZi2\nKJl9R0rqbqSUUmfg3e+z+XLrQWZeMYiE8EC7w3GJTyUEgC4BrZg/JYHyyipuez+R46X6OKpSqnH9\nkn2EZ77czhVDuzP1N33tDsdlPpcQAM4Kac9bt8SRXVDM3UtSqKissjskpVQzkXeilBlLU+gT2I7n\nbojyiKJ1rvLJhABw3lnB/O91w/h+ZwFPrNmqj6MqpRqsorKKe5alcry0nDcnxdLBQ4rWucpnnjKq\nzY0JvckqKOatf+4iIqS9V13aKaU8zwvrdvBT1hFeGDecQd09p2idq3w6IQD86fKB7C4o5m+fpRMW\n2M6jfzSilPJc69IP8ea3u5gwog9j40LtDueMuNRlJCKjRSRDRDJF5NFaloeJyHoR2Swi34pIqNOy\nPiKyVkS2iUi6iIRb80VEnhKRHdayexprp+qjRQvhpZuiGdarE/csT2VrbqEdYSilvNjewyU8sHIj\nQ3t15ImrI+0O54zVmRBExA+YC1wBRAITRKTmHj8PLDLGRAGzgaedli0C5hhjBgMjgDxr/hSgNzDI\nWra8AfvRIG1b+fHurfF0atuS2xcmceh4qV2hKKW8THXRuhYiHlu0zlWuXCGMADKNMVnGmDIcH9xj\naqwTCay3pjdUL7cSh78xZh2AMabIGFP98P90YLYxpspaloeNunZsw/zJCRw/Wc7tC5MoKauwMxyl\nlJd44pOtbM09zks3Dad3oGcWrXOVKwmhF7DP6XWONc/ZJmCsNX0d0EFEgoABwDERWSUiqSIyx7ri\nADgLuElEkkTkCxGxvcBQZM+OvDohhq25hdy/YiNVVfrkkVLq9FYm7WNF0j7uvvgsRg3y/vuPriSE\n2h6irflJ+RAwUkRSgZHAfqACx03rC6zlCUAEjq4igNZAqTEmHngHWFDrxkWmWUkjKT8/34VwG+aS\nwd147KpIvtp6iGe/2t7k21NKeaetuYX8ZfUWzjsriAcuG2h3OI3ClYSQg6Ovv1ookOu8gjEm1xhz\nvTEmBphlzSu02qZa3U0VwGog1ul9P7KmPwaiatu4MWaeMSbeGBMfEhLi4m41zO/PD2fSOX14+59Z\nrEjc65ZtKqW8R+HJcu5akkLndo6idX4tvOfHZ7/GlYSQCPQXkb4i0goYD6xxXkFEgkWk+r1m8p9v\n+4lAFxGp/iQfBaRb06ut1+C4qthxZrvQ+ESEJ68ewgX9g5n18RZ+zCywOySllIcwxvDwB5vYf/Qk\nb0yMJbi95xetc1WdCcH6Zj8D+ArYBqw0xmwVkdkico212kVAhojsALoBT1ltK3F0F60XkTQc3U/v\nWG2eAcZa858Gbm+0vWoE/n4tmDsxlr7BAdy5OJld+UV2h6SU8gDzvstibfohZl45mLgw7yha5yrx\nppIN8fHxJikpya3b3HekhGvn/kD7Nv58fNf5BAa0cuv2lVKe4+esw9z87s+MHtKd12+O8Zo6RSKS\nbN2v/VU+W8vIVb0D2zHv1ngOFJZy59+TdVxmpXxU3vFSZixLJSywHc+MHeY1yaA+NCG4IC6sC8+P\nG84vu48w86M0LYSnlI+pqKxixrJUikoreHNSnNcVrXOVz9cyctU1wx3jMr+4bgcRIQHMGGX7zyaU\nUm4yZ20Gv2Qf4aWbhjOwewe7w2kymhDq4Y+j+pFdUMzza3cQHhzA76J62h2SUqqJfbX1IG//M4uJ\nZ/fhuhjvLFrnKu0yqgcR4Zmxw0gI78IDKzeRsveo3SEppZrQ7oJiHlq5iajQTjzuxUXrXKUJoZ4c\n4zLH071jG6YtStJxmZVqphxF61Jo0UKYe3Msrf29t2idqzQhnIHAgFYsmJLAqYoqpi7UcZmVao7+\nsnoL2w4c5+Wbor2+aJ2rNCGcoX5d2/PWpDiy8ouZsTRVx2VWqhlZkbiXD5Jz+OOoflw8qKvd4biN\nJoQGOL9fMH+9dijf7chn9qfpdTdQSnm8LfsL+csnW/lNv2Duu3SA3eG4lT5l1EATRvQhu6CYed9l\nEREcwJTzdVxmpbxVddG6oIBWvDI+utkUrXOVJoRG8MjoQewuKGb2p+n0CWrXLOqiK+VrqqoMD67c\nRO6xk6y441yCmlHROldpl1Ej8GshvDw+msieHfnj0lS2HThud0hKqXp6+7ssvt52iFlXDSYurIvd\n4dhCE0IjadfKn3dvTaBDm5ZMfT+RPB2XWSmv8e9dh5nz1XauiurBlPPC7Q7HNpoQGlH3Tm14d3I8\nR0vK+cOiJE6WaSE8pTxd3vFS/rgslb7BATw7NqpZFq1zlSaERja0VydenRDD5v2FPLBSx2VWypOV\nV1Zx99IUik85ita1b+3bt1U1ITSByyK7MevKwXyx5SDPr82wOxyl1Gk89+V2Encf5ZmxwxjQrfkW\nrXOVb6fDJjT1N33ZlV/MG9/uIjw4gBvje9fdSCnlNl9uOcA732dzyzlhjInuZXc4HsGlKwQRGS0i\nGSKSKSKP1rI8TETWi8hmEflWREKdlvURkbUisk1E0kUk3Jr/vohki8hG6090Y+2UJxARZo8Zwm/6\nBTPr4zT+veuw3SEppSzZBcU8/MFmhvfuzGO/G2x3OB6jzoQgIn7AXOAKIBKYICI1y/49DywyxkQB\ns3GMkVxtETDHGDMYGAHkOS172BgTbf3Z2ID98EgtrXGZw4Ic4zJn6bjMStnuZFkl0xcn4+cnzL05\nxieK1rnKlSuEEUCmMSbLGFMGLAfG1FgnElhvTW+oXm4lDn9jzDoAY0yRMcanyoN2atuSBZMT8Gsh\nTF2YxNHiMrtDUspnGWN4bPUWMg6d4OWbognt4htF61zlSkLoBexzep1jzXO2CRhrTV8HdBCRIGAA\ncExEVolIqojMsa44qj1ldTO9JCLN9meBfYLaMe+WOPYfPcmdi5Mpq9BCeErZYXniPj5KyeGPo/pz\n0UDfKVrnKlcSQm0P5dZ8lvIhYKSIpAIjgf1ABY6b1hdYyxOACGCK1WYmMMiaHwg8UuvGRaaJSJKI\nJOXn57sQrmeKDw9kzrgofs4+wsxVOi6zUu62ZX8hT6zZygX9g7n3Eh0CtzauJIQcwPkRmVAg13kF\nY0yuMeZ6Y0wMMMuaV2i1TbW6myqA1UCstfyAcTgFvIeja+r/MMbMM8bEG2PiQ0JC6rl7nmVMdC/u\nvaQ/H6Xk8Ma3u+wORymfUVhSzp2LkwkOaMUr42N8rmidq1xJCIlAfxHpKyKtgPHAGucVRCRYRKrf\nayawwKltFxGp/iQfBaRbbXpYfwtwLbClITviLe67tD/XDO/JnK8y+DztgN3hKNXsVVUZHli5kUPH\nS5k7MZbAgFZ2h+Sx6kwI1jf7GcBXwDZgpTFmq4jMFpFrrNUuAjJEZAfQDXjKaluJo7tovYik4eh+\nesdqs8SalwYEA39rtL3yYCLCczdEERfWhftXbGTjvmN2h6RUs/bmP3exfnsej10VSUwf3yxa5yrx\npr7s+Ph4k5SUZHcYjeJw0SmufeMHTpZV8cmM8+nVua3dISnV7Py4q4BJ7/7MVVE9eXV8tM/WKRKR\nZGNMfF3raekKmwS1b82CyQmcKq9k6vuJnNBxmZVqVAcLS7lnWSoRIe155vphPpsM6kMTgo36d+vA\nG5Ni2ZlXxD3LdFxmpRpLeWUVM5amUFJWyVuTYgnw8aJ1rtKEYLML+ocwe8wQNmTk87fPttkdjlLN\nwjNfbCdpz1GeGRtFv65atM5VmjY9wMSzw8jOL+bdf2UTERLAreeG2x2SUl7r87QDzP9XNpPPDeOa\n4T3tDseraELwEDOvHMzuwyU8uWYrfQLb6a8olToDWflF/OnDzUT37sysq2qWXFN10S4jD+HXQnhl\nfDSDundkxtJUMg6esDskpbxKSVkF0xen0NJPmDsxllb++vFWX3rEPEhAa3/mT4knoLUft72fSN4J\nHZdZKVcYY3js4y3syDvBK+Nj9DHuM6QJwcP06NSWd29N4EhxGdMWJVNaruMyK1WXpb/sZVXqfu69\npD8XDvDuEjd20oTggYaFduLl8dFsyjnGgys36bjMSv2KzTnH+J816YwcEMI9o7RoXUNoQvBQlw/p\nzqOjB/FZ2gFeXLfD7nCU8kjHSsqYvjiFkA6tefmmaFpo0boG0aeMPNi0CyPILijm9Q2Z9A0OYGxc\naN2NlPIRVVWG+1dsJO9EKR/ceR5dtGhdg+kVggcTEf567VDOOyuIR1dt5ucsHZdZqWpvfJvJhox8\nHv9dJNG9O9sdTrOgCcHDtfRrwZsT4+gd2I47Fiezu6DY7pCUst2/dhbwwrodjInuyaRzwuwOp9nQ\nhOAFOrVryXtTEhDgtvcTOVai4zIr33Wg8CT3LE+lX0h7ntaidY1KE4KXCAsK4O1b4sk5epLpi1N0\nXGblk8oqqrh7SQqnyit5c1Ic7VrpbdDGpAnBi4zoG8izNwzj31mHeWy1jsusfM/TX2wjZe8xnr0h\nin5d29sdTrOj6dXLXBcTSnZ+Ma9+k0lESHvuHHmW3SEp5Rafbs7lvR92M+W8cH4XpUXrmoJLVwgi\nMlpEMkQkU0QerWV5mIisF5HNIvKtiIQ6LesjImtFZJuIpItIeI22r4lIUUN3xJfcf9kAfhfVg2e+\n2M6XW3RcZtX8ZeYV8ciHm4nt05k/XznY7nCarToTgoj4AXOBK4BIYIKI1Cwj+DywyBgTBcwGnnZa\ntgiYY4wZDIwA8pzeOx7Q58XqSUR4ftxwYvp05r4VG9mco+Myq+arpKyCu5Yk07qlnxata2KuHNkR\nQKYxJssYUwYsB8bUWCcSWG9Nb6hebiUOf2PMOgBjTJExpsRa5gfMAf7U4L3wQW1a+jHvlniCAloz\ndWESucdO2h2SUo3OGMOfV6WxM6+IV8ZH06OTFq1rSq4khF7APqfXOdY8Z5uAsdb0dUAHEQkCBgDH\nRGSViKSKyBwrEQDMANYYY7TP4wyFdGjNe79PoLSskqkLkyg6VWF3SEo1qsU/72X1xlweuHQAF/TX\nonVNzZWEUNtDvjUfb3kIGCkiqcBIYD9QgeOm9QXW8gQgApgiIj2BccBrdW5cZJqIJIlIUn5+vgvh\n+pYB3Trw+sRYdhw6wT3LUqnUQniqmdi07xh//Uc6Fw8M4e6L+9kdjk9wJSHkAL2dXocCuc4rGGNy\njTHXG2NigFnWvEKrbarV3VQBrAZigRigH5ApIruBdiKSWdvGjTHzjDHxxpj4kBD9hlCbkQNCePKa\nIXyzPY+ndFxm1QwcLS7jriWOonUvadE6t3HlsdNEoL+I9MXxzX88cLPzCiISDBwxxlQBM4EFTm27\niEiIMSYfGAUkGWM+A7o7tS8yxuhXgAa45ZwwsvKLWPBDNn1DArhFf86vvFRVleG+FRvJP3GKD6ef\nS+d2WrTOXeq8QrC+2c8AvgK2ASuNMVtFZLaIXGOtdhGQISI7gG7AU1bbShzdRetFJA1H99M7jb4X\nCoDHropk1KCuPLlmK//cod1ryju99k0m/9yRz+NXRxIVqg8hupN4069d4+PjTVJSkt1heLSiUxXc\n8OaP7D96ko/uOo8B3TrYHZJSLvtuRz6T3/uFa6N78eKNw7VOUSMRkWRjTHxd6+kDvc1M+9b+LJiS\nQJtWjnGZC4pO2R2SUi7JPXaSe5en0r9re566bqgmAxtoQmiGenZuy7u3xlNQdIo/LErScZmVxyur\nqOKuJSmUVxotWmcjTQjN1PDenXn5pmhS9x7j4Q83ayE85dH+9/NtbNx3jOduiOKsEC1aZxdNCM3Y\n6KE9eGT0IP6xKZeXvt5pdzhK1WrNplze/3E3t53flyuH9bA7HJ+m12XN3J0jI8guKOLV9TvpG9yO\n62J0XGblOTLzTvDoR5uJC+vCzCsH2R2Oz9MrhGZORPjbtcM4JyKQRz5MI3H3EbtDUgqA4lMV3Lk4\nhbYt/Zh7cywt/fTjyG76L+ADWvm34K1JcYR2acu0RUnsOazjMit7GWOYuSqNrPwiXp0QQ/dObewO\nSaEJwWd0bteK+VMSMDjGZS4sKbc7JOXD/v7THtZsyuXB3w7k/H7BdoejLJoQfEjf4ADenhTH3iMl\nTF+STHmljsus3C9171H++mk6lwzqynQd8c+jaELwMWdHBPH09VH8uOswf1m9RR9HVW51pLiMu5ek\n0K1jG168UYvWeRp9ysgH3RAXSnZBEXM37CIiJIBpF+q3NNX0KqsM9y5PpaCojI+mn0endi3tDknV\noAnBRz142UB2F5Tw9BfbCQsK4PIh3etupFQDvLp+J9/vLOB/rxvGsNBOdoejaqFdRj6qRQvhhRuH\nExXamfuWb2TL/kK7Q1LN2LcZebz6zU6uj+3FhBG9626gbKEJwYe1aenHO7fGERjQiqkLEzlQqOMy\nq8a3/9hJ7luxkYHdOvDUtcO0aJ0H04Tg47p2aMP8KfEUn6pk6vtJFOu4zKoRnaqo5K4lKVRUGt6Y\nGEvbVn51N1K20YSgGNS9I6/dHMP2g8e5d/lGHZdZNZqnPtvGpn3HeH5cFBFatM7jaUJQAFw8sCtP\nXD2Er7cd4unPdVxm1XCfbNzPon/v4fbf9GX0UC1a5w1cSggiMlpEMkQkU0QerWV5mIisF5HNIvKt\niIQ6LesjImtFZJuIpItIuDV/vohsstp8KCL69cFmk88LZ/K5Ybz7r2yW/rzX7nCUF9t56ASPfpRG\nQngXHrlCi9Z5izoTgoj4AXOBK4BIYIKIRNZY7XlgkTEmCpgNPO20bBEwxxgzGBgB5Fnz7zfGDLfa\n7MUxbrOy2V9+F8lFA0P4yydb+NfOArvDUV6o6FQFdy5OJqC1H69r0Tqv4sq/1Agg0xiTZYwpA5YD\nY2qsEwmst6Y3VC+3Eoe/MWYdgDGmyBhTYk0ft9YRoC2gHdcewN+vBa9NiKFfSHumL0kmM++E3SEp\nL2KM4dGPNpNdUMyrE2Lo1lGL1nkTVxJCL2Cf0+sca56zTcBYa/o6oIOIBAEDgGMiskpEUkVkjnXF\nAYCIvAccBAYBr53hPqhG1qFNS+ZPiae1vx+/fz+Rwzous3LR+z/u5tPNB3jo8oGcd5YWrfM2riSE\n2h4arvlt/iFgpIikAiOB/UAFjl9CX2AtTwAigCn//02M+T3QE9gG3FTrxkWmiUiSiCTl5+e7EK5q\nDKFd2vHu5Hjyjp9i2t+TdVxmVafkPUd56rNtXDq4K3dqORSv5EpCyAGcf1oYCuQ6r2CMyTXGXG+M\niQFmWfMKrbapVndTBbAaiK3RthJYwX+uMKixfJ4xJt4YEx8SEuLibqnGEN27My/eGE3ynqM88pGO\ny6xO73DRKWYsTaFH5za8ME6L1nkrVxJCItBfRPqKSCtgPLDGeQURCRaR6veaCSxwattFRKo/yUcB\n6eLQz2orwNXA9obtimoKV0X14OHLB/LJxlxeWa/jMqv/y1G0biOHi8t4c2KcFq3zYnUmBOub/Qzg\nKxxdOyuNMVtFZLaIXGOtdhGQISI7gG7AU1bbShzdRetFJA1H99M71t8LrXlpQA8cTycpD3TXRWcx\nNjaUl7/eyScb99sdjvIwr3y9g39lFjD7miEM7aVF67yZeFM3QHx8vElKSrI7DJ9UVlHFpPk/s3Hv\nMZZNO5u4sEC7Q1IeYENGHr9/L5Eb4kKZc0OU1inyUCKSbIyJr2s9fUBYuaSVfwvenhRHz85tmLYo\nmb2HS+wOSdks52gJ96/YyKDuHfjrmKGaDJoBTQjKZV0CWrFgSgIVVYbbFiZSeFLHZfZV1UXrKisN\nb02K06J1zYQmBFUvESHteWtSHLsLipmxNEXHZfZRf/00nc05hcwZN5zw4AC7w1GNRBOCqrdzzwri\nf68fxvc7C3hizVZ9HNXHrE7dz+Kf9jLtwghGD9WR9poTHUJTnZEb43uTXVDMm9/uIiI4gNsviLA7\nJOUGOw6dYOaqNEaEB/KnywfaHY5qZJoQ1Bl7+LcD2V1QzFOfbyMsKIDLIrvZHZJqQidKy7nz78kE\ntPbn9Ztj8Neidc2O/ouqM9aihfDijdEM69WJe5en6rjMzZgxhkc+2syeIyW8fnMMXbVoXbOkCUE1\nSNtWfrx7azyd27bk9oVJHCwstTsk1QQW/LCbz9MO8vDlAzknIsjucFQT0YSgGqxrxza8OzmBE6Xl\n3L4okZIyHZe5OUnafYSnP9/GZZHduONCvVfUnGlCUI0isqdjXOb03OPct3wjVTouc7NQUHSKu5em\n0KtLW54fN1x/fNbMaUJQjWbUoG48dlUka9MP8eyXWqvQ2zmK1qVyrKScNybG0qmtFq1r7vQpI9Wo\nfn9+ONkFxbz9XRZ9gwMYP6KP3SGpM/TSuh38kHmY58ZGMaSnFq3zBZoQVKMSEZ64OpI9R0p4bPUW\nege24/x+OnKWt/lm+yFe35DJjfGh3JjQu+4GqlnQLiPV6Pz9WvD6zTFEhAQwfXEymXlFdoek6mHf\nkRLuX7GJyB4dmT1mqN3hKDfShKCaRMc2LZk/OYFW/i247f1EjhSX2R2SckFpuaNoXZUxvDkpljYt\ntWidL9GEoJpM78B2zLs1noPHS7nj70mcqtBxmT3d7E/TSdtfyAvjhhMWpEXrfI0mBNWkYvt04YVx\nw0ncfZRHP0rTQnge7KPkHJb+vJc7Rkbw2yFatM4XuZQQRGS0iGSISKaIPFrL8jARWS8im0XkWxEJ\ndVrWR0TWisg2EUkXkXBr/hLrPbeIyAIR0Wfamqmrh/fkwcsG8HHqfl7/JtPucFQtth88zqzVaZzd\nN5CHf6tF63xVnQlBRPyAucAVQCQwQUQia6z2PLDIGBOFY2zkp52WLQLmGGMGAyOAPGv+EmAQMAxo\nC9zegP1QHm7GqH5cH9OLF9bt4B+bcu0ORzk5XlrO9MUpdGjTkte0aJ1Pc+VffgSQaYzJMsaUAcuB\nMTXWiQTWW9MbqpdbicPfGLMOwBhTZIwpsaY/NxbgFyAU1WyJCE+PHUZCeBce/GATyXuO2h2SwlG0\n7k8fbGbvkRLm3hxL1w5atM6XuZIQegH7nF7nWPOcbQLGWtPXAR1EJAgYABwTkVUikioic6wrjv/P\n6iq6BfjyTHZAeY/W/n68fUs8PTq1YdqiJPYd0XGZ7Tb/X9l8ufUgj4weyIi+gXaHo2zmSkKorXhJ\nzTuDDwEjRSQVGAnsBypw/PCdkAs6AAATsElEQVTtAmt5AhABTKnR9g3gO2PM97VuXGSaiCSJSFJ+\nfr4L4SpPFhjQivmTEyivrGLqwkSOl+q4zHZJ3H2Ep7/YzuVDuvEHHeBI4VpCyAGcf6oYCvxXJ7Ax\nJtcYc70xJgaYZc0rtNqmWt1NFcBqILa6nYg8AYQAD5xu48aYecaYeGNMfEhIiIu7pTxZv66OcZmz\n8ou5e0kKFTous9vlnzjF3UtS6N2lLXO0aJ2yuJIQEoH+ItJXRFoB44E1ziuISLCIVL/XTGCBU9su\nIlL9ST4KSLfa3A5cDkwwxugngo85r18wf7t2KN/vLODJf+i4zO5UUVnFPctSKTxZzhsT4+jYRh/w\nUw51JgTrm/0M4CtgG7DSGLNVRGaLyDXWahcBGSKyA+gGPGW1rcTRXbReRNJwdD+9Y7V5y1r33yKy\nUUQeb7zdUt5g/Ig+3HFhBIt/2st7P+y2Oxyf8eK6Hfw76zB/u3YokT072h2O8iAuFbczxnwOfF5j\n3uNO0x8CH56m7Togqpb5WlhP8cjoQWQXFPO3z9IJC2rHJYN1XOam9HX6Id74dhfjE3ozLl6L1qn/\npg8cK1u1aCG8PD6ayJ4d+eOyVNJzj9sdUrO193AJD6zcyJCeHXnymiF2h6M8kCYEZbt2rfyZPzmB\njm1aMnVhInnHdVzmxlZaXsn0JckAvDkxTovWqVppQlAeoVvHNrw7OZ7Ck+XcviiJk2VaCK8xPblm\nK1tzj/PijdH0CWpndzjKQ2lCUB5jaK9OvDo+hrT9hdy/QsdlbiwfJO1jeeI+pl90FpdG6j0adXqa\nEJRHuTSyG7OuHMyXWw/y3FcZdofj9dJzj/PY6i2cGxHEg5cNsDsc5eH0SR/lcab+pi9ZBcW89c9d\nRAQH6BCOZ+h4aTl3LUmmU9uWvDpBi9apumlCUB5HRPifa4aw70gJf/44jdDAtpx3lo7LXB/GGB5a\nuYl9R0+yfNo5hHRobXdIygvoVwblkVr6teD1m2MJDw5g+uIUsvJ1XOb6eOf7LNamH2LmFYNICNei\ndco1mhCUx+rUtiXvTUnAv4Vw2/uJHNVxmV3yc9Zhnv0ygyuGdmfqb/raHY7yIpoQlEdzjMscR25h\nKXcsTtZxmeuQd6KUGctS6RPYjuduiNKidapeNCEojxcXFsicG6L4JfsIf161RQvhnUZFZRV/XJrK\nidJy3pwUSwctWqfqSW8qK68wJroX2QXFvPz1TiJCArj74n52h+Rxnl+7g5+zj/DCuOEM6q5F61T9\naUJQXuPeS/qTXVDMnK8yCA8K4KqoHnaH5DHWpR/irX/uYsKIPoyN09Fo1ZnRLiPlNUSEZ8dGERfW\nhQdWbiR1r47LDLDncDEPrNzI0F4deeLqSLvDUV5ME4LyKm1a+jHvlji6dmzNHxYlk3PUt8dlLi2v\n5M7FKbQQ0aJ1qsE0ISivE9S+Ne9NSeBURSVT30/ihA+Py/z4J1vYduA4L900nN6BWrRONYwmBOWV\n+nXtwJsT48jML+KPy1J9clzmlYn7WJmUw90Xn8WoQVq0TjWcSwlBREaLSIaIZIrIo7UsDxOR9SKy\nWUS+FZFQp2V9RGStiGwTkXQRCbfmz7Dez4iI1iVQ9fab/sH8dcxQvs3I56+fptsdjlttzS3kL59s\n4fx+QTxw2UC7w1HNRJ0JQUT8gLnAFUAkMEFEat65eh5YZIyJAmYDTzstWwTMMcYMBkYAedb8H4BL\ngT0N2gPl024+uw9/uKAvC/+9h/d/yLY7HLcoPFnO9MUpdGnXilfGx+DXQn98phqHK1cII4BMY0yW\nMaYMWA6MqbFOJLDemt5QvdxKHP7WuMoYY4qMMSXWdKoxZnfDd0H5ukevGMylg7sx+9N0NmzPq7uB\nFzPG8NAHm8g9dpK5E2MIbq9F61TjcSUh9AL2Ob3OseY52wSMtaavAzqISBAwADgmIqtEJFVE5lhX\nHEo1Gr8WwivjoxnUvSMzlqaw7UDzHZf57e+yWJd+iJlXDiYuTIvWqcblSkKo7Xq0Zu2Ah4CRIpIK\njAT2AxU4fvh2gbU8AYgAptQnQBGZJiJJIpKUn59fn6bKhwS09mf+lHjat/Fn6vuJ5J1ofuMy/5R1\nmOe+3M5Vw3pw2/nhdoejmiFXEkIO4DxCSSiQ67yCMSbXGHO9MSYGmGXNK7TaplrdTRXAaiC2PgEa\nY+YZY+KNMfEhISH1aap8TI9ObZk/OYGjJeX8YVFysxqXOe94KTOWphIeFMAzY4dp0TrVJFxJCIlA\nfxHpKyKtgPHAGucVRCRYRKrfayawwKltFxGp/iQfBfjW4yDKrYb26sTL46PZnHOMBz9oHuMyV1RW\nMWNZKsWnKnhzUpwWrVNNps6EYH2znwF8BWwDVhpjtorIbBG5xlrtIiBDRHYA3YCnrLaVOLqL1otI\nGo7up3cAROQeEcnBccWxWUTebdQ9Uz7r8iHdmXnFID5PO8gL67x/XOY5X2XwS/YR/vf6oQzs3sHu\ncFQzJt5USjg+Pt4kJSXZHYbyAsYY/vxxGst+2cecG6IYF++d4zJ/ueUgdy5OZuLZfXjqumF2h6O8\nlIgkG2Pi61pPf6msmiURYfaYoZzfL4g/f5zGT1mH7Q6p3rILinn4g01EhXbicS1ap9xAE4Jqtlr6\nteCNm+PoE9iOOxcnk11QbHdILjtZVsn0xcm0aCHMvTmW1v76tLZqepoQVLPWqV1LFkxJQIDb3k/k\nWInnj8tsjOEvn2xh+8ETvHxTtBatU26jCUE1e2FBAcy7NZ79R09y5+Jkyio8uxDeisR9fJicwz2j\n+nHxoK52h6N8iCYE5RMSwgN57oYofso6wqyP0zx2XOYt+wt5fM1WLugfzL2XDrA7HOVjdAhN5TOu\njelFVkExr67fSURIe6ZfdJbdIf2XwpJypi9JJiigFS/fFK1F65TbaUJQPuX+Sx3jMj/75XbCg9px\nxTDPGJe5qsrw4AcbOXCslBV3nEuQFq1TNtAuI+VTRIQ5N0QR26cz96/cyKZ9x+wOCYC3vtvF19vy\nmHXVYOLCutgdjvJRmhCUz2nT0o95t8YT3L41ty9KYv+xk7bG8+OuAp7/KoOronow5bxwW2NRvk0T\ngvJJwe1bs2BKAqVllUx9P5GiUxW2xHHoeCn3LEulb3AAz46N0qJ1ylaaEJTPGtCtA3MnxrIzr4h7\nlqVS6eZCeOWVVcxYmkLxqUrenBRH+9Z6S0/ZSxOC8mkXDgjhyWuG8M32PP72mXsL8T77xXYSdx/l\nmbHDGNBNi9Yp++lXEuXzbjknjOz8Yhb8kE1EcAC3nBve5Nv8Iu0A7/4rm1vOCWNMdM0BCJWyhyYE\npYBZVw1mz+FinvxHOr0D23HRwKb7hXBWfhEPf7iZ4b0789jvBjfZdpSqL+0yUgprXOYJMQzo1oEZ\nS1PJOHiiSbZzsqySu5ak4O8nzL05RovWKY+iCUEpS/vW/syfHE+7Vn7c9n4i+SdONer7G2OYtTqN\njEOOonWhXbRonfIsmhCUctKzc1venRzP4eJT/GFREqXljTcu87Jf9rEqZT/3jOrfpF1SSp0plxKC\niIwWkQwRyRSRR2tZHiYi60Vks4h8KyKhTsv6iMhaEdkmIukiEm7N7ysiP4vIThFZYY3XrJTtokI7\n8/JN0Wzcd4yHPtjUKOMyp+UU8qRVtO6eS/o3QpRKNb46E4KI+AFzgSuASGCCiNQcvul5YJExJgqY\nDTzttGwRMMcYMxgYAeRZ858FXjLG9AeOAlMbsiNKNabRQ3vw6BWD+HTzAV76ekeD3utYSRnTlyQT\n3L4Vr4yP0aJ1ymO5coUwAsg0xmQZY8qA5cCYGutEAuut6Q3Vy63E4W+MWQdgjCkyxpSI4+eYo4AP\nrTYLgWsbtCdKNbI7LozgxvhQXvsmk4+Sc87oPaqqDA+s3MSh46XMnRhLYIBeCCvP5UpC6AXsc3qd\nY81ztgkYa01fB3QQkSBgAHBMRFaJSKqIzLGuOIKAY8aYil95T6VsJSL87dphnBsRxKOrNvNL9pF6\nv8eb/9zFN9vzeOyqSGL6aNE65dlcSQi1Xd/W7FR9CBgpIqnASGA/UIHjdw4XWMsTgAhgiovv6di4\nyDQRSRKRpPz8fBfCVarxtPJvwVuT4ujdpR13/D2J3fUYl/mHzAJeWJvB1cN7cuu5YU0YpVKNw5WE\nkAP0dnodCuQ6r2CMyTXGXG+MiQFmWfMKrbapVndTBbAaiAUKgM4i4n+693R673nGmHhjTHxISEg9\ndk2pxlE9LrMBbluYSGFJeZ1tDhY6itZFhLTnmeuHadE65RVcSQiJQH/rqaBWwHhgjfMKIhIsItXv\nNRNY4NS2i4hUf5KPAtKNY/zCDcAN1vzJwCdnvhtKNa3w4ADenhTHviMlTF+STHnl6cdlLq+s4u6l\nKZwsr+StSbEEaNE65SXqTAjWN/sZwFfANmClMWariMwWkWus1S4CMkRkB9ANeMpqW4mju2i9iKTh\n6Cp6x2rzCPCAiGTiuKcwv9H2SqkmcHZEEM9cH8WPuw7z2MdbTjsu89Ofbyd5z1GeGRtFv65atE55\nD5e+uhhjPgc+rzHvcafpD/nPE0M1264DomqZn4XjCSalvMbYuFCyC4p5fUMmESEB3DHyv8dl/mzz\nARb8kM3kc8O4ZnhPm6JU6szotaxS9fTAZQPIPlzMM19uJywogNFDuwOwK7+IP324iejenZl1Vc2f\n6ijl+bR0hVL11KKF8MK44QwP7cx9K1JJyymkpKyC6YuTaeXfgrkTY2nlr/+1lPfRs1apM9CmpR/v\n3BpPUEBrpi5M5L7lG9mZV8Qr42Po1bmt3eEpdUY0ISh1hkI6OMZlLimrZG36Ie67ZAAXDtBHo5X3\n0nsISjXAwO4dWDAlgR93FfDHUf3sDkepBtGEoFQDjegbyIi+gXaHoVSDaZeRUkopQBOCUkopiyYE\npZRSgCYEpZRSFk0ISimlAE0ISimlLJoQlFJKAZoQlFJKWeR0Nd09kYjkA3vOsHkwjpHaPI3GVT8a\nV/1oXPXTXOMKM8bUWVfFqxJCQ4hIkjEm3u44atK46kfjqh+Nq358PS7tMlJKKQVoQlBKKWXxpYQw\nz+4ATkPjqh+Nq340rvrx6bh85h6CUkqpX+dLVwhKKaV+hdcnBBFZICJ5IrLlNMtFRF4VkUwR2Swi\nsU7LJovITuvPZDfHNdGKZ7OI/Cgiw52W7RaRNBHZKCJJbo7rIhEptLa9UUQed1o2WkQyrGP5qJvj\netgppi0iUikigdaypjxevUVkg4hsE5GtInJvLeu4/RxzMS63n2MuxuX2c8zFuNx+jolIGxH5RUQ2\nWXH9Ty3rtBaRFdYx+VlEwp2WzbTmZ4jI5Q0OyBjj1X+AC4FYYMtpll8JfAEIcA7wszU/EMiy/u5i\nTXdxY1znVW8PuKI6Luv1biDYpuN1EfBpLfP9gF1ABNAK2AREuiuuGuteDXzjpuPVA4i1pjsAO2ru\ntx3nmItxuf0cczEut59jrsRlxzlmnTPtremWwM/AOTXWuQt4y5oeD6ywpiOtY9Qa6GsdO7+GxOP1\nVwjGmO+AI7+yyhhgkXH4CegsIj2Ay4F1xpgjxpijwDpgtLviMsb8aG0X4CcgtLG23ZC4fsUIINMY\nk2WMKQOW4zi2dsQ1AVjWWNv+NcaYA8aYFGv6BLAN6FVjNbefY67EZcc55uLxOp0mO8fOIC63nGPW\nOVNkvWxp/al5Y3cMsNCa/hC4RETEmr/cGHPKGJMNZOI4hmfM6xOCC3oB+5xe51jzTjffDlNxfMOs\nZoC1IpIsItNsiOdc6xL2CxEZYs3ziOMlIu1wfKh+5DTbLcfLulSPwfEtzpmt59ivxOXM7edYHXHZ\ndo7VdbzcfY6JiJ+IbATycHyBOO35ZYypAAqBIJrgePnCmMpSyzzzK/PdSkQuxvGf9TdOs883xuSK\nSFdgnYhst75Bu0MKjp+5F4nIlcBqoD8ecrxwXMr/YIxxvppo8uMlIu1xfEDcZ4w5XnNxLU3cco7V\nEVf1Om4/x+qIy7ZzzJXjhZvPMWNMJRAtIp2Bj0VkqDHG+V6a284vX7hCyAF6O70OBXJ/Zb7biEgU\n8C4wxhhzuHq+MSbX+jsP+JgGXgbWhzHmePUlrDHmc6CliATjAcfLMp4al/JNfbxEpCWOD5ElxphV\ntaxiyznmQly2nGN1xWXXOebK8bK4/Ryz3vsY8C3/t1vx/x8XEfEHOuHoXm3849WYN0js+gOEc/qb\npFfx3zf8frHmBwLZOG72dbGmA90YVx8cfX7n1ZgfAHRwmv4RGO3GuLrzn9+njAD2WsfOH8dN0b78\n54bfEHfFZS2v/o8Q4K7jZe37IuDlX1nH7eeYi3G5/RxzMS63n2OuxGXHOQaEAJ2t6bbA98Dvaqxz\nN/99U3mlNT2E/76pnEUDbyp7fZeRiCzD8dRCsIjkAE/guDGDMeYt4HMcT4FkAiXA761lR0Tkr0Ci\n9VazzX9fIjZ1XI/j6Ad8w3F/iArjKF7VDcdlIzj+gyw1xnzpxrhuAKaLSAVwEhhvHGdfhYjMAL7C\n8TTIAmPMVjfGBXAdsNYYU+zUtEmPF3A+cAuQZvXzAvwZx4etneeYK3HZcY65Epcd55grcYH7z7Ee\nwEIR8cPRY7PSGPOpiMwGkowxa4D5wN9FJBNHshpvxbxVRFYC6UAFcLdxdD+dMf2lslJKKcA37iEo\npZRygSYEpZRSgCYEpZRSFk0ISimlAE0ISimlLJoQlFJKAZoQlFJKWTQhKKWUAuD/AdOrLvSrtKwW\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(k_list, accuracy_list)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.857023809524\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn import metrics\n",
    "\n",
    "clf = DecisionTreeClassifier(random_state=0).fit(X_tr, y_tr)\n",
    "\n",
    "y_pred = clf.predict(X_test)\n",
    "\n",
    "print(\"Accuracy:\",metrics.accuracy_score(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.85702380952380952"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Predictions using KNN with 3 Nearest Neigbors model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pixel0</th>\n",
       "      <th>pixel1</th>\n",
       "      <th>pixel2</th>\n",
       "      <th>pixel3</th>\n",
       "      <th>pixel4</th>\n",
       "      <th>pixel5</th>\n",
       "      <th>pixel6</th>\n",
       "      <th>pixel7</th>\n",
       "      <th>pixel8</th>\n",
       "      <th>pixel9</th>\n",
       "      <th>...</th>\n",
       "      <th>pixel775</th>\n",
       "      <th>pixel776</th>\n",
       "      <th>pixel777</th>\n",
       "      <th>pixel778</th>\n",
       "      <th>pixel779</th>\n",
       "      <th>pixel780</th>\n",
       "      <th>pixel781</th>\n",
       "      <th>pixel782</th>\n",
       "      <th>pixel783</th>\n",
       "      <th>Label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 785 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   pixel0  pixel1  pixel2  pixel3  pixel4  pixel5  pixel6  pixel7  pixel8  \\\n",
       "0       0       0       0       0       0       0       0       0       0   \n",
       "1       0       0       0       0       0       0       0       0       0   \n",
       "2       0       0       0       0       0       0       0       0       0   \n",
       "3       0       0       0       0       0       0       0       0       0   \n",
       "4       0       0       0       0       0       0       0       0       0   \n",
       "\n",
       "   pixel9  ...    pixel775  pixel776  pixel777  pixel778  pixel779  pixel780  \\\n",
       "0       0  ...           0         0         0         0         0         0   \n",
       "1       0  ...           0         0         0         0         0         0   \n",
       "2       0  ...           0         0         0         0         0         0   \n",
       "3       0  ...           0         0         0         0         0         0   \n",
       "4       0  ...           0         0         0         0         0         0   \n",
       "\n",
       "   pixel781  pixel782  pixel783  Label  \n",
       "0         0         0         0      2  \n",
       "1         0         0         0      0  \n",
       "2         0         0         0      9  \n",
       "3         0         0         0      9  \n",
       "4         0         0         0      3  \n",
       "\n",
       "[5 rows x 785 columns]"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Final Model with all the train data:\n",
    "knn = KNeighborsClassifier(n_neighbors=3).fit(X, y)\n",
    "test_data['Label'] = knn.predict(test_data.values)\n",
    "\n",
    "test_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test Data: Label of image at Index 16784 is 8\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAABDElEQVR4nGNgGFgwm5mZibkAu9wB\nThYvNRZOJ6fg45iSXuKv/3w54cudZ8H1FkPSSZKBgYGB4cSnHJbXEBEmDDUWfAzKnBiiPWwpPxkY\nGBg2MEZgcVEXSwoDA8MaPsE9WCS/mrFMPVPJxXMIq2deywtKsOjtg/NZkCXP/f3EyBDkhE3fOXNm\ntaxiNubN2OTcuYu/MjBsY2HpQZf6NYlb7ykDAwPDv4uKbBPR5FJZeNdB2Xdl0fz5ioVlI5yTI/ge\nxbWH/i/3h0u+Q7PyAss1OPsOM5qxb4RFQi+8YWBgYHizRJFtElSUEUq/zjz5nN2H4T/j5l8MncVo\nkgwMzw+f+b7iHSNHtb8Wuj/pCQChSEfpt1FvrwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<PIL.Image.Image image mode=L size=28x28 at 0x26B8959B7B8>"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Visualizing and checking label of a random image\n",
    "index_of_img = random.randrange(test_data.shape[0])\n",
    "print('Test Data: Label of image at Index {} is {}'.format(index_of_img,test_data['Label'][index_of_img]))\n",
    "img_visualizer(test_data.iloc[:,:-1].iloc[index_of_img,:].values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction file \"final_predictions0.csv\" written to \"predictions folder\" in root directory\n"
     ]
    }
   ],
   "source": [
    "final = test_data['Label'].reset_index(drop=False)\n",
    "final[\"ImageId\"] = final[\"index\"] + 1\n",
    "\n",
    "i = 0\n",
    "while os.path.exists(\"./predictions/final_predictions%s.csv\" % i):\n",
    "    i += 1\n",
    "\n",
    "final[[\"ImageId\", \"Label\"]].to_csv('./predictions/final_predictions%s.csv' %i,index=False)\n",
    "\n",
    "print('Prediction file \"{}\" written to \"predictions folder\" in root directory'.\\\n",
    "      format('final_predictions%s.csv' %i))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:Anaconda3]",
   "language": "python",
   "name": "conda-env-Anaconda3-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}