2.1 Building Pony Classifier from scratch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's build a simple Pony classifier from scratch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below we will explore a simple strategy around build a Convolutional Neural Network from scratch using the data we have. Sadly, I don't have access to Hasbro's image assets, so will work with a fairly small dataset.\n",
    "\n",
    "Disclaimer: We kind of expect our results to be not so great! Deep Learning learns representations from LOTS OF DATA!\n",
    "\n",
    "However, we will look at other possible strategies to improve this baseline."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To start let's confirm we have our dataset. As mentioned earlier, I used a Python script from the Google Images Download [repository](https://github.com/hardikvasa/google-images-download) to get some samples. I store them locally in the /images/pony folder: "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start building a Pony Classifier CNN. Our images are downloaded and prepared to be of size (150px, 150px)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "from keras import layers\n",
    "from keras import models\n",
    "\n",
    "model = models.Sequential()\n",
    "model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(150, 150, 3)))\n",
    "model.add(layers.MaxPooling2D((2, 2)))\n",
    "model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n",
    "model.add(layers.MaxPooling2D((2, 2)))\n",
    "model.add(layers.Conv2D(128, (3, 3), activation='relu'))\n",
    "model.add(layers.MaxPooling2D((2, 2)))\n",
    "model.add(layers.Conv2D(128, (3, 3), activation='relu'))\n",
    "model.add(layers.MaxPooling2D((2, 2)))\n",
    "model.add(layers.Flatten())\n",
    "model.add(layers.Dense(256, activation='relu'))\n",
    "model.add(layers.Dense(3, activation='softmax'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_1 (Conv2D)            (None, 148, 148, 32)      896       \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 74, 74, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_2 (Conv2D)            (None, 72, 72, 64)        18496     \n",
      "_________________________________________________________________\n",
      "max_pooling2d_2 (MaxPooling2 (None, 36, 36, 64)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_3 (Conv2D)            (None, 34, 34, 128)       73856     \n",
      "_________________________________________________________________\n",
      "max_pooling2d_3 (MaxPooling2 (None, 17, 17, 128)       0         \n",
      "_________________________________________________________________\n",
      "conv2d_4 (Conv2D)            (None, 15, 15, 128)       147584    \n",
      "_________________________________________________________________\n",
      "max_pooling2d_4 (MaxPooling2 (None, 7, 7, 128)         0         \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 6272)              0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 256)               1605888   \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 3)                 771       \n",
      "=================================================================\n",
      "Total params: 1,847,491\n",
      "Trainable params: 1,847,491\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras import optimizers\n",
    "model.compile(loss = 'categorical_crossentropy', optimizer = optimizers.RMSprop(lr=1e-4), metrics=['acc'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We know that data should be formatted appropriately before being fed into the network. At this point we have a bunch of image files (jpeg, png, etc.) sitting on Azure blob storage. So we need to perform a few steps before these could be used for training and testing.\n",
    "1. Read a file\n",
    "2. Decode its content to RGB grid of pixels\n",
    "3. Convert those into floating-point multi-dimensional arrays (or tensors)\n",
    "4. Rescale pixel values (between 0 and 255) to the [0,1] interval\n",
    "\n",
    "In Keras we could use ImageDataGenerator object to perform these steps as shown below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 225 images belonging to 3 classes.\n",
      "Found 81 images belonging to 3 classes.\n"
     ]
    }
   ],
   "source": [
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "\n",
    "train_dir = 'images/ponies/train'\n",
    "validation_dir = 'images/ponies/validate'\n",
    "\n",
    "train_datagen = ImageDataGenerator(\n",
    "        rescale=1./255,\n",
    "        shear_range=0.2,\n",
    "        zoom_range=0.2,\n",
    "        horizontal_flip=True)\n",
    "\n",
    "test_datagen = ImageDataGenerator(rescale=1./255)\n",
    "\n",
    "train_generator = train_datagen.flow_from_directory(\n",
    "        train_dir,\n",
    "        target_size=(150, 150),\n",
    "        batch_size=16,\n",
    "        class_mode='categorical')\n",
    "\n",
    "validation_generator = test_datagen.flow_from_directory(\n",
    "        validation_dir,\n",
    "        target_size=(150, 150),\n",
    "        batch_size=16,\n",
    "        class_mode='categorical')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data batch shape: (16, 150, 150, 3)\n",
      "labels batch shape: (16, 3)\n"
     ]
    }
   ],
   "source": [
    "for data_batch, labels_batch in train_generator:\n",
    "    print('data batch shape:', data_batch.shape)\n",
    "    print('labels batch shape:', labels_batch.shape)\n",
    "    break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "label_map = (train_generator.class_indices)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/7\n",
      "20/20 [==============================] - 2s 114ms/step - loss: 0.9802 - acc: 0.4926 - val_loss: 0.7334 - val_acc: 0.8138\n",
      "Epoch 2/7\n",
      "20/20 [==============================] - 2s 83ms/step - loss: 0.6125 - acc: 0.6539 - val_loss: 0.3982 - val_acc: 0.8897\n",
      "Epoch 3/7\n",
      "20/20 [==============================] - 2s 87ms/step - loss: 0.3882 - acc: 0.8747 - val_loss: 0.3571 - val_acc: 0.8828\n",
      "Epoch 4/7\n",
      "20/20 [==============================] - 1s 75ms/step - loss: 0.3300 - acc: 0.8778 - val_loss: 0.3370 - val_acc: 0.8759\n",
      "Epoch 5/7\n",
      "20/20 [==============================] - 2s 84ms/step - loss: 0.2653 - acc: 0.8901 - val_loss: 0.2176 - val_acc: 0.9379\n",
      "Epoch 6/7\n",
      "20/20 [==============================] - 2s 86ms/step - loss: 0.2242 - acc: 0.9217 - val_loss: 0.2235 - val_acc: 0.9103\n",
      "Epoch 7/7\n",
      "20/20 [==============================] - 2s 76ms/step - loss: 0.1957 - acc: 0.9248 - val_loss: 0.2096 - val_acc: 0.9172\n"
     ]
    }
   ],
   "source": [
    "history = model.fit_generator(\n",
    "        train_generator,\n",
    "        steps_per_epoch=20,\n",
    "        epochs=7,\n",
    "        validation_data=validation_generator,\n",
    "        validation_steps=10) \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.save('my_little_pony_scratch.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8VPWd//HXJ+FmliTcEQSCQKVVooUqlov+omwNUBRL\nsAIFcesqtaVgWqjWaonWrVp1cW1rK13qBWpRcKtYRPBnDdV6gcilqCAINtwEK6tcROWSz/4xJ3ES\nMpkJTDLJ4f18PObBnDnf+Z7PmZB3vvM9Z+aYuyMiIo1fWqoLEBGR5FCgi4iEhAJdRCQkFOgiIiGh\nQBcRCQkFuohISCjQQ8bM0sxsn5l1SWbbVDKznmZWVgf9DjGzd6OW15vZoETaHsO2fmdmNxzr80US\n0STVBZzozGwfUP5hgH8BPgOOBI9Ncvc/1qY/dy8DMpPdtgGoqw9MVPTr7l9MRg1mdhUw3t0viOr7\n6mMrTyRxCvQUc/eKQDWzzcBV7v5CrPZmlu7uR+qlODlWRt39AWpQ9P+xYdGUS8Niwe3zB8x+Zmbz\nzOxRM9sDfMvMvmpmr5jZh2a23cz+y8zSg/bpZlZmZt2C5TnB+mfMbK+Z/c3McmrbNlg/zMzeDrZ7\nn5m9ZGZXVLsjidV4jZltNLPdZvZfUc9NM7OZZvaBmb0DDI35gpndaGZ/rPLYr83s7uD+VWb2VrA/\nG4PRc6y+tprZ+cH9k4LX43/NbC3wlSptf2Jmm4J+15rZxcHjfYBfAucF01nvR722P416/neCev5p\nZv9jZicn8trU5nUO1uea2XNBPzvMbFrUdm42s3fMbI+ZLTezk6ub3jKzF8t/zsHruSzYzm7gJ2bW\ny8z+EmzjfTN7xMyiByrdzOxPwbr3g59t86Dm3lHtTjazj82sdaz9lTjcXbcGcgPeBS6s8tjPgE+B\n4cFycyLhcg6R8O8OrAe+G6xPJzJl0y1YngO8D/QN1s0DHjmGth2AvcCIYF0hkemhK2LsS7way4A/\nAS2BHGB3+b4Dk4G1QCegNbAMOBJjO6cGdZ0U1fcuoG+w/HUgJ7ifBxwA+gTLQ4DNUX1tBc4P7t8N\n/AXIAroCb1ZpOxroENwfA+wD2gfLVwF/qVLnHOCnwf2LgJ1ALtAM+DXwfCKvTS1f56xgO5OBpkF/\nZwfrfgysAnoEy2cCrYCeVV9r4MXyn3Owb4eAa4JtNge+AFwQ1N4uaP+LqP1ZC9wJnBS0HxCs+y3w\ns6jt/AB4ItW/h435lvICdIv6YcQO9P8f53k/BB4L7pcHQnRI3x/V9mLg78fQ9t+AZVW2u4MYgZ5g\njedErX8C+EFwfxnw7ah1w6qGTJW+XwbGRLVdV0Pbp4Frg/s1BXopcEHUumuj21bT71pgWHA/XqA/\nBNwWtS4TOAx0jvfa1PJ1Hg+8FqPdO8DQah5PJNDfiVNDQfl2gfOC/ydWTbuBVV7/VcClyf69OpFu\nmnJpHLZGL5hZbzP7s5m9Z5FpmFuIjIxi2Rl1/wCRkVpt23auWgewLVYnCda4K8FtldZQL8AfgbHB\n/bHAo1F1jDCzV4PpgA+Br1VTR3U6UXn/KtVgZlea2epgSuZDoHeC/UJk/yr6c/d9wIfAKVFtYr02\nlcR5nbsCm2LU0BXYnGC9VVX9/9jRzB4zs21m9hGRP1jlNXQB/uFBYkdz95eBQ2Y2yMzOCGpadIw1\nCZpDbyyq/jI8QGRE2MPds4EZVJl7rwPvEfmFi3ZKdQ0Dx1Nj1W3lxGoYeBz4VzPrDIwkCHQzawHM\nB/6DyHRIa+C5BOvYGasGMzsVuJ/IWUhtgn7fjuo33gHRHVX6yyQytRTzD2QNanqdtwK9YjxvC5HR\neFUfBzW1iHrs5Cptqu7fnUSmBc9w91bAlVVqyDGzWK/5I8CE4Pa4ux+K0U4SoEBvnDKBPe7+iZl9\nCZhUD9v8M9DXzL4eHFC7jppHpMdT4+PAdWbW2czaAj+qqbG77wL+RmRkuN7dy0elzYnMHX8AuJmN\nIDLNkmgNN5pZtkUOGn8val1LItMiHwSvxdVA9CmPu4AuZhbrLLI/AleZWR8zaw7cDvzV3d9LsLZo\nNb3OC4GuZvZdM2tmZplmdk6wbjZwm5n1ADCzs8yslbvvJPLHbLxFDk5fQ/w/qJlE/hDsM7OuwLSo\nda8QOQbw8+BAcwszGxi1fi6R4xFjiYS7HAcFesOS6KluPwSuNLO9wG+IHLyM1U+8PhNq6+7vA5cD\nM4kE5KlE5jw/S0KNVZd/AzxPZOT5GpFRdjyPEgnrP0TVvIfIwdsniYTKKCJz6LFE1zCDSLD9g8g0\nwMNR/a4lcibLCiKj7S8Ar0Y99zlgI7DLzHYctRH3JcCtQV3biUxLfCtGHdUtR4v5Orv7XiJTTKOJ\n/JF5Gzg/WH1XsP3ng6maB4DyUfnVwE+AfwI9quxbdWYA5wIfBX0uiKrhCJED6acTGa2XEpljL19f\nSuTn/Jm7x9uOxGHVTG1VbmA2m8gPZJe7nxmjzX1EDkZ9DFzp7quTXag0LGaWRiTMCtz9b6muRxov\nM3sY2OTut6a6lsYukRH6g0B+rJVmNgzo6e5fIPJ277dJqk0aGDPLD6YgmgM/BQ4Cy1NcljRiwZTP\nJcDvU11LGMQNdHd/icgR+FhGEsx9uftrQLaZdUxOedLADCZyZsQuIm/lL9VBLDlWZvZzItN2/+Hu\nx3JAWKpIxkf/T6HyaUzbg8d2Vd9cGit3vxm4OdV1SDi4+43AjamuI0x0UFREJCSSMULfTuXzdbsE\njx3FzE6ILywSEUk2d4/7+YlER+hHfWlUlIVA+Rf3fBX4KDgvOFZRob3NmDEj5TVo/7R/J9q+nQj7\nl6i4I3Qze5TIlxq1NbMtRM45bRbJZp/l7s+Y2XCLfCvex0S+80NEROpZ3EB393EJtJmcnHJERORY\n6aBoEuXl5aW6hDql/Wu8wrxvEP79S1TcT4omdWNmXp/bExEJAzPDEzgoqkvQiTQg3bt3p7Q03rcF\nS1jl5OTwj3/845ifrxG6SAMSjMRSXYakSKyff6IjdM2hi4iEhAJdRCQkFOgiIiGhQBeReldWVkZm\nZibbtsX/ksXatD3R6aCoSAPSUA+KZmZmUn5Z0I8//pjmzZuTnp6OmfHAAw8wduzYOD1IIo73oKgC\nXaQBqe4XurDwLlau3F8RqBD5TqR+/Voyc+b0hPpNRh/levTowezZs7ngggtitjly5Ajp6em16ld0\nlotI6A0alEtJyUCWLSuquJWUDGDw4GqvCFlnfZSr7gujbr75ZsaMGcO4cePIzs7mD3/4A6+++ioD\nBgygdevWnHLKKUydOpUjR44AkcBPS0tjy5YtAEyYMIGpU6cyfPhwsrKyGDRoUMX5+LVpC7B48WJ6\n9+5N69atmTJlCoMHD+aRR6q//nRNNQKsXbuWr33ta7Rt25bOnTtz9913V9T0s5/9jF69epGdnU3/\n/v3ZuXNnrV/LpKvnbwxzEYmtut+RsrIyP/fc6xzKHNwhslxWVpZwv8noo1z37t39+eefr/TYTTfd\n5M2bN/dFixa5u/unn37qJSUlvnz5ci8rK/N3333Xe/fu7b/+9a/d3f3w4cOelpbmpaWl7u4+fvx4\nb9++va9cudIPHz7sl19+uU+YMKHWbXft2uWZmZn+9NNP++HDh/0///M/vVmzZv7www9Xuy811bhn\nzx7v2LGj//KXv/SDBw/6vn37fMWKFe7u/vOf/9y//OUv+6ZNm9zdfc2aNf7hhx/W+rWsKlZGBo/H\nzViN0EUaODNj2rR8MjKWApCRsYTp04dWmj6pjz7iGTx4MMOHDwegefPmfOUrX+Gcc87BzOjevTtX\nX301y5Ytq2jvVUb5o0ePpm/fvqSnp/Otb32L1atX17rtokWL6Nu3LyNGjCA9PZ3CwkLatm0bs+aa\naly4cCE5OTlMnjyZpk2b0rJlS84++2wAZs+eze23306PHj0AOPPMM2nVqtWxvnRJo0AXaQQKCvLJ\nzX0WcHJzlzBq1EUp6aMmXbt2rbT89ttvM2LECDp16kR2djYzZszggw8+iPn8k08+ueJ+RkYG+/fv\nr3XbHTt2HFVHly5dYvZTU41bt26lZ8+e1T5v69atFWHekCjQRRqB8hF2ZuYPjnlknYw+4vUfbdKk\nSeTm5rJ582b27NnDLbfcUudn8HTq1ImtW7dWemz79movoBa3xq5du/LOO+9U+7xu3bqxadOm5BWe\nJAp0kUaioCCf736303GNrJPRR6L27dtHdnY2J510EuvWreOBBx6o822OGDGCVatWsWjRIo4cOcK9\n995b47uCmmq85JJL2Lp1K/fffz8HDx5k3759rFixAoCrrrqKm266ic2bNwOwZs0aPvroo7rduQQo\n0EUaCTPjjjt+dFwj62T1kYh77rmHhx56iKysLK699lrGjBkTs594fSbatkOHDjz22GMUFhbSrl07\n3n33Xfr27Uvz5s1rXWNWVhbPPfccCxYsoGPHjvTu3Zu//vWvAEyfPp1LL72UIUOGkJ2dzaRJk/j0\n009r3If6oPPQRRqQhvrBosaqrKyMzp0788QTTzBo0KBUlxOXzkMXEYmyZMkS9uzZw2effcatt95K\ns2bN6N+/f6rLqhcKdBEJlZdeeokePXrQsWNHnnvuOZ588kmaNm2a6rLqhaZcRBoQTbmc2DTlIiIi\ngK4pKiLSIFX3hWrxKNBFRBqgDe+9zIvbNuFH2iT8HE25iIg0QFeOHYf1Xwf/tix+44ACXUSkARp9\nyWh67uoFtThGrkAXkTpXWlpKWloaZWVlAAwfPpw5c+Yk1La2br/9dq655ppjrrWhMDP+Y+qtpL2d\n+My4TlsUaUAa6mmLw4YN49xzz6WoqKjS40899RTf+c532L59O2lpsceHpaWl9OjRg0OHDtXYrrZt\nly1bxvjx44/6Qq7GqurP393pPeR0Nr6wPqHTFhOKfjMbCtxLZEQ/293vrLK+FfB7oCfwCfBtd38r\n8d0QkVgKf1rIytKVR18+LqcfM2+dWS99TJw4kZtuuumoQJ87dy4TJkyIG7x1xd2T/q2RDUn5KP2b\nL3wzofZxfwpmlgb8CsgHzgDGmtkXqzS7EVjl7mcBE4H7alW1iMQ06OxBlKSXsOzUZRW3krQSBp8z\nuN76uPTSS9m9ezcvvfRSxWMfffQRf/7zn7niiisAeOaZZ+jXrx/Z2dnk5ORwyy23xOzvggsu4Pe/\n/z0Q+b6VadOm0b59e3r16sWiRYsqtX3ooYc4/fTTycrKolevXsyaNQuAAwcOMHz4cHbs2EFmZiZZ\nWVns3LmTW265hQkTJlQ8f+HChfTp04c2bdpw4YUXsn79+op1p556Kvfccw9nnXUWrVu3ZuzYsRw8\neLDamjdv3syQIUNo164dHTp0YPz48ezdu7di/bZt2ygoKKBDhw60b9+eKVOmVKz73e9+V7EPffr0\nqXTxjnhGXzI64baJ/FntD2x091J3PwTMA0ZWaXM68BcAd38b6G5m7ROuQkRiKri4gNx9uZ8fHHPI\n3Z/LqBGj6q2PFi1acNlll1W6Nudjjz3Gl770Jfr06QNAy5YtmTNnDnv27GHRokX89re/ZeHChXH7\nnjVrFs888wxr1qyhpKSEBQsWVFrfsWNHnnnmGfbu3cuDDz5IYWEhq1evJiMjg8WLF9O5c2f27dvH\n3r17Ky58UT5q37BhA+PGjeO+++7jn//8J8OGDePiiy/m8OHDFf3Pnz+fpUuX8u6777JmzRoeeuih\naut0d2688UZ27tzJunXr2LZtW8U7lrKyMkaMGMGpp57Kli1b2L59e8U3N86fP59bb72VuXPnsnfv\nXhYuXFjjVZSqqs07kEQC/RQgeoJqW/BYtDXAqGDj/YFuQOzLhIhIwsyMaROmkbElA4CM0gymXzG9\n9pegO84+Jk6cyPz58ytGsHPmzGHixIkV688//3zOOOMMAPr06cOYMWMqXXIulvnz53PdddfRuXNn\nWrVqxY9//ONK64cNG0b37t0BOO+887jooot48cUXE6r58ccfZ8SIEVx44YWkp6czbdo0PvnkE15+\n+eWKNlOnTqVjx460atWKiy++OObouWfPngwZMoQmTZrQtm1bCgsLK/bvtdde47333uMXv/gFLVq0\noFmzZgwcOBCIXK7uRz/6Ef369QOgR48eR11VKVmSNfF1B9DazFYC3wNWAUdqfoqIJCp6hF3b0Xmy\n+hg0aBDt27fnySefZPPmzaxYsYJx48ZVrF++fDkXXnghHTp0oFWrVjzwwAM1XlyiXNXLxuXk5FRa\nv3jxYgYMGEDbtm1p3bo1ixcvTqjf8r6j+zMzunbtWukqRh07dqy4X9Ol795//33Gjh1Lly5daNWq\nFePHj6+oY9u2beTk5FR7LKGmS9klWyIHRbcTGXGX6xI8VsHd9wHfLl82s3eBzdV1Fn1QJS8vj7y8\nvISLFTlRlY+wv33Pt5k+rXYj62T2MWHCBB5++GHWr19Pfn4+7dt/PrM6btw4pkyZwpIlS2jatCmF\nhYXs3r07bp9VLxtXWlpacf/gwYOMHj2auXPnMnLkSNLS0vjGN75RcSZIvH3o3Lkzb7zxRqXHtm7d\nWuN1RmO58cYbSUtL48033yQ7O5unnnqK73//+0DkcnVbtmyhrKzsqFDv2rVrrS9XV1xcTHFxca1r\nTCTQVwC9zCwHeA8YA4yNbmBm2cABdz9kZlcDy9y92j9zVY+SN3bJOANBJBEFFxdQsqrkmEbnyerj\niiuu4LbbbmPt2rXMnFn5//f+/ftp3bo1TZs2Zfny5Tz66KPk5+dXrI91OuY3v/lN7rvvPr7+9a+T\nkZHBnXd+fhLdwYMHOXjwIO3atSMtLY3FixezdOlScnNzgcjoevfu3ezdu5esrKxq+77zzjt54YUX\nOO+887j33ntp0aIFAwYMqPW+79u3j1atWpGZmcn27du56667Ktb179+fTp06ccMNN1BUVER6ejqv\nv/46AwcO5N///d/54Q9/yKBBg+jXrx+bNm2iadOmdOvWLea2qg52azrAHC3ulIu7HwEmA0uBN4F5\n7r7OzCaZWfnZ+18C3jCzdUTOhpma0NZDIBlnIIgkwsy4Y8Ydx38JuuPoIycnh4EDB3LgwAEuueSS\nSuvuv/9+br75ZrKzs7ntttu4/PLLj9p2dfevvvpq8vPzOeusszj77LMpKCioWNeyZUvuu+8+Lrvs\nMtq0acO8efMYOfLzczJ69+7N2LFj6dGjB23atGHnzp2Vtnnaaacxd+5cJk+eTPv27Vm0aBFPP/00\nTZo0OaqOeGbMmMHrr79eMdceXWdaWhpPP/00GzdupFu3bnTt2pXHH38cgNGjR/OTn/yEcePGkZWV\nxTe+8Q0+/PDDhLdbG/pg0XFydwZ8cwCvnfEaGOBw7pvn8srjr4Ti/Fi9A6lfDfWDRVI/9H3oKZaM\nswcaMr0DEWk8FOhJkIwzEBqqZJwDLSL1Q9+HngTJOHugoSrft4lPTuRAzoHQvQPRlJKEiQI9SZJx\nBkJDVXBxAXfPuZvX/LXQjc43vPEBLzV/hbIvHqp4LG19UzLfyKnhWSINk6ZckiQZZyA0RIWFd5GX\nV8S+d7qRvqgZ+97pRl5eEYWFd8V/ciNw5dhx+Ms9K00p+cs9uTLqAzMijYUCXWo0aFAuJSUDeWv1\nYxx5o5C3Vj9GSckABg8+M9WlJcXo0UPp2bIHrIsc1GZdBj1b9qSgIL/mJ4o0QAp0qVFBQT65uc9G\nFj69A4Dc3CWMGnVRCqtKHjPj50WTsVe7gYO92o3bb5mcsndaOTk5mJluJ+it6tce1JbOQ5e4Fix4\nlokTjQMH8snIeJZHHrFQjWDdndP6jOCd5i/S67Pz2fDG01iKAl2kOmY6D12S5PNRuodqdF7OLDJK\nb7qpd0pH5yLHS4EucZkZ06blk5n5A6ZPHxrKwBs9eig/uPayUL3zkBOPplwkIe7Oj398F7ffHp5z\n0EUai0SnXBToIiINnObQRUROMAp0EZGQUKCLiISEAl1EJCQU6CIiIaFAFxEJCQW6iEhIKNBFREJC\ngS4iEhIKdBGRkFCgi4iEhAJdRCQkFOgiIiHRpL43mHdlHu5Ov5x+zLx1Zn1vXkQktOp9hL7s1GWU\npJUw+JzB9b1pEZFQq/8pF4fc/bmMGjGq3jctIhJm9R7oGaUZTL9CV70REUm2hALdzIaa2Xoz22Bm\n11ezPsvMFprZajNba2ZXxupLo3MRkboRN9DNLA34FZAPnAGMNbMvVmn2PeBNd/8ycAFwj5lVe8BV\no3MRkbqRyAi9P7DR3Uvd/RAwDxhZpY0DmcH9TGC3ux+urjONzkVE6kYigX4KsDVqeVvwWLRfAaeb\n2Q5gDTA1VmcanYuI1I1knYeeD6xy9wvNrCfwnJmd6e77qzYsKiqquJ+Xl0deXl6SShARCYfi4mKK\ni4tr/Txz95obmH0VKHL3ocHyDYC7+51Rbf4M3O7ufwuWnweud/eSKn15vO2JiEhlZoa7x53eSGTK\nZQXQy8xyzKwZMAZYWKVNKfCvwYY7AqcBm2tXsoiIHI+4Uy7ufsTMJgNLifwBmO3u68xsUmS1zwJu\nAx4ys78HT/uRu/9vnVUtIiJHiTvlktSNacpFRKTWkjnlIiIijYACXUQkJBToIiIhoUAXEQkJBbqI\nSEgo0EVEQkKBLiISEgp0EZGQUKCLiISEAl1EJCQU6CIiIaFAFxEJCQW6iEhIKNBFREJCgS4iEhIK\ndBGRkFCgi4iEhAJdRCQkFOgiIiGhQBcRCQkFuohISCjQRURCQoEuIhISCnQRkZBQoIuIhIQCXUQk\nJBToIiIhoUAXEQmJhALdzIaa2Xoz22Bm11ezfpqZrTKzlWa21swOm1mr5JcrIiKxmLvX3MAsDdgA\nDAF2ACuAMe6+Pkb7EcB17v6v1azzeNsTEZHKzAx3t3jtEhmh9wc2unupux8C5gEja2g/FvhjYmWK\niEiyJBLopwBbo5a3BY8dxcxOAoYCTxx/aSIiUhtNktzfxcBL7v5RrAZFRUUV9/Py8sjLy0tyCSIi\njVtxcTHFxcW1fl4ic+hfBYrcfWiwfAPg7n5nNW3/B3jc3efF6Etz6CIitZToHHoigZ4OvE3koOh7\nwHJgrLuvq9IuG9gMdHH3T2L0pUAXEamlRAM97pSLux8xs8nAUiJz7rPdfZ2ZTYqs9llB00uBJbHC\nXERE6lbcEXpSN6YRuohIrSXztEUREWkEFOgiIiGhQBcRCQkFuohISCT7g0UnnMLCu1i5cj9mnx+v\ncHf69WvJzJnTU1iZiJxoFOjHadCgXGbNMg4cyK94LCPjWaZMiXtAWkQkqTTlcpwKCvLJzX0WKD8d\n08nNXcKoURelsiwROQEp0I+TmTFtWj4ZGUsByMhYwvTpQytNwYiI1AcFehJEj9I1OheRVFGgJ0H5\nKD0z8wcanYtIyuigaJIUFORTUvJ3jc5FJGX0XS4iIg2cvstFROQEo0AXEQkJBbqISEgo0EVEQkKB\nLiISEgp0EZGQUKCLiISEAl1EJCQU6CIiIaFAFxEJCQW6iEhIKNBFREJCgS4iEhIKdBGRkFCgi4iE\nhAJdRCQkEgp0MxtqZuvNbIOZXR+jTZ6ZrTKzN8zsheSWKSIi8cS9YpGZpQEbgCHADmAFMMbd10e1\nyQZeBi5y9+1m1s7dP6imL12xSESklpJ5xaL+wEZ3L3X3Q8A8YGSVNuOAJ9x9O0B1YS4iInUrkUA/\nBdgatbwteCzaaUAbM3vBzFaY2YRkFSgiIolpksR++gEXAv8CvGJmr7j7O1UbFhUVVdzPy8sjLy8v\nSSWIiIRDcXExxcXFtX5eInPoXwWK3H1osHwD4O5+Z1Sb64EW7n5LsPzfwGJ3f6JKX5pDFxGppWTO\noa8AeplZjpk1A8YAC6u0eQoYbGbpZpYBnAusq23RIiJy7OJOubj7ETObDCwl8gdgtruvM7NJkdU+\ny93Xm9kS4O/AEWCWu79Vp5WLiEglcadckroxTbmIiNRaMqdcRESkEVCgi4iEhAJdRCQkFOgiIiGh\nQBcRCQkFuohISCjQRURCQoEuIhISCnQRkZBQoIuIhIQCXUQkJBToIiIhoUAXEQkJBbqISEgo0EVE\nQkKBLiISEgp0EZGQUKCLiISEAl1EJCQU6CIiIaFAFxEJCQW6iEhIKNBFREJCgS4iEhIKdBGRkFCg\ni4iEhAJdRCQkFOgiIiGRUKCb2VAzW29mG8zs+mrW/z8z+8jMVga3m5JfqoiI1KRJvAZmlgb8ChgC\n7ABWmNlT7r6+StO/uvsldVCjiIgkIJERen9go7uXuvshYB4wspp2ltTKRESkVhIJ9FOArVHL24LH\nqhpgZqvNbJGZnZ6U6kREJGFxp1wS9DrQzd0PmNkw4EngtCT1LSIiCUgk0LcD3aKWuwSPVXD3/VH3\nF5vZ/WbWxt3/t2pnRUVFFffz8vLIy8urZckiIuFWXFxMcXFxrZ9n7l5zA7N04G0iB0XfA5YDY919\nXVSbju6+K7jfH3jc3btX05fH256IiFRmZrh73OOUcUfo7n7EzCYDS4nMuc9293VmNimy2mcBo83s\nWuAQ8Alw+fGVLyIitRV3hJ7UjWmELiJSa4mO0PVJURGRkFCgi4iEhAJdRCQkFOgiIiGRrA8WJSwv\nrwh3p1+/lsycOb2+Ny8iElr1fpYLOBkZz/LII0ZBQX69bVtEpLFqwGe5OLm5Sxg16qL637SISIjV\ne6BnZCxh+vShmOnLGUVEkqneA12jcxGRulHvga7RuYhI3aj3g6JlZWUKdBGRWmiwB0UV5iIidUMf\nLBIRCQkFuohISCjQRURCQoEuIhISCnQRkZBQoIuIhIQCXUQkJBToIiIhoUAXEQkJBbqISEgo0EVE\nQkKBLiISEgp0EZGQUKCLiISEAl1EJCQU6CIiIZFQoJvZUDNbb2YbzOz6GtqdY2aHzGxU8koUEZFE\nxA10M0sDfgXkA2cAY83sizHa3QEsSXaRjUVxcXGqS6hT2r/GK8z7BuHfv0QlMkLvD2x091J3PwTM\nA0ZW0+77wALg/STW16iE/T+V9q/xCvO+Qfj3L1GJBPopwNao5W3BYxXMrDNwqbv/BtBFQ0VEUiBZ\nB0XvBaJuoEr4AAADYUlEQVTn1hXqIiL1zNy95gZmXwWK3H1osHwD4O5+Z1SbzeV3gXbAx8A17r6w\nSl81b0xERKrl7nEHyokEejrwNjAEeA9YDox193Ux2j8IPO3u/1PrikVE5Jg1idfA3Y+Y2WRgKZEp\nmtnuvs7MJkVW+6yqT6mDOkVEJI64I3QREWkc6u2Tool+OKkxMrPZZrbLzP6e6lqSzcy6mNlfzOxN\nM1trZlNSXVMymVlzM3vNzFYF+zcj1TXVBTNLM7OVZrYwfuvGxcz+YWZrgp/h8lTXk2xmlm1m881s\nXfB7eG7MtvUxQg8+dLSByDz8DmAFMMbd19f5xuuBmQ0G9gOPuPuZqa4nmczsZOBkd19tZi2B14GR\nYfnZAZhZhrsfCI4X/Q2Y4u6hCgYzKwS+AmS5+yWprieZgpMyvuLuH6a6lrpgZg8By9z9QTNrAmS4\n+97q2tbXCD3RDyc1Su7+EhDK/0zuvtPdVwf39wPrqPI5hMbO3Q8Ed5sTOa4UqnlIM+sCDAf+O9W1\n1BEjpN9LZWZZwHnu/iCAux+OFeZQfy9C3A8nScNnZt2BLwOvpbaS5AqmI1YBO4Hn3H1FqmtKspnA\ndEL2hyqKA8+Z2QozuzrVxSTZqcAHZvZgMGU2y8xOitU4lH/VJPmC6ZYFwNRgpB4a7l7m7n2BLsC5\nZnZ6qmtKFjP7OrAreJdlhPNDf4PcvR+RdyHfC6ZAw6IJ0A/4dbCPB4AbYjWur0DfDnSLWu4SPCaN\nQDBvtwCY4+5PpbqeuhK8lX0BGJrqWpJoEHBJMM/8R+ACM3skxTUllbu/F/z7T+BPRKZ4w2IbsNXd\nS4LlBUQCvlr1FegrgF5mlmNmzYAxQNiOtod19APwe+Atd/+vVBeSbGbWzsyyg/snAV8DQnPA191v\ndPdu7t6DyO/dX9z9ilTXlSxmlhG8e8TM/gW4CHgjtVUlj7vvAraa2WnBQ0OAt2K1j/vBoiQVVe2H\nk+pj2/XBzB4F8oC2ZrYFmFF+EKOxM7NBwLeAtcE8swM3uvuzqa0saToBDwdnYqUBj7n7MymuSRLX\nEfhT8LUiTYA/uPvSFNeUbFOAP5hZU2Az8G+xGuqDRSIiIaGDoiIiIaFAFxEJCQW6iEhIKNBFREJC\ngS4iEhIKdBGRkFCgi4iEhAJdRCQk/g9gDv6XosZ2lwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f7c20344e80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.image as mpimg\n",
    "\n",
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(len(acc))\n",
    "\n",
    "plt.plot(epochs, acc, 'bv', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'gv', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEKCAYAAAACS67iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8VdWZ//HPk3BXEy5q5KIBtKFSraIjqKAcxSogiL+p\nykXBTvsTbAdFHX9KnVpDL1Y6pZ3SWiVTi4oKeBkRBSp26omD17QVh1YgVBAhCKJCwEG5hOf3x9nE\nk5Oc5Jx4kpNsvu/X67y6L2uv/exjebLP2muvZe6OiIi0fjnZDkBERDJDCV1EJCSU0EVEQkIJXUQk\nJJTQRURCQgldRCQklNClmpnlmNluM+uVybLZZGYnmtnBJqh3mJltiFtfY2aDUynbiHP9h5lNb+zx\n9dT7QzP7Xabrlexpk+0ApPHMbDdw6EWCI4C9QFWwbYq7z0+nPnc/CByV6bItQFO9bFFdr7t/ORMx\nmNm3gGvc/YK4uq9rXHhyuFFCb8XcvTqhmtl64Fvu/mKy8maW6+5VzRKcNJbRdH+AJOTU5BIeFnw+\n3xD7Sb3AzB4zs0rgajM728xeNbMdZlZhZr80s9ygfK6ZHTSzE4L1ecH+pWa2y8xeNrPCdMsG+0eY\n2drgvLPNbIWZTarzQlKLcbKZrTOzj8zsl3HH5pjZL8zsQzP7OzA86RdmdoeZzU/Ydq+Z/SxY/paZ\nvR1cz7rg7jlZXZvM7PxguWPwfXxsZquAMxPK/quZvRPUu8rMRgfbTwF+BZwXNGd9EPfdfj/u+OuD\neLab2X+a2XGpfDcNMbP/Y2Z/DeL+g5kVJXxXFWZWGXwnh651kJn9Odj+vpnNTPV80gTcXZ8QfIAN\nwIUJ234IfAaMDNbbE0suZxFL/r2BNcB3gv25xJpsTgjW5wEfAAOCfQuAhxtR9lhgFzAq2Hczseah\nSUmupaEYDwJPA0cChcBHh64dmAqsAroDXYBSoCrJefoEcXWMq3sbMCBYvxQoDJYjwB7glGB9GLA+\nrq5NwPnB8s+APwJ5wPHA3xLKXgEcGyyPA3YDxwTr3wL+mBDnPOD7wfLFwFbgVKAdcC/wX6l8N3Vc\n/w+B3wXLJwdxDA3q+W7wvecC/YF342IsBHoHy28AY4PlI4Czsv1v4XD+6A49/Fa4+1IAd9/r7n92\n9zKPeRf4D2L/iA+xhOOfdPc3PdZU8yhweiPKXgq86e7PuXuVu/+CWKKpUwoxAtzt7p+4+0YgGneu\nK4FfuPv77r4DuKee82wA/gqMCTZdDHzs7m8G+5cE9ePuUeC/gPOS1RfnSuCH7r7L3TcBv04475Pu\n/kGwvIBYsvyHFOoFmAD81t1Xufs+YDow1Mx6xJVJ9t3UZyzwjLuXBv/97gHygUHAAWI3A6cGzXYb\ng/8uAPuAL5lZV3f/X3cvS/E6pAkooYffpvgVM+tnZs8FP48rgRnA0fUcvzVueQ+xO790y/ZIjAPY\nnKySFGPcluK5NtYTL8B8YHywPB54LC6OUWb2WtB0sQP4Wh1x1KU7Na+vRgxm9g0zWxk0bewA+qVY\nL8Sur7o+d98N7AB6xpVJ9t2kU68H19DT3cuBfwF+AGwzs0fNrCAo+k/AV4C1wXc1IsXrkCaghB5+\niQ/Y5hBrkujr7vnAXdS+086094k1PcTrWVfBwBeJMfFchckKBh4HLgrucMcQJHQz6wA8AfyYWFND\nF+CFFOPYmiwGM+sD/IZYL6SuQb1r4+pt6IHoloT6jiLWtJT0D2SKEus1oBdQAeDuj7n7EGLNVG2A\nu4Pt69x9vLsfA/wceMrM2n3BWKSRlNAPP0cBle7+qZmdDExphnM+Bwwws0uDB3c3Uf8d6ReJ8XHg\nJjPrYWbdgNvqK+zu24CXgQeBNe7+TrCrPdAW+BBwMxtFrN081RjuMLP84KHxP8ftO5JYO/eHwXdx\nHRDf5XEb0MvMkvVAmw98y8xOMbP2wE+Al9z9/RRjqy/my8zs/ODctxF7vvC6mX3ZzCJBot4LfBpc\nA2Z2TfA9E5Q/eGifND8l9PBItavbvwDfMLNdwH3EHl4mq6ehOlMqG7QXjwV+QSxB9gHeJJYcvmiM\niev3EWvrXgW8TuwuuyGPEUvWj8bFXEns4e0iYu39/wg8W08d8THcRewu/V1gCfBQXL2riPVkKSN2\nV/wl4LW4Y18A1hFr2thS6yTuzxNr+lhE7O65F3B1kjjqWq87ePe3gWuB+4k93L4YuCxoT28P/BTY\nHsTcGfjX4NCRwOqgaeynwFXufiCVc0rmWayprJ4CZg8Q652wzd2/mqTMbGAE8L/AN9x9ZaYDlfAw\nsxxiieHr7v5ytuMRCYtU7tDnApck2xk8BDnR3b9E7Kfx/RmKTULEzC4JmiDaA98n1jvijSyHJRIq\nDSZ0d19B7Cl6MmOAh4OyrwP5cU/ARQ4ZAqwn1kb8NeByd9+f3ZBEwiUTr/73pGY3sYpg27a6i8vh\nyN3vBO7MdhwiYaaHoiIiIZGJO/QKava5re67msjMNOiQiEgjuHuD70Ckeodea+CnOIuBSRAbVAnY\nGfTtTRZUaD933XVX1mPQ9en6DrdrOxyuL1UN3qGb2WPEBibqZmbvEetj2y6Wm73E3Zea2chgZLv/\nJfYqcFKRSDEARUUdKCnJ+Jj9IiKHrQYTurtPSKHM1FRPWFpaHCwV11NKRETSpYeiGRSJRLIdQpPS\n9bVeYb42CP/1parBN0UzejIzP/Qm8tChxUSjxc12bhGR1srM8BQeimoKOpEs6927Nxs3NjTKrxwO\nCgsLeffddxt9vBK6SJZt3LgxrZ4MEl6xUYsbr9kT+tChxUCsl4uIiGROs7eh605EpKagfTTbYUgL\nkOz/C6m2oauXi4hISCihi0izOHjwIEcddRSbNzc8W146ZdN155138s1vfjPj9bYESugiUqejjjqK\nvLw88vLyyM3NpVOnTtXb5s+fn3Z9OTk57N69m169emW0rHxOvVxEWqDJk++hvPyzWtvTGTLji9ax\ne/fu6uW+ffvywAMPcMEFFyQtX1VVRW5ubkqxSdPQHbpIC1Re/hmlpcW1PnUl6Kas45C6Bom68847\nGTduHBMmTCA/P59HH32U1157jXPOOYcuXbrQs2dPpk2bRlVVFRBL+Dk5Obz33nsATJw4kWnTpjFy\n5Ejy8vIYPHhwdX/8dMoCLFu2jH79+tGlSxduvPFGhgwZwsMPP5zStT399NOccsopdO3alYsuuojy\n8vLqfXfffTc9e/YkPz+f/v3789JLLwHw+uuvc+aZZ5Kfn0/37t25/fbb0/5Om4ISuog02qJFi7jm\nmmuorKxk7NixtG3bltmzZ/Pxxx/z8ssv8/zzzzNnzpzq8on9rOfPn8+Pf/xjduzYwfHHH8+dd96Z\ndtkPPviAsWPHMmvWLD788EP69OlDWVlZSvGvXr2aSZMmce+997J9+3aGDRvGZZddRlVVFW+//TYl\nJSWsXLmSyspKli1bxgknnADADTfcwG233UZlZSV///vfueKKKxr1/WWaErqINNqQIUMYOXIkAO3b\nt+fMM8/krLPOwszo3bs31113HaWlpdXlE+/yr7jiCgYMGEBubi5XX301K1euTLvskiVLGDBgAKNG\njSI3N5ebb76Zbt26pRT/woULGTNmDEOHDiU3N5fp06dTWVnJ66+/Tps2bdi7dy+rVq2iqqqKwsJC\nevfuDUC7du1Yt24dH3/8MUcccQRnnXVW2t9dU1BCF5FGO/7442usr127llGjRtG9e3fy8/O56667\n+PDDD5Mef9xxx1Uvd+rUiU8++STtslu2bKkVR6oPU7ds2UJhYWH1upnRq1cvKioqKCoqYtasWXz/\n+9+noKCAq6++mm3bYlM9zJ07l7/97W/069ePs88+m2XLlqV0vqamhC4ijZbYLDJlyhROPfVU1q9f\nT2VlJTNmzGjyl6a6d+/Opk2bamyrqKhz0rRaevToUaMt3t3ZvHkzPXv2BGDChAmsWLGCDRs2cODA\nAe644w4AvvSlLzF//ny2b9/OLbfcwte//nX27duXoStqPPVyEWmBYkNjFCfZ3nx1pGv37t3k5+fT\nsWNHVq9ezZw5c5q86+GoUaO46aabWLJkCcOHD+dXv/pVvb8K4l111VWcffbZvPTSS5x77rnMmjWL\nvLw8Bg0axJo1a9i6dSvnnnsu7du3p2PHjuTkxO6BH3nkEUaMGEG3bt3Iy8sjJyenel82KaGLtECZ\nmM0rkzOCpTpo1KxZs7j++uu5++67OeOMMxg3bhwrVqyos56G6ky17LHHHsvChQuZNm0a11xzDZMm\nTWLAgAG0b9++wXj79+/PQw89xPXXX8/WrVsZMGAAixcvJjc3l71793Lbbbexdu1a2rZty5AhQygp\nKQFg6dKl3HLLLezdu5fCwkIef/xx2rTJfjrVWC4iWaaxXDLr4MGD9OjRg6eeeorBgwdnO5y0aCwX\nETnsPf/881RWVrJ3715+8IMf0K5dOwYOHJjtsJpdSgndzIab2RozKzezWj3ozayzmf2nmb1lZq+Z\nWf/MhyoiUrcVK1bQt29fCgoKeOGFF1i0aBFt27bNdljNrsEmFzPLAcqBYcAWoAwY5+5r4sr8FNjt\n7j80s37Ave5+UR11qclFJIGaXOSQ5mhyGQisc/eN7r4fWACMSSjTH/gjgLuvBXqb2TEp1C0iIhmS\nSkLvCcR38twcbIv3FvCPAGY2EDgB0DBpIiLNKFP9bO4BfmlmfwFWAW8CVXUVLC4url6ORCJEIpEM\nhSAiEg7RaJRoNJr2cam0oZ8NFLv78GB9OuDuPrOeYzYAp7r7Jwnb1YYukkBt6HJIc7ShlwEnmVmh\nmbUDxgGLE06Wb2Ztg+XrgNLEZC4iIk2rwYTu7lXAVGA58DdggbuvNrMpZjY5KHYy8FczWw1cAkxr\nqoBFpHXYuHEjOTk5HDx4EICRI0cyb968lMqm6yc/+QmTJ09uuGCaHnroIc4777yM19tUUmpDd/ff\nA/0Sts2JW34tcb+ItG4jRoxg0KBBNZ57ATzzzDNcf/31VFRUNDh+Sfwr+0uXLk25bH1KS0u55ppr\nagzI9d3vfjelYxsj1bhaguwPPiAitUy+fTLl28prbS8qKKJkZkmz1HHttdfyve99r1ZCf+SRR5g4\ncWLWBqNy91aVZJuTXv0XaYHKt5VT2qe01qeuBN1UdVx++eV89NFHNQbX2rlzJ8899xyTJk0CYnfd\nZ5xxBvn5+RQWFjJjxoyk9V1wwQX87ne/A2Ljrdx6660cc8wxnHTSSSxZsqRG2QcffJD+/fuTl5fH\nSSedVD0o1p49exg5ciRbtmypnrB669atzJgxg4kTJ1Yfv3jx4upp5S688ELWrKl+D5I+ffowa9Ys\nTjvtNLp06cL48eNTHvr2lVdeYeDAgXTp0oVBgwbx6quv1oj5xBNPJC8vjxNPPLF6Iu133nmHSCRC\n586dOfbYYxk/fnxK52oMJXQRqVOHDh248sora8zNuXDhQk4++WROOeUUAI488kjmzZtHZWUlS5Ys\n4f7772fx4sXJqqxWUlLC0qVLeeutt/jTn/7Ek08+WWN/QUEBS5cuZdeuXcydO5ebb76ZlStX0qlT\nJ5YtW0aPHj3YvXs3u3btqp744tBde3l5ORMmTGD27Nls376dESNGMHr0aA4cOFBd/xNPPMHy5cvZ\nsGEDb731Fg8++GCDMe/YsaN6qN6PPvqIm2++mUsvvZQdO3awZ88epk2bxvPPP8+uXbt45ZVXOP30\n04HY3KuXXHIJO3fuZPPmzdxwww0NnquxlNBFJKlrr72WJ554ovoOdt68eVx77bXV+88//3y+8pWv\nAHDKKacwbty4GlPOJfPEE09w00030aNHDzp37lyrDXzEiBHV072dd955XHzxxfz3f/93SjE//vjj\njBo1igsvvJDc3FxuvfVWPv30U1555ZXqMtOmTaOgoIDOnTszevToGlPfJbNkyRKKioqYMGECOTk5\njBs3ji9/+cs8++yzAOTm5rJq1So+++wzCgoKOPnkkwFo27YtGzdupKKignbt2nHuueemdB2NoYQu\nIkkNHjyYY445hkWLFrF+/XrKysqYMGFC9f433niDCy+8kGOPPZbOnTszZ86clCaXSJw2Ln4aOIBl\ny5Zxzjnn0K1bN7p06cKyZctSnrSirmnljj/++BqzGBUUFFQvNzT1XbJ6D8VdUVFBp06dWLhwIffd\ndx/du3dn9OjRrF27FoB/+7d/4+DBgwwcOJBTTz2VuXPnpnQdjaGELiL1mjhxIg899BCPPPIIl1xy\nCccc8/kwTRMmTODyyy+noqKCnTt3MmXKlJRekkqcNi5+Grh9+/ZxxRVXcNttt7F9+3Z27NjBiBEj\nqutt6IFo4rRyAJs2bfrCMyf16NGDd999t8a29957r3q6uq997WssX76crVu30q9fP6677jogNgFH\nSUkJFRUV3H///XznO99h/fr1XyiWZJTQRVqgooIihm4YWutTVFDUrHUATJo0iT/84Q/89re/rdHc\nAvDJJ5/QpUsX2rZtyxtvvMFjjz1WY3+y5H7VVVcxe/ZsKioq2LFjBzNnfv7i+b59+9i3bx9HH300\nOTk5LFu2jOXLl1fvLygo4KOPPmLXrl1J616yZAkvvvgiBw4c4Gc/+xkdOnTgnHPOSeu6E40cOZJ1\n69axYMECqqqqWLhwIatXr2bUqFF88MEHLF68mD179tC2bVuOPPJIcnNzAXjyySerfx107ty5Saer\nU7dFkRYo1a6JTV0HxJoVzj33XFatWsVll11WY99vfvMbbrnlFqZOncrQoUMZO3YsO3furN6fbBq5\n6667jnXr1nHaaaeRn5/PrbfeyosvvgjEHrTOnj2bK6+8kn379jF69GjGjPl8gNd+/foxfvx4+vbt\ny8GDB3n77bdrxFRUVMQjjzzC1KlT2bJlC6effjrPPvts9RRxje3y2LVrV5577jluvPFGvv3tb1f3\nzunatStbt27l5z//Oddeey1mxumnn859990HQFlZGTfddBO7du2ioKCA2bNnVz8fyDRNQSeSZRrL\nRQ7RFHQiIgIooYuIhIYSuohISCihi4iEhBK6iEhIKKGLiISE+qGLZFlhYaGGgxWg9hAI6VI/dBGR\nFk790EVEDjNK6CIiIZFSQjez4Wa2xszKzez2OvbnmdliM1tpZqvM7BsZj1REROrVYBu6meUA5cAw\nYAtQBoxz9zVxZb4L5Ln7d83saGAtUODuBxLqUhu6iEiaMtmGPhBY5+4b3X0/sAAYk1DGgaOC5aOA\njxKTuYiINK1UEnpPYFPc+uZgW7xfA/3NbAvwFjAtM+GJiEiqMtUP/RLgTXe/0MxOBF4ws6+6e615\nnYqLi6uXI5EIkUgkQyGIiIRDNBolGo2mfVwqbehnA8XuPjxYnw64u8+MK/Mc8BN3fzlY/y/gdnf/\nU0JdakMXEUlTJtvQy4CTzKzQzNoB44DFCWU2AhcFJy4AioCmmTRPRETq1GCTi7tXmdlUYDmxPwAP\nuPtqM5sS2+0lwI+AB83sf4LDbnP3j5ssahERqUWv/ouItHB69V9E5DCjhC4iEhJK6CIiIaGELiIS\nEkroIiIhoYQuIhISSugiIiGhhC4iEhJK6CIiIaGELiISEkroIiIhkanx0FMW+UYEgKKCIkpmljT3\n6UVEQqvZE3ppn9LYwobmPrOISLipyUVEJCSU0EVEQkIJXUQkJJTQRURCotkfig7dMBSI9XIREZHM\n0RR0IiItXEanoDOz4Wa2xszKzez2OvbfamZvmtlfzGyVmR0ws86NCVxERBqnwTt0M8sByoFhwBag\nDBjn7muSlB8F3OTuF9WxT3foIiJpyuQd+kBgnbtvdPf9wAJgTD3lxwPzUwtTREQyJZWE3hPYFLe+\nOdhWi5l1BIYDT33x0EREJB2Z7uUyGljh7juTFSguLq5ejkQiRCKRDIcgItK6RaNRotFo2sel0oZ+\nNlDs7sOD9emAu/vMOsr+J/C4uy9IUpfa0EVE0pRqG3oqCT0XWEvsoej7wBvAeHdfnVAuH1gP9HL3\nT5PUpYQuIpKmVBN6g00u7l5lZlOB5cTa3B9w99VmNiW22w+NgXs58HyyZC4iIk1LLxaJiLRwGX2x\nSEREWj4ldBGRkFBCFxEJCSV0EZGQUEIXEQkJJXQRkZBQQhcRCQkldBGRkFBCFxEJCSV0EZGQUEIX\nEQkJJXQRkZBQQhcRCYlMz1h02Jk8+R7Kyz+rtb2oqAMlJdOzEJGIHK6U0L+g8vLPKC0trmNPXdtE\nRJqOmlxEREJCCV1EJCSU0EVEQkIJXUQkJFJ6KGpmw4F/5/NJomfWUSYC/AJoC2x39wsyGGeLVVTU\ngboegMa2i4g0nwYniTazHKAcGAZsAcqAce6+Jq5MPvAKcLG7V5jZ0e7+YR11aZJoEZE0ZXKS6IHA\nOnff6O77gQXAmIQyE4Cn3L0CoK5kLiIiTSuVhN4T2BS3vjnYFq8I6GpmL5pZmZlNzFSAIiKSmky9\nWNQGOAO4EDgCeNXMXnX3vycWLC4url6ORCJEIpEMhSAiEg7RaJRoNJr2cam0oZ8NFLv78GB9OuDx\nD0bN7Hagg7vPCNZ/Cyxz96cS6lIbuohImjLZhl4GnGRmhWbWDhgHLE4o8wwwxMxyzawTMAhYnW7Q\nIiLSeA02ubh7lZlNBZbzebfF1WY2JbbbS9x9jZk9D/wPUAWUuPvbTRq5iIjU0GCTS0ZPpiYXEZG0\nZbLJRUREWgEldBGRkFBCFxEJCSV0EZGQUEIXEQkJJXQRkZBQQhcRCQkldBGRkFBCFxEJCSV0EZGQ\nUEIXEQkJJXQRkZBQQhcRCQkldBGRkFBCFxEJCSV0EZGQUEIXEQkJJXQRkZBQQhcRCYmUErqZDTez\nNWZWbma317F/qJntNLO/BJ/vZT5UERGpT5uGCphZDvBrYBiwBSgzs2fcfU1C0Zfc/bImiFFERFKQ\nyh36QGCdu2909/3AAmBMHeUanJFaRESaTioJvSewKW59c7At0TlmttLMlphZ/4xEJyIiKWuwySVF\nfwZOcPc9ZjYCWAQU1VWwuLi4ejkSiRCJRDIUgohIOESjUaLRaNrHmbvXX8DsbKDY3YcH69MBd/eZ\n9RyzATjT3T9O2O4NnU9ERGoyM9y9wWbtVJpcyoCTzKzQzNoB44DFCScriFseSOwPxceIiEizabDJ\nxd2rzGwqsJzYH4AH3H21mU2J7fYS4Aoz+zawH/gUGNuUQYuISG0NNrlk9GRqchERSVsmm1xERKQV\nUEIXEQkJJXQRkZBQQhcRCQkldBGRkFBCFxEJCSV0EZGQUEIXEQkJJXQRkZBQQhcRCQkldBGRkFBC\nFxEJCSV0EZGQUEIXEQkJJXQRkZBQQhcRCQkldBGRkFBCFxEJCSV0EZGQSCmhm9lwM1tjZuVmdns9\n5c4ys/1m9o+ZC1FERFLRpqECZpYD/BoYBmwByszsGXdfU0e5e4DnmyLQlmry7ZMp31Zea3tRQREl\nM0uyEJGIHK4aTOjAQGCdu28EMLMFwBhgTUK5G4AngbMyGmELV76tnNI+pbV3bGj+WETk8JZKQu8J\nbIpb30wsyVczsx7A5e5+gZnV2Cet2+TJ91Be/lmt7UVFHSgpmZ6FiEQkmVQSeir+HYhvW7dkBYuL\ni6uXI5EIkUgkQyFIUygv/4zS0uI69tS1TUQyIRqNEo1G0z4ulYReAZwQt94r2BbvH4AFZmbA0cAI\nM9vv7osTK4tP6CIiUlvize6MGTNSOi6VhF4GnGRmhcD7wDhgfHwBd+97aNnM5gLP1pXMRUSk6TSY\n0N29ysymAsuJdXN8wN1Xm9mU2G5P7MrhTRBni1VUUFTnA9CigqLmD0ZEDmsptaG7+++Bfgnb5iQp\n+80MxNVqqGuiiLQUmXooKiFVVNSBuh6AxraLSEti7s3XQmJm3pznExEJAzPD3ZP2HjxEY7mIiISE\nErqISEioDV3qpbFqRFoPJXSpl8aqEWk91OQiIhISSugiIiGhhC4iEhJK6CIiIaGHolKvsI9Vo148\nEiZ6U1QOa93P6cvW4bX/Yh33+z68/+r6LEQkUpveFBVJwaefHkxru0hLpoQuIhISSugiIiGhhC4i\nEhLq5SKHtY77ulE5t3ft7Z12N38wIl+QEroc1kYPuZLy8s9qbdcEHtIapdRt0cyGA//O53OKzkzY\nfxnwQ+AgsB+42d1frqMedVsUEUlTqt0WG0zoZpYDlAPDgC1AGTDO3dfElenk7nuC5VOBx9395Drq\nUkIXEUlTJvuhDwTWuftGd98PLADGxBc4lMwDRxK7UxcRkWaUSkLvCWyKW98cbKvBzC43s9XAs8A3\nMxOeiIikKmPdFt19UdDMcjnwo0zVKyIiqUmll0sFcELceq9gW53cfYWZ9TWzru7+ceL+4uLi6uVI\nJEIkEkk5WBFJ3eTJ9yTtwVNSMj0LEUmqotEo0Wg07eNSeSiaC6wl9lD0feANYLy7r44rc6K7vxMs\nnwE84+7H11GXHoqKNJNIpJjS0uJa24cOLSYarb1dWq5UH4o2eIfu7lVmNhVYzufdFleb2ZTYbi8B\nvm5mk4B9wKfAVV8sfBERSVdKLxa5+++Bfgnb5sQt/xT4aWZDExGRdGgsFxGRkFBCFxEJCY3lIhJS\nsfFoipNslzDSFHQiIi2cpqATETnMKKGLiISEErqISEgooYuIhIQSuohISCihi4iEhBK6iEhIKKGL\niISE3hQVkVZJ473XpoQuIq1SeflndY73XtdwB4cLvfovIq1S9/5nsnXPUbW2H9dpN++//ecsRNR0\nMjbBhYhIS/Rpu49g7F9qb3+6MAvRtAxK6CIhNfn2yZRvK6+1vaigiJKZJVmISJqaErpISJVvK6e0\nT2ntHRuaPxZpHkroItIqdeyYQ2WS7WGQ7BdWfVK6cjMbbmZrzKzczG6vY/8EM3sr+Kwws1PTikJE\nJE39+p2Q1vbW5tAvrDp/ZSXRYEI3sxzg18AlwFeA8Wb25YRi64Hz3f004EfAf6QcQYhEo9Fsh9Ck\ndH2tWAibWYoKihi6YShDNwzltFdPq14uKijKdmgZsXbte2kfk0qTy0BgnbtvBDCzBcAYYM2hAu7+\nWlz514B73JGhAAADzUlEQVSeaUcSAtFolEgkku0wmoyurxV7F+iT7SAyK/7BbnFxMcXFxdkLpgl8\n+unBtI9JJaH3BDbFrW8mluST+b/AsrQjEZGMKiooqr4zf3fnu/Te0Pvz7RJKGX0oamYXAP8EDMlk\nvSKSvrDfwUptDb4pamZnA8XuPjxYnw64u89MKPdV4ClguLu/k6QuvSYqItIImXpTtAw4ycwKgfeB\nccD4+AJmdgKxZD4xWTJPNSAREWmcBhO6u1eZ2VRgObFeMQ+4+2ozmxLb7SXAnUBX4DdmZsB+d6+v\nnV1ERDKsWQfnEhGRptNsr1Q19HJSa2ZmD5jZNjP7n2zHkmlm1svM/mhmfzOzVWZ2Y7ZjyiQza29m\nr5vZm8H13ZXtmJqCmeWY2V/MbHG2Y8k0M3s3eKnxTTN7I9vxZJqZ5ZvZE2a2Ovh3OChp2ea4Qw9e\nTioHhgFbiLXLj3P3NfUe2EqY2RDgE+Bhd/9qtuPJJDM7DjjO3Vea2ZHAn4ExYflvB2Bmndx9j5nl\nAi8DN7p7qBKDmd0MnAnkuftl2Y4nk8xsPXCmu+/IdixNwcweBErdfa6ZtQE6ufuuuso21x169ctJ\n7r4fOPRyUii4+woglP9ncvet7r4yWP4EWE3IXhxz9z3BYntiz5VC1Q5pZr2AkcBvsx1LEzFCOp2m\nmeUB57n7XAB3P5AsmUPzfQl1vZwUqqRwODCz3sDpwOvZjSSzguaIN4GtwAvuXpbtmDLsF8D/I2R/\nqOI48IKZlZnZddkOJsP6AB+a2dygyazEzDomKxzKv2qSeUFzy5PAtOBOPTTc/aC7DwB6AYPMrH+2\nY8oUM7sU2Bb8yrLgEzaD3f0MYr9C/jloAg2LNsAZwL3BNe4Bkk6Y2lwJvQKIHwKtV7BNWoGg3e5J\nYJ67P5PteJpK8FP2RWB4tmPJoMHAZUE783zgAjN7OMsxZZS7vx/873bgaeofmqS12Qxscvc/BetP\nEkvwdWquhF79cpKZtSP2clLYnraH9e4H4HfA2+7+y2wHkmlmdrSZ5QfLHYGvETfwXGvn7ne4+wnu\n3pfYv7s/uvukbMeVKWbWKfj1iJkdAVwM/DW7UWWOu28DNpnZoQF4hgFvJyvfLBNcJHs5qTnO3RzM\n7DEgAnQzs/eAuw49xGjtzGwwcDWwKmhnduAOd/99diPLmO7AQ0FPrBxgobsvzXJMkroC4OlgWJE2\nwKPuvjzLMWXajcCjZtaW2FDl/5SsoF4sEhEJCT0UFREJCSV0EZGQUEIXEQkJJXQRkZBQQhcRCQkl\ndBGRkFBCFxEJCSV0EZGQ+P9pr69/NyPlkAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f7c204102e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(epochs, loss, 'bs', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'gs', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "plt.show()\n",
    "display()\n",
    "plt.clf()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use our Pony CNN to score test images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.preprocessing import image\n",
    "import matplotlib.image as mpimg\n",
    "import numpy as np\n",
    "from keras.models import load_model\n",
    "\n",
    "def predict_image(image_url):\n",
    "  img = image.load_img(img_path, target_size=(150, 150))\n",
    "  x = image.img_to_array(img)\n",
    "  x = np.expand_dims(x, axis = 0)\n",
    "  x /= 255\n",
    "\n",
    "  model = load_model('my_little_pony_scratch.h5')\n",
    "\n",
    "  # retrieve the most likely result, e.g. highest probability\n",
    "\n",
    "  y_prob = model.predict(x) \n",
    "  y_classes = y_prob.argmax(axis=-1)\n",
    "\n",
    "  class_name = [k for k, v in label_map.items() if v == y_classes[0]]\n",
    "  # print the classification\n",
    "  print('Prediction: class: %s, score: (%.5f)' % (class_name[0], y_prob[0][0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------CUSTOM MODEL -------------------\n",
      "processing image: images/ponies/test/Applejack1.png\n",
      "Prediction: class: Applejack, score: (0.99425)\n",
      "-----------------------------------------------------\n",
      "processing image: images/ponies/test/Applejack2.png\n",
      "Prediction: class: Applejack, score: (0.99881)\n",
      "-----------------------------------------------------\n",
      "processing image: images/ponies/test/Applejack3.png\n",
      "Prediction: class: Applejack, score: (0.99941)\n",
      "-----------------------------------------------------\n",
      "processing image: images/ponies/test/RainbowDash-1.png\n",
      "Prediction: class: Rainbow_Dash, score: (0.00088)\n",
      "-----------------------------------------------------\n",
      "processing image: images/ponies/test/RainbowDash2.png\n",
      "Prediction: class: Rainbow_Dash, score: (0.00841)\n",
      "-----------------------------------------------------\n",
      "processing image: images/ponies/test/Twilight-Sparkle-1.png\n",
      "Prediction: class: Twilight_Sparkle, score: (0.00020)\n",
      "-----------------------------------------------------\n",
      "processing image: images/ponies/test/Twilight2.png\n",
      "Prediction: class: Twilight_Sparkle, score: (0.00022)\n",
      "-----------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "#print(label_map)\n",
    "img_folder = \"images/ponies/test/\"\n",
    "print('----------------CUSTOM MODEL -------------------')\n",
    "for filename in os.listdir(img_folder):\n",
    "      img_path = os.path.join(img_folder,filename)\n",
    "      print ('processing image:', img_path)\n",
    "      predict_image(img_path)\n",
    "      print('-----------------------------------------------------')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The End!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}