3b - Saving a Model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 3b - Saving a Model\n",
    "\n",
    "In this notebook we will cover the following topics:\n",
    "\n",
    "* Saving a model to disk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "np.warnings.filterwarnings('ignore')  # Hide np.floating warning\n",
    "\n",
    "import keras\n",
    "\n",
    "from keras.datasets import cifar10\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout, Flatten\n",
    "from keras.layers import Conv2D, MaxPooling2D\n",
    "\n",
    "# Prevent TensorFlow from grabbing all the GPU memory\n",
    "import tensorflow as tf\n",
    "\n",
    "gpu_devices = tf.config.experimental.list_physical_devices('GPU')\n",
    "for device in gpu_devices:\n",
    "    tf.config.experimental.set_memory_growth(device, True)\n",
    "\n",
    "import holoviews as hv\n",
    "hv.extension('bokeh')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.datasets import cifar10\n",
    "import keras.utils\n",
    "\n",
    "(x_train, y_train), (x_test, y_test) = cifar10.load_data()\n",
    "\n",
    "# Save an unmodified copy of y_test for later, flattened to one column\n",
    "y_test_true = y_test[:,0].copy()\n",
    "\n",
    "x_train = x_train.astype('float32')\n",
    "x_test = x_test.astype('float32')\n",
    "x_train /= 255\n",
    "x_test /= 255\n",
    "\n",
    "num_classes = 10\n",
    "y_train = keras.utils.to_categorical(y_train, num_classes)\n",
    "y_test = keras.utils.to_categorical(y_test, num_classes)\n",
    "\n",
    "# The data only has numeric categories so we also have the string labels below \n",
    "cifar10_labels = np.array(['airplane', 'automobile', 'bird', 'cat', 'deer', \n",
    "                           'dog', 'frog', 'horse', 'ship', 'truck'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the Model\n",
    "\n",
    "Let's quickly train our simple model so we can save the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential()\n",
    "model.add(Conv2D(32, kernel_size=(3, 3),\n",
    "                 activation='relu',\n",
    "                 input_shape=x_train.shape[1:]))\n",
    "model.add(Conv2D(64, (3, 3), activation='relu'))\n",
    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "model.add(Dropout(0.25))\n",
    "model.add(Flatten())\n",
    "model.add(Dense(128, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(num_classes, activation='softmax'))\n",
    "\n",
    "model.compile(loss=keras.losses.categorical_crossentropy,\n",
    "              optimizer=keras.optimizers.Adadelta(),\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "history = model.fit(x_train, y_train,\n",
    "          batch_size=128,\n",
    "          epochs=10,\n",
    "          verbose=1,\n",
    "          validation_data=(x_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Saving the model is as easy as calling the `save()` method.  This records the weights and the structure of the model so that it can be recreated by another program entirely from this file.  (Assuming that program is using the same version of Keras.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save('cifar10_model.hdf5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the file format is HDF5, which is a common data format for numerical data.  Keras requires the `h5py` Python package be present in order to read and write HDF5 files.\n",
    "\n",
    "Depending on the number of weights in the model, this file can get very big:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! ls -lh cifar10_model.hdf5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can poke around to see the structure of it using the HDF5 command line tools:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! h5ls -r cifar10_model.hdf5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interestingly, we can see that the HDF5 file also records the state of the optimizer, so that we can resume training from a saved model.  This is a useful way to checkpoint your work.  In fact, Keras has a [ModelCheckpoint callback](https://keras.io/callbacks/#modelcheckpoint) that does this automatically after every epoch."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Experiments to Try\n",
    "\n",
    "* Try using the `ModelCheckpoint` callback to save the model in every epoch.\n",
    "\n",
    "If you screw everything up, you can use File / Revert to Checkpoint to go back to the first version of the notebook and restart the Jupyter kernel with Kernel / Restart."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}