deploy_pytorch_model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploy fast.ai model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import os\n",
    "import random\n",
    "from io import BytesIO\n",
    "import subprocess\n",
    "from glob import glob\n",
    "\n",
    "from PIL import Image\n",
    "import requests\n",
    "import boto3\n",
    "\n",
    "import sagemaker\n",
    "from sagemaker.predictor import RealTimePredictor, json_deserializer\n",
    "from sagemaker.pytorch import PyTorchModel\n",
    "from sagemaker.utils import name_from_image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! if [ -e ~/.aws/credentials ]; then rm ~/.aws/credentials; fi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sagemaker_session = sagemaker.Session()\n",
    "\n",
    "bucket_name = sagemaker_session.default_bucket()\n",
    "prefix = 'sagemaker/DEMO-shirts-classification'\n",
    "\n",
    "role = sagemaker.get_execution_role()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure docker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make sure docker is configured for local training "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "pushd utils\n",
    "bash setup.sh\n",
    "popd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Host locally"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create local model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `PyTorch` model uses a npy serializer and deserializer by default. For this example, since we have a custom implementation of all the hosting functions and plan on using JSON instead, we need a predictor that can serialize and deserialize JSON."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ImagePredictor(RealTimePredictor):\n",
    "    def __init__(self, endpoint_name, sagemaker_session):\n",
    "        super(ImagePredictor, self).__init__(endpoint_name, sagemaker_session=sagemaker_session, serializer=None, \n",
    "                                            deserializer=json_deserializer, content_type='image/jpeg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since hosting functions implemented outside of train script we can't just use estimator object to deploy the model. Instead we need to create a `PyTorchModel` object using the latest training job to get the S3 location of the trained model data. Besides model data location in S3, we also need to configure `PyTorchModel` with the script and source directory (because our `serve.py` script requires model and data classes from source directory), an IAM role."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "if [ ! -e ~/.aws/credentials ]; then\n",
    "    echo \"Writing new credentials file\"\n",
    "    accesskey=$(aws secretsmanager get-secret-value --secret-id \"SageMakerNbAccessKey\" --query 'SecretString' --output text)\n",
    "    secretkey=$(aws secretsmanager get-secret-value --secret-id \"SageMakerNbSecretKey\" --query 'SecretString' --output text)\n",
    "\n",
    "    cat > ~/.aws/credentials <<EOF\n",
    "[default]\n",
    "aws_access_key_id=${accesskey}\n",
    "aws_secret_access_key=${secretkey}\n",
    "EOF\n",
    "\n",
    "else\n",
    "    echo \"Credentials file already exists\"\n",
    "fi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = PyTorchModel(model_data='s3://sagemaker-eu-west-1-934676248949/fastai-shirts-2018-12-15-16-00-44-721/model.tar.gz',\n",
    "                     role=role,\n",
    "                     entry_point='src/shirts-jit/serve.py',\n",
    "                     predictor_cls=ImagePredictor,\n",
    "                     framework_version='1.0.0')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deploy model locally"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now call `deploy()` with an instance_count and instance_type, which is `1` and `local`. This invokes our fast.ai container with `'serve'`, which setups our container to handle prediction requests as defined [here](https://github.com/aws/sagemaker-pytorch-container/blob/master/src/sagemaker_pytorch_container/serving.py#L103). What is returned is a predictor, which is used to make inferences against our trained model.\n",
    "\n",
    "After our prediction, we can delete our endpoint.\n",
    "\n",
    "We recommend testing and training your training algorithm locally first, as it provides quicker iterations and better debuggability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = model.deploy(initial_instance_count=1, instance_type='local')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Infer locally"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are ready to call our locally deployed endpoint to test it is working fine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Motorhead T-Shirt\n",
    "#url = 'https://images.backstreetmerch.com/images/products/bands/clothing/mthd/bsi_mthd281.jpg'\n",
    "# Judas Priest T-Shirt\n",
    "#url = 'https://thumbs2.ebaystatic.com/d/l225/m/m7Lc1qRuFN3oFIlQla5V0IA.jpg'\n",
    "# Iron Maiden T-Shirt\n",
    "url = 'https://www.ironmaidencollector.com/assets/pages/ab9ed-20180815_121042.jpg'\n",
    "# All Blacks rugby jersey\n",
    "#url = 'https://images.sportsdirect.com/images/products/38153703_l.jpg'\n",
    "# Australia Rugby T-Shirt\n",
    "#url = 'https://www.lovell-rugby.co.uk/products/products_580x387/40378.jpg'\n",
    "# Chicago Bulls top\n",
    "#url = 'https://i.ebayimg.com/images/g/qc0AAOSwBahVN~qm/s-l300.jpg'\n",
    "# Masters Golf Shirt\n",
    "#url = 'https://s-media-cache-ak0.pinimg.com/originals/29/6a/15/296a15200e7dd3ed08e12d9052ea4f97.jpg'\n",
    "img_bytes = requests.get(url).content\n",
    "img = Image.open(BytesIO(img_bytes))\n",
    "img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = predictor.predict(img_bytes)\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cleanup Endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Host model with SageMaker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deploy model to SageMaker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will take the PyTorch specific model created earlier and call the `deploy()` method giving the different instance type so that it will be deployed to the SageMaker hosting service. The instance type does not need to be a GPU instance, a CPU is perfectly fine for model inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = PyTorchModel(model_data='s3://sagemaker-eu-west-1-934676248949/fastai-shirts-2018-12-15-16-00-44-721/model.tar.gz',\n",
    "                     role=role,\n",
    "                     entry_point='src/shirts-jit/serve.py',\n",
    "                     predictor_cls=ImagePredictor,\n",
    "                     framework_version='1.0.0')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = model.deploy(initial_instance_count=1, instance_type='ml.m5.xlarge')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Call SageMaker endpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are ready to call the SageMaker endpoint to see if it is making correct inferences against some test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use on existing endpoint\n",
    "#endpoint_name = 'fastai-shirts-2018-11-27-20-21-19-215'\n",
    "#predictor = ImagePredictor(endpoint_name=endpoint_name, sagemaker_session=sagemaker_session)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Motorhead T-Shirt\n",
    "#url = 'https://images.backstreetmerch.com/images/products/bands/clothing/mthd/bsi_mthd281.jpg'\n",
    "# Judas Priest T-Shirt\n",
    "#url = 'https://thumbs2.ebaystatic.com/d/l225/m/m7Lc1qRuFN3oFIlQla5V0IA.jpg'\n",
    "# Iron Maiden T-Shirt\n",
    "#url = 'https://www.ironmaidencollector.com/assets/pages/ab9ed-20180815_121042.jpg'\n",
    "# All Blacks rugby jersey\n",
    "#url = 'https://images.sportsdirect.com/images/products/38153703_l.jpg'\n",
    "# Australia Rugby T-Shirt\n",
    "#url = 'https://www.lovell-rugby.co.uk/products/products_580x387/40378.jpg'\n",
    "# Chicago Bulls top\n",
    "#url = 'https://i.ebayimg.com/images/g/qc0AAOSwBahVN~qm/s-l300.jpg'\n",
    "# Masters Golf Shirt\n",
    "#url = 'https://s-media-cache-ak0.pinimg.com/originals/29/6a/15/296a15200e7dd3ed08e12d9052ea4f97.jpg'\n",
    "img_bytes = requests.get(url).content\n",
    "img = Image.open(BytesIO(img_bytes))\n",
    "img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = predictor.predict(img_bytes)\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cleanup endpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you're done with the endpoint, you should clean it up.\n",
    "\n",
    "All of the training jobs, models and endpoints we created can be viewed through the SageMaker console of your AWS account."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_pytorch_p36",
   "language": "python",
   "name": "conda_pytorch_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}