chalk_talk_demo-neo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# re:Invent Chalk Talk - Building, Training, and Deploying Fast.ai Models Using Amazon SageMaker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*This notebook was created and tested on an ml.p3.2xlarge notebook instance.*\n",
    "\n",
    "Let's start by creating a SageMaker session and specifying:\n",
    "\n",
    "* The **S3 bucket** and prefix that you want to use for training and model data. This should be within the same region as the Notebook Instance, training, and hosting.\n",
    "* The **IAM role** arn used to give training and hosting access to your data. See the documentation for how to create these. Note, if more than one role is required for notebook instances, training, and/or hosting, please replace the sagemaker.get_execution_role() with appropriate full IAM role arn string(s). \n",
    "\n",
    "**IMPORTANT** please make sure the IAM role associated to your SageMaker notebook instance has the following managed IAM policies attached:\n",
    "\n",
    "* **arn:aws:iam::aws:policy/AmazonSageMakerFullAccess**\n",
    "* **arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess**\n",
    "\n",
    "We also need to ensure there are no AWS credentials setup on the instance. We will set these up later in order to be able to train and deploy locally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import os\n",
    "import random\n",
    "from io import BytesIO\n",
    "import subprocess\n",
    "from glob import glob\n",
    "\n",
    "from IPython.display import display, HTML, Image\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from PIL import Image\n",
    "import requests\n",
    "import boto3\n",
    "\n",
    "import sagemaker\n",
    "from sagemaker.analytics import TrainingJobAnalytics\n",
    "from sagemaker.local import local_session\n",
    "from sagemaker.pytorch import PyTorch\n",
    "from sagemaker.predictor import RealTimePredictor, json_deserializer\n",
    "from sagemaker.pytorch import PyTorchModel\n",
    "from sagemaker.utils import name_from_image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def display_images(images, header=None, width=\"100%\"):\n",
    "    if type(width)==type(1): width = \"{}px\".format(width)\n",
    "    html = [\"<table style='width:{}'><tr>\".format(width)]\n",
    "    if header is not None:\n",
    "        html += [\"<th>{}</th>\".format(h) for h in header] + [\"</tr><tr>\"]\n",
    "\n",
    "    for image in images:\n",
    "        html.append(\"<td><img src='{}' /></td>\".format(image))\n",
    "    html.append(\"</tr></table>\")\n",
    "    display(HTML(''.join(html)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! if [ -e ~/.aws/credentials ]; then rm ~/.aws/credentials; fi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sagemaker_session = sagemaker.Session()\n",
    "\n",
    "bucket_name = sagemaker_session.default_bucket()\n",
    "prefix = 'sagemaker/DEMO-shirts-classification'\n",
    "\n",
    "role = sagemaker.get_execution_role()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also need to pull the needed Docker images from DockerHub that are specific for running fastai based models on SageMaker. \n",
    "\n",
    "There is a project with the source code and Dockerfile that can be found at the GitHub project: https://github.com/aws-samples/amazon-sagemaker-container-with-fastai.\n",
    "\n",
    "Once we download the Docker images we then need to upload them to ECR so that they can be used by SageMaker."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "# get the Dockerfile from GitHub\n",
    "wget https://raw.githubusercontent.com/aws-samples/amazon-sagemaker-container-with-fastai/master/Dockerfile\n",
    "\n",
    "IMAGE=\"sagemaker-fastai\"\n",
    "\n",
    "# parameters\n",
    "FASTAI_VERSION=\"1.0\"\n",
    "PY_VERSION=\"py36\"\n",
    "\n",
    "# Get the account number associated with the current IAM credentials\n",
    "account=$(aws sts get-caller-identity --query Account --output text)\n",
    "\n",
    "if [ $? -ne 0 ]\n",
    "then\n",
    "    exit 255\n",
    "fi\n",
    "\n",
    "# Get the region defined in the current configuration (default to us-west-2 if none defined)\n",
    "region=$(aws configure get region)\n",
    "region=${region:-us-west-2}\n",
    "\n",
    "# If the repository doesn't exist in ECR, create it.\n",
    "\n",
    "aws ecr describe-repositories --repository-names \"${IMAGE}\" > /dev/null 2>&1\n",
    "\n",
    "if [ $? -ne 0 ]\n",
    "then\n",
    "    aws ecr create-repository --repository-name \"${IMAGE}\" > /dev/null\n",
    "fi\n",
    "\n",
    "# Get the login command from ECR and execute it directly\n",
    "$(aws ecr get-login --region ${region} --no-include-email)\n",
    "\n",
    "# Get the login command from ECR in order to pull down the SageMaker PyTorch image\n",
    "$(aws ecr get-login --registry-ids 520713654638 --region ${region} --no-include-email)\n",
    "\n",
    "# loop for each architecture (cpu & gpu)\n",
    "for arch in gpu cpu\n",
    "do  \n",
    "    echo \"Building image with arch=${arch}, region=${region}\"\n",
    "    TAG=\"${FASTAI_VERSION}-${arch}-${PY_VERSION}\"\n",
    "    FULLNAME=\"${account}.dkr.ecr.${region}.amazonaws.com/${IMAGE}:${TAG}\"\n",
    "    docker build -t ${IMAGE}:${TAG} --build-arg ARCH=\"$arch\"  --build-arg REGION=\"${region}\"  .\n",
    "    docker tag ${IMAGE}:${TAG} ${FULLNAME}\n",
    "    docker push ${FULLNAME}\n",
    "done"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download training data to notebook instance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be utilizing a custom data set with a mixture of pictures of heavy metal t-shirts and sport shirts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "if [ ! -d data/shirts ]; then\n",
    "    mkdir -p data/shirts\n",
    "    wget -q https://s3-eu-west-1.amazonaws.com/sagemaker-934676248949-eu-west-1/data/shirts_imgs.tar.gz\n",
    "    tar zxf shirts_imgs.tar.gz -C data/shirts\n",
    "    rm shirts_imgs.tar.gz\n",
    "fi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DATA_PATH=f'{os.getcwd()}/data/shirts'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%ls {DATA_PATH}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View sample images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at some of the images in the folders."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metal_img = random.choice(glob('data/shirts/metal/*.jpg'))\n",
    "sport_img = random.choice(glob('data/shirts/sport/*.jpg'))\n",
    "\n",
    "display_images([metal_img, sport_img],\n",
    "       header=['Metal', 'Sport'], width=\"60%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Upload training data to S3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are going to use the sagemaker.Session.upload_data function to upload our datasets to an S3 location. The return value inputs identifies the location -- we will use later when we start the training job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s3 = boto3.client('s3')\n",
    "key = f'{prefix}/metal/'\n",
    "response = s3.list_objects_v2(\n",
    "        Bucket=bucket_name,\n",
    "        Prefix=key,\n",
    ")\n",
    "\n",
    "if response['KeyCount'] > 0:\n",
    "    print(\"Images exist in S3!\")\n",
    "    data_location=f's3://{bucket_name}/{prefix}'\n",
    "else:\n",
    "    print(\"Training images not uploaded to S3. Uploading now\")\n",
    "    data_location = sagemaker_session.upload_data(path=DATA_PATH, bucket=bucket_name, key_prefix=prefix)\n",
    "\n",
    "print(f'training images location: {data_location}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train on SageMaker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have tested the training and hosting locally, we are ready to train our model using the Amazon SageMaker training service.\n",
    "\n",
    "Training a model on SageMaker with the Python SDK is done in a way that is similar to the way we trained it locally. This is done by changing our train_instance_type from `local` to one of our [supported EC2 instance types](https://aws.amazon.com/sagemaker/pricing/instance-types/).\n",
    "\n",
    "In addition, we must now specify the ECR image URL, which we just pushed above.\n",
    "\n",
    "Finally, our local training dataset has to be in Amazon S3 and the S3 URL to our dataset is passed into the `fit()` call.\n",
    "\n",
    "Let's first fetch our ECR image url that corresponds to the image we just built and pushed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region = boto3.session.Session().region_name\n",
    "\n",
    "client = boto3.client('sts')\n",
    "account = client.get_caller_identity()['Account']\n",
    "\n",
    "base_image_name = 'sagemaker-fastai'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "image_name = f'{account}.dkr.ecr.{region}.amazonaws.com/{base_image_name}:1.0-gpu-py36'\n",
    "print(f'Using ECR image for SageMaker training: {image_name}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will create a `PyTorch` estimator object using the SageMaker SDK. The input parameters are almost exactly the same as when we trained locally except we will provide an instance type that will be a specfic instance type used to train the model using the SageMaker training service. In this specific example we will use the `ml.p3.2xlarge` instance type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "estimator = PyTorch(entry_point='src/shirts-neo/train.py',\n",
    "                    role=role,\n",
    "                    train_instance_count=1,\n",
    "                    train_instance_type='ml.p3.2xlarge',\n",
    "                    image_name=image_name,\n",
    "                    framework_version='1',\n",
    "                    hyperparameters={\n",
    "                        'epochs': 6, \n",
    "                        'batch-size': 64\n",
    "                    },\n",
    "                    metric_definitions=[\n",
    "                        {'Name': 'valid:loss',     'Regex': '#quality_metric: host=\\S+, epoch=\\S+, valid_loss=(\\S+)'},\n",
    "                        {'Name': 'train:loss',     'Regex': '#quality_metric: host=\\S+, epoch=\\S+, train_loss=(\\S+)'},\n",
    "                        {'Name': 'valid:accuracy', 'Regex': '#quality_metric: host=\\S+, epoch=\\S+, accuracy=(\\S+)'}\n",
    "                    ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_location=f's3://{bucket_name}/{prefix}'\n",
    "print(f'Training data location: {data_location}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_job_name=name_from_image('fastai-shirts')\n",
    "estimator.fit(data_location, job_name=training_job_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can plot the accuracy metric on a graph pulling the data from CloudWatch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Graph training metrics from SageMaker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the dataframe of training metrics\n",
    "df = TrainingJobAnalytics(training_job_name=training_job_name,metric_names=['train:loss', 'valid:loss', 'valid:accuracy']).dataframe()\n",
    "\n",
    "# plot the dataframe with matplotlib\n",
    "fig, (ax1, ax2) = plt.subplots(1,2, sharey=False)\n",
    "ax1.set_title('Loss')\n",
    "ax1.set_ylabel('loss')\n",
    "for key, grp in df.loc[df['metric_name'] != 'valid:accuracy'].groupby(['metric_name']):\n",
    "    ax = grp.plot(ax=ax1, kind='line', x='timestamp', y='value', label=key)\n",
    "\n",
    "ax2.set_title('Accuracy')\n",
    "ax2.set_ylabel('accuracy')\n",
    "df.loc[df['metric_name'] == 'valid:accuracy'].plot(ax=ax2, kind='line', x='timestamp', y='value', label='accuracy')\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compile Model with SageMaker Neo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will compile the model with SageMaker Neo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_path = '/'.join(estimator.output_path.split('/')[:-1])\n",
    "optimized_ic = estimator.compile_model(target_instance_family='ml_c5', \n",
    "                                input_shape={'actual_input_1':[1, 3, 224, 224]},  # Batch size 1, 3 channels, 224x224 Images.\n",
    "                                output_path=output_path,\n",
    "                                framework='onnx', \n",
    "                                framework_version='1.3')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimized_ic.name = 'deployed-shirts-classification'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Host model with SageMaker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimized_ic_classifier = optimized_ic.deploy(initial_instance_count = 1,\n",
    "                                              instance_type = 'ml.c5.4xlarge')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Call SageMaker endpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are ready to call the SageMaker endpoint to see if it is making correct inferences against some test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use on existing endpoint\n",
    "#endpoint_name = 'fastai-shirts-2018-11-27-20-21-19-215'\n",
    "#predictor = ImagePredictor(endpoint_name=endpoint_name, sagemaker_session=sagemaker_session)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Motorhead T-Shirt\n",
    "#url = 'https://images.backstreetmerch.com/images/products/bands/clothing/mthd/bsi_mthd281.jpg'\n",
    "# Judas Priest T-Shirt\n",
    "#url = 'https://thumbs2.ebaystatic.com/d/l225/m/m7Lc1qRuFN3oFIlQla5V0IA.jpg'\n",
    "# Iron Maiden T-Shirt\n",
    "#url = 'https://www.ironmaidencollector.com/assets/pages/ab9ed-20180815_121042.jpg'\n",
    "# All Blacks rugby jersey\n",
    "#url = 'https://images.sportsdirect.com/images/products/38153703_l.jpg'\n",
    "# Australia Rugby T-Shirt\n",
    "#url = 'https://www.lovell-rugby.co.uk/products/products_580x387/40378.jpg'\n",
    "# Chicago Bulls top\n",
    "url = 'https://i.ebayimg.com/images/g/qc0AAOSwBahVN~qm/s-l300.jpg'\n",
    "# Masters Golf Shirt\n",
    "#url = 'https://s-media-cache-ak0.pinimg.com/originals/29/6a/15/296a15200e7dd3ed08e12d9052ea4f97.jpg'\n",
    "img_bytes = requests.get(url).content\n",
    "img = Image.open(BytesIO(img_bytes))\n",
    "img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = optimized_ic_classifier.predict(img_bytes)\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cleanup endpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you're done with the endpoint, you should clean it up.\n",
    "\n",
    "All of the training jobs, models and endpoints we created can be viewed through the SageMaker console of your AWS account."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimized_ic_classifier.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "## Reference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "- [How Amazon SageMaker interacts with your Docker container for training](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo.html)\n",
    "- [How Amazon SageMaker interacts with your Docker container for inference](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-inference-code.html)\n",
    "- [SageMaker Python SDK](https://github.com/aws/sagemaker-python-sdk)\n",
    "- [Dockerfile](https://docs.docker.com/engine/reference/builder/)\n",
    "- [PyTorch extending container example](https://github.com/awslabs/amazon-sagemaker-examples/blob/master/advanced_functionality/pytorch_extending_our_containers/pytorch_extending_our_containers.ipynb)\n",
    "- [scikit-bring-your-own example](https://github.com/awslabs/amazon-sagemaker-examples/blob/master/advanced_functionality/scikit_bring_your_own/scikit_bring_your_own.ipynb)\n",
    "- [SageMaker fast.ai container](https://github.com/aws-samples/amazon-sagemaker-container-with-fastai)\n",
    "- [SageMaker fast.ai example](https://github.com/mattmcclean/sagemaker-fastai-example)\n",
    "- [SageMaker PyTorch container](https://github.com/aws/sagemaker-pytorch-container)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_pytorch_p36",
   "language": "python",
   "name": "conda_pytorch_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}