example_classification_tensorflow_sagemaker_latest.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Image Classification Model Training with TensorFlow in Sagemaker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a tutorial guide for training an image classification model that is optimized for the FireFly-DL camera. SageMaker is an easy-to-use machine learning platform that allows us to train computer vision models using AWS cloud platform resources. In this tutorial we use the Flowers dataset as an example dataset to train a classification model that can classify five different types of flowers. Alternatively, you can upload your own dataset to an S3 bucket and train your own classification model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Libraries\n",
    "\n",
    "First, we import Sagemaker and several other python libraries needed in this tuturial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "# AWS SageMaker python SDK\n",
    "import sagemaker\n",
    "\n",
    "# Additionl libraries\n",
    "import numpy as np\n",
    "import random\n",
    "import os\n",
    "from PIL import Image\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import tarfile\n",
    "import urllib\n",
    "import boto3\n",
    "\n",
    "\n",
    "print('Libraries imported')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "First, we must setup a few things before starting the workflow.\n",
    "1.\tCreate a cloud storage bucket on S3. By default, we create a new s3 bucket named `flowerdataset` under the same region as your current notebook. You can change the name and region by assigning new values to the variables `bucket_name` and `region` in the code block below.\n",
    "2.\tInitialize SageMaker session and get the execution IAM role for accessing your AWS resources.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bucket_name = 'flowerdataset'\n",
    "region = None\n",
    "\n",
    "if len(bucket_name) < 3 :\n",
    "    raise ValueError('You must provide a valid bucket_name')\n",
    "if region == None:\n",
    "    region = sagemaker.Session().boto_region_name\n",
    "\n",
    "# list all bucket in your s3 repository\n",
    "s3 = boto3.client('s3')\n",
    "response = s3.list_buckets()\n",
    "buckets = [bucket['Name'] for bucket in response['Buckets']]\n",
    "\n",
    "# check if bucket exists and create new bucket\n",
    "if bucket_name not in buckets:\n",
    "    s3.create_bucket(Bucket= bucket_name, CreateBucketConfiguration={'LocationConstraint': region})\n",
    "    print(f' New S3 bucket name created. Bucket name: \"{bucket_name}\".')\n",
    "else:\n",
    "    print(f' Existing S3 bucket found. Bucket name: \"{bucket_name}\" ')\n",
    "    \n",
    "sess = sagemaker.Session() # Initialize sagemaker session\n",
    "role = sagemaker.get_execution_role() # Get notebook instance IAM role"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Option 1: Flowers Dataset \n",
    "\n",
    "Here we provide an example image dataset of five different types of flowers. This section is optional, and if you have your own dataset you can skip ahead to Option 2."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Download and Extract Flower Dataset\n",
    "As an example; we will use the Oxford Flowers dataset to train our model. This dataset can be downloaded from the following link http://download.tensorflow.org/example_images/flower_photos.tgz.\n",
    "The flower images are annotated using the parent directory name, and are split between five classes/folders according to the flower type:\n",
    "1. Daisy\n",
    "2. Sunflowers\n",
    "3. Roses\n",
    "4. Tulips\n",
    "5. Dandelion\n",
    "\n",
    "The following code downloads the flower photos and extracts the content to the *'/flower_photos'* directory in your current Jupyter notebook instance.\n",
    "current Jupyter notebook instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def download_and_extract(url, data_dir, download_dir):\n",
    "    target_file = url.split('/')[-1]\n",
    "    if target_file not in os.listdir(download_dir):\n",
    "        print('Downloading', url)\n",
    "        urllib.request.urlretrieve(url, os.path.join(download_dir, target_file))\n",
    "        tf = tarfile.open(url.split('/')[-1])\n",
    "        tf.extractall(data_dir)\n",
    "    else:\n",
    "        print('Already downloaded', url)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "image_dir = 'flower_photos' # Path to image directory folder. This must point to parent directery of the class folders.\n",
    "\n",
    "flowers_url = 'http://download.tensorflow.org/example_images/flower_photos.tgz'\n",
    "download_and_extract(flowers_url, './', '.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Visulization Flower Dataset\n",
    "\n",
    "The code below loops over the downloaded image dataset and randomly display’s some the images from the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import glob\n",
    "dirs = [f for f in os.listdir(image_dir) if '.txt' not in f]\n",
    "print('list of class labels', dirs)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create list of all images\n",
    "file_list = list()\n",
    "for root, dirs, files in os.walk(image_dir):\n",
    "    for file in files:\n",
    "        if file.endswith('.jpg'):\n",
    "            #print(root, file)\n",
    "            file_list.append((root,file))\n",
    "\n",
    "\n",
    "fig=plt.figure(figsize=(8, 8))\n",
    "columns = 2\n",
    "rows = 2\n",
    "\n",
    "for i in range(1, columns*rows +1):\n",
    "    img_path =random.choice([os.path.join(root,file) for root, file in file_list])\n",
    "    img = Image.open(img_path, 'r').convert('RGB')\n",
    "    ax = fig.add_subplot(rows, columns, i)\n",
    "    ax.title.set_text(img_path.split('/')[-2])\n",
    "    plt.imshow(img)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Upload Training Images to Your S3 bucket\n",
    "\n",
    "Next, we upload the training images to your S3 cloud storage bucket.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get a list of all bucket names from the response\n",
    "s3 = boto3.client('s3')\n",
    "response = s3.list_buckets()\n",
    "buckets = [bucket['Name'] for bucket in response['Buckets']]\n",
    "\n",
    "if bucket_name in buckets:\n",
    "    print('Uploading data to S3')\n",
    "    response = s3.list_objects_v2(\n",
    "                Bucket=bucket_name,\n",
    "                Prefix =image_dir,\n",
    "                MaxKeys=10)\n",
    "    # print(response)\n",
    "    if 'Contents' not in list(response.keys()):\n",
    "        s3_data_path = sess.upload_data(path=image_dir, bucket=bucket_name, key_prefix=image_dir)\n",
    "    else:\n",
    "        s3_data_path = f's3://{bucket_name}/{image_dir}' \n",
    "    print('Uploaded to', s3_data_path)\n",
    "else:\n",
    "    print(f' S3 bucket name \"{bucket_name}\" does not exists.')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can skip Option 2 if you wish and go directly to *Train with TensorFlow Estimator* section if you want to train the model on the flowers dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Option 2: Prepare and Upload Your Own Images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Collect Your Own Data\n",
    "\n",
    "First, you must collect and label some images that you would like to use to train the classification model on. \n",
    "1. Collect training images. \n",
    "    * The train.py script only supports the following image formats *'jpg', 'jpeg', 'png', and 'bmp'*.\n",
    "\n",
    "\n",
    "2. Label the images into classes using the parent directory name.\n",
    "    * Each image most be save into only one folder (representing the class)\n",
    "    * The ground-truth label for each image is taken from the parent directory name.\n"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "S3 bucket Directory Structure:\n",
    "\n",
    "bucket_name\n",
    "|\n",
    "|-- image_dir\n",
    "    |\n",
    "    |-- class_1\n",
    "    |   |\n",
    "    |   |--image_1.jpg\n",
    "    |   |--image_2.jpg\n",
    "    |           :\n",
    "    |           :\n",
    "    |-- class_2\n",
    "    |   |\n",
    "    |   |--image_1.jpg\n",
    "    |   |--image_2.jpg\n",
    "    |           :\n",
    "    |           :\n",
    "    |-- class_3\n",
    "    |   |\n",
    "    |   |--image_1.jpg\n",
    "    |   |--image_2.jpg\n",
    "    |           :\n",
    "                :"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Upload Training Images to Your S3 Bucket\n",
    " \n",
    "Next, upload your training images directly to the S3 bucket.\n",
    "1.\tCreate a folder (*image_dir*) inside your S3 bucket (*bucket_name*).\n",
    "2.\tAll the class folder (e.g. class_1, class_2 ...), which contain the images must be uploaded under the *image_dir* folder.\n",
    "\n",
    "**Important Note:**\n",
    "Verify that the bucket (*bucket_name*) and image folder (*image_dir*) variable names match the S3 bucket and image folder names, where your images were uploaded to. The above diagram shows the expected S3 image folder and file structure.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "image_dir = 'flower_photos' # MUST PROVIDE CORRECT IMAGE FOLDER NAME\n",
    "\n",
    "s3_data_path = f's3://{bucket_name}/{image_dir}' \n",
    "print('s3 image path', s3_data_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3.\tRun the next code block to check your S3 bucket folder structure is correct. If the folder structure is correct, the code output's a list of classes and the number of images per class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# Get a list of all bucket names from the response\n",
    "def check_s3_response(response, dic):\n",
    "    if 'correct_image_format' not in dic.keys() and 'wrong_image_format' not in dic.keys():\n",
    "        dic = {'correct_image_format':{}, 'wrong_image_format':list()}\n",
    "        \n",
    "    for key in response['Contents']:\n",
    "#         print(key['Key'].split('/'))\n",
    "        # Create file path list\n",
    "        file_path_list = key['Key'].split('/')\n",
    "        # check images\n",
    "        if len(file_path_list) > 2:\n",
    "            if file_path_list[-1].split('.')[1] in ['jpg', 'jpeg', 'png','bmp']:\n",
    "                # check class exists and append image to list\n",
    "                if file_path_list[-2] not in dic['correct_image_format'].keys():\n",
    "                    dic['correct_image_format'][file_path_list[-2]] = list()\n",
    "                dic['correct_image_format'][file_path_list[-2]].append(file_path_list[-1])\n",
    "            else:\n",
    "                dic['correct_image_format'].append('/'.join(file_path_list))\n",
    "    return dic\n",
    "\n",
    "print(f\"Scanning S3 bucket '{s3_data_path}' for images \\n\")\n",
    "\n",
    "# Get a list of all bucket names from the response\n",
    "s3 = boto3.client('s3')\n",
    "response = s3.list_buckets()\n",
    "buckets = [bucket['Name'] for bucket in response['Buckets']]\n",
    "\n",
    "if bucket_name in buckets:\n",
    "    response = s3.list_objects_v2(\n",
    "                Bucket=bucket_name,\n",
    "                Prefix =image_dir,\n",
    "                MaxKeys=1000)\n",
    "    # print(response)\n",
    "    if 'Contents' in list(response.keys()):\n",
    "        dic = {}\n",
    "        dic = check_s3_response(response, dic)    \n",
    "        while(response['IsTruncated']):\n",
    "            response = s3.list_objects_v2(\n",
    "                    Bucket=bucket_name,\n",
    "                    Prefix=image_dir,\n",
    "                    ContinuationToken=response['NextContinuationToken'],\n",
    "                    MaxKeys=1000)\n",
    "        #         print(response)         \n",
    "            dic = check_s3_response(response, dic)\n",
    "        print(f\"Class folders found in {image_dir} {list(dic['correct_image_format'].keys())}\")\n",
    "        print('Number of images found in each class')\n",
    "        for class_dir in dic['correct_image_format'].keys():\n",
    "            num_images = len(dic['correct_image_format'][class_dir])\n",
    "            print(f'{class_dir}: {num_images}')\n",
    "    else:\n",
    "        s3_data_path = ''\n",
    "        print(f\"'{image_dir}' does not exists in '{bucket_name}' s3 bucket\")\n",
    "        \n",
    "else:\n",
    "    print(f' S3 bucket name \"{bucket_name}\" does not exists.')\n",
    "    s3_data_path = ''\n",
    "\n",
    "print('\\n')\n",
    "print(f'S3 image path set to {s3_data_path}')\n",
    "\n",
    "## TODO:  Visulize random samples of the images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building and Registering A Docker container\n",
    "\n",
    "With Amazon SageMaker, you can package an training algorithm that can than be used in the SageMaker environment. This section will guide you through the process of building a Docker container for SageMaker.\n",
    "The shell code below first creates a privet ECR repository if the repository doesn't exist under your default region. Then, it uses the `dockerfile` to build the training docker images and push's that image to your ECR repository."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%sh\n",
    "\n",
    "# Specify an ecr image name\n",
    "ecr_image_name=tensorflow_image_classifier\n",
    "\n",
    "account=$(aws sts get-caller-identity --query Account --output text)\n",
    "\n",
    "# Get the region defined in the current configuration (default to us-west-2 if none defined)\n",
    "# region=$(aws configure get region)\n",
    "region=${region:-us-west-2}\n",
    "\n",
    "ecr_image_fullname=\"${account}.dkr.ecr.${region}.amazonaws.com/${ecr_image_name}:latest\"\n",
    "\n",
    "# If the repository doesn't exist in ECR, create it.\n",
    "aws ecr describe-repositories --repository-names \"${ecr_image_name}\" > /dev/null 2>&1\n",
    "if [ $? -ne 0 ]\n",
    "then\n",
    "aws ecr create-repository --repository-name \"${ecr_image_name}\" > /dev/null\n",
    "fi\n",
    "\n",
    "# Get the login command from ECR and execute it directly\n",
    "$(aws ecr get-login --region ${region} --no-include-email)\n",
    "\n",
    "# Build the docker image locally with the image name and then push it to ECR\n",
    "# with the full name.\n",
    "sudo chown -R $USER .\n",
    "docker build -f Dockerfile -t ${ecr_image_name} .\n",
    "docker tag ${ecr_image_name} ${ecr_image_fullname}\n",
    "\n",
    "docker push ${ecr_image_fullname}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Algorithm Resource\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can create an algorithm resource using the docker image created in the previous step. This algorithm resource defines the reference docker image, parameters, and settings for your training job. The algorithm can then be used to lunch multiple training jobs. You will only need to create this algorithm once and it will automatically be saved under your Amazon SageMaker account."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Define algorithm "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from src.training_specification import TrainingSpecification\n",
    "from src.training_channels import TrainingChannels\n",
    "from src.metric_definitions import MetricDefinitions\n",
    "from src.tuning_objectives import TuningObjectives\n",
    "import json\n",
    "\n",
    "account = sess.boto_session.client(\"sts\").get_caller_identity()[\"Account\"]\n",
    "region = sess.boto_session.region_name\n",
    "ecr_image = \"{}.dkr.ecr.{}.amazonaws.com/tensorflow_image_classifier:latest\".format(account, region)\n",
    "\n",
    "training_specification = TrainingSpecification().get_training_specification_dict(\n",
    "    ecr_image=ecr_image,\n",
    "    supports_gpu=True,\n",
    "    supported_channels=[\n",
    "        TrainingChannels(\n",
    "            \"train\",\n",
    "            description=\"Input channel that provides training data\",\n",
    "            supported_content_types=[\"png\"],\n",
    "        )\n",
    "    ],\n",
    "    supported_metrics=[MetricDefinitions(\"Loss\", \"loss:(.*?);\")],\n",
    "    supported_tuning_job_objective_metrics=[TuningObjectives(\"Minimize\", \"Loss\")],\n",
    ")\n",
    "\n",
    "# print(json.dumps(training_specification, indent=2, sort_keys=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Create algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import time\n",
    "import boto3\n",
    "\n",
    "smmp = boto3.client(\"sagemaker\")\n",
    "\n",
    "algorithm_name = \"tensorflow-image-classifier-algorithm\"\n",
    "\n",
    "create_algorithm_input_dict = {\n",
    "    \"AlgorithmName\": algorithm_name,\n",
    "    \"AlgorithmDescription\": \"Image classification training algorihtm for FireFly-DL\",\n",
    "    \"CertifyForMarketplace\": False,\n",
    "}\n",
    "create_algorithm_input_dict.update(training_specification)\n",
    "\n",
    "print(\"Now creating an algorithm in SageMaker\")\n",
    "\n",
    "smmp.create_algorithm(**create_algorithm_input_dict)\n",
    "\n",
    "while True:\n",
    "    response = smmp.describe_algorithm(AlgorithmName=algorithm_name)\n",
    "    status = response[\"AlgorithmStatus\"]\n",
    "    print(status)\n",
    "    if status == \"Completed\" or status == \"Failed\":\n",
    "        break\n",
    "    time.sleep(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start a training job"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the training algorithm has succefully been created. We can start training a classification model using either \n",
    "1. Option 1: AWS SageMaker console (Recommended).\n",
    "2. Option 2: SageMaker Python API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Option 1: Using AWS Console\n",
    "\n",
    "1. On your SageMaker console, navigate to the `Trining > algorihtms` tab on the left side of the console. Under the `My algorithms` tab you should see the recently created algorithm `tensorflow-image-classifier-algorithm` as shown in the image below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"images/train_job_select_algo.PNG\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. After selecting the algorithm `tensorflow-image-classifier-algorithm`, and under the `Action` tab menu select `Create training job`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"images/start_train_job.PNG\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Under job setting choose a name for the training job."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"images/train_job_name.PNG\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. Under resource configureation select the `ml-p2-xlarge` instance type and increase the `additional storage volume per instance` to 30GB (You can add more if your training dataset size is lareger)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"images/train_job_resources.PNG\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. Configure the hyperparameter. The defaults hyperparameter settings should provide good results. Alternativly, you can experiment with the different settings to improve the model performance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"images/train_job_hparameters.PNG\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6. Input data configuration. Provide the S3 location path for your training image. Leave the reset the setting as the default."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"images/train_job_input.PNG\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "7. Output data configuration. Provide the S3 output path where you want to save training output files."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"images/train_job_output.PNG\"/></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "8. Finaly, click Create training job to start the trianing job."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- ![Image of Yaktocat](https://github.com/FLIR/amazon-sagemaker-firefly-dl/blob/main/images/start_train_job.PNG?raw=true) -->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Option 2: Using SageMaker API"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Hyperparameter setting \n",
    "\n",
    "Specify the Hyperparameters for the training job. Note that the default parameters generally work very well and give good results. Below is a list of available hyperparameters.\n",
    "\n",
    "- batch_size: The number of samples in each batch.\n",
    "- num_of_trainable_layers: Number of trainable layers, options 1, 2, 3 or 4 layers. By default, only one layer is trained.\n",
    "- max_number_of_steps: The maximum number of training steps.\n",
    "- apply_image_augmentation: Enable random image augmentation during preprocessing for training. Below augmentation flags are only enabled if this flag is enabled.\n",
    "    - random_image_flip: Enable random image flip (horizontally).\n",
    "    - random_image_crop: Enable random cropping of images. \n",
    "    - min_object_cover: Fraction of whole image remaining after cropping.\n",
    "    - random_image_rotation: Enable random image rotation counter-clockwise by 90, 180, 270, or 360 degrees.\n",
    "    - random saturation, contrast, and brightness augmentation is applied by default.\n",
    "- learning_rate: Initial learning rate.\n",
    "- model_name: The name of the architecture to train, one of, mobilenet_v1_075, mobilenet_v1_050, mobilenet_v1_025, mobilenet_v1, inception_v1.\n",
    "- optimizer: The name of the optimizer, one of adadelta, adagrad, adam, ftrl, momentum, sgd or rmsprop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hyperparameters = {\"model_name\":'mobilenet_v1', \n",
    "                    \"max_number_of_steps\":100, \n",
    "                    \"learning_rate\":0.01, \n",
    "                    \"batch_size\":16, \n",
    "                    \"num_of_trainable_layers\":2}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We create an ` AlgorithmEstimator` object and define the train job configuration:\n",
    "\n",
    "* Algorithm_arn. Amazon Resource Name (ARN) or the name of the algorithm.\n",
    "* Role. As defined above.\n",
    "* Instance count which is the number of machines to use for training.\n",
    "* Instance type which is the type of machine to use for training.\n",
    "* Output path determines where the model artifact will be written.\n",
    "* Hyperparameters: List of available hyperparameters are give above.\n",
    "\n",
    "Then we use fit() on the estimator to train against the data that we uploaded above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker import AlgorithmEstimator\n",
    "\n",
    "algo = AlgorithmEstimator(\n",
    "        algorithm_arn=algorithm_name,\n",
    "        role=role,\n",
    "        instance_count=1,\n",
    "        hyperparameters=hyperparameters,\n",
    "        output_path=f's3://{bucket_name}',\n",
    "        instance_type='ml.p2.xlarge')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "algo.fit({'train':s3_data_path})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "## Training Job Cost Estimate\n",
    "\n",
    "For a typical training job the total cost is split mainly between:\n",
    "1.\tData storage and access costs.\n",
    "2.\tModel training cost. For example an ml.p2.xlarge (GPU) instance type costs 1.26 USD per hour (majority of the cost).\n",
    "3.\tRunning the current Notebook instance.\n",
    "You can use the code block below to estimate the total cost. Copy the *Billable seconds* amount that is printed out at the end of the training process to the variable *Billable_time_in_seconds*.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Billable_time_in_seconds = 428 # Enter the bilable time in seconds\n",
    "print(f'Training cost ${Billable_time_in_seconds * 1.26 / 3600}') # $1.26 per hour"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy The TensorFlow Model\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the trained model artifact from your S3 bucket.\n",
    "\n",
    "After training is complete Sagemaker automatically compresses and copies the trained model artifact (*model.tar.gz*) to your S3 bucket. Please note the following:\n",
    "* The trained model artifact is saved under the following directory path *s3://bucket_name/tensorflow-training-... /output/* (e.g. s3://firefly-flowers/tensorflow-training-2020-07-03-20-50-48-055/output).\n",
    "* Select the compressed file (*model.tar.gz*) in your S3 console and click the download button to download the file.\n",
    "* Decompress the file using your preferred file decompression tool. Inside the model folder you should find the trained model grpah `firefly.graph`\n",
    "\n",
    "### Upload model to camera\n",
    "\n",
    "Make sure the following camera setting are updated after uploading the `firefly.graph` model to the camera using SpinView.\n",
    "* Network input image width and height to 224.\n",
    "* Channel pixel format is set to BGR8 or Mono depending on your training image format.\n",
    "* Channel mean and scaler values are set to 127.5 for all the channels.\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_tensorflow_p36",
   "language": "python",
   "name": "conda_tensorflow_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}