4-deploy-model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright (C) Microsoft Corporation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploy model as a Web Service"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Important**: Make sure the kernel is set to \"Your project name local\" which can be done from the *Kernel* menu under *Change kernel*. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The web services can be deployed locally or to a cluster. Best practice is to start with a local deployment, validate that your model and code work, and then deploy to a cluster for production scale use. In this notebook, you will first download your operationalization files, register  machine learning compute resource provider in your subscription and create a model management account.\n",
    "\n",
    "Next, **If you have Docker engine running locally**, you can continue with the instuctions for [local environment setup](#Local-environment-setup) section and follow the rest of the instructions to deploy your models locally. **If you are not running Docker locally**, then you will need to skip the local environment setup section and follow the steps for [cluster environment setup](#Cluster-environment-setup) and the rest of the instructions to deploy your web service to a cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import base64\n",
    "import requests\n",
    "import json\n",
    "import zipfile\n",
    "from azure.storage.blob import BlockBlobService"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download operationalization files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, you will download the zip file you uploaded to blob in the earlier notebook and extract it to get the model and scoring script. Locate your storage account name and key as described in the Getting Started document of this tutorial and provide your values in the following cell where indicated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "ACCOUNT_NAME = 'Your Storage Account Name Here!!'\n",
    "ACCOUNT_KEY = 'Your Storage Account Key Here!!'\n",
    "CONTAINER_NAME = \"deploy\"\n",
    "ZIP_FILE = 'o16n.zip'\n",
    "DESTINATION_FILE = ZIP_FILE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download and extract the files to the current working directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "block_blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)\n",
    "block_blob_service.get_blob_to_path(container_name=CONTAINER_NAME, blob_name=ZIP_FILE, file_path=DESTINATION_FILE)\n",
    "zip = zipfile.ZipFile(ZIP_FILE)\n",
    "zip.extractall()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Register resource provider"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, make sure the Azure resource provider Microsoft.MachineLearningCompute is registered in your subscription.This is a one time task per subscription."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Registering is still on-going. You can monitor using 'az provider show -n Microsoft.MachineLearningCompute'\n"
     ]
    }
   ],
   "source": [
    "!az provider register -n Microsoft.MachineLearningCompute"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make sure you see \"registrationState\": \"Registered\" in the next cell output before you proceed to the next steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"id\": \"/subscriptions/xxxxxxx-xxxx-xxxx-xxxxx-xxxxxxxx/providers/Microsoft.MachineLearningCompute\",\n",
      "  \"namespace\": \"Microsoft.MachineLearningCompute\",\n",
      "  \"registrationState\": \"Registered\",\n",
      "  \"resourceTypes\": [\n",
      "    {\n",
      "      \"aliases\": null,\n",
      "      \"apiVersions\": [\n",
      "        \"2017-08-01-preview\",\n",
      "        \"2017-06-01-preview\"\n",
      "      ],\n",
      "      \"locations\": [\n",
      "        \"East US 2\",\n",
      "        \"West Central US\",\n",
      "        \"Australia East\",\n",
      "        \"West Europe\",\n",
      "        \"Southeast Asia\"\n",
      "      ],\n",
      "      \"properties\": null,\n",
      "      \"resourceType\": \"operationalizationClusters\"\n",
      "    },\n",
      "    {\n",
      "      \"aliases\": null,\n",
      "      \"apiVersions\": [\n",
      "        \"2017-08-01-preview\",\n",
      "        \"2017-06-01-preview\"\n",
      "      ],\n",
      "      \"locations\": [\n",
      "        \"East US 2\"\n",
      "      ],\n",
      "      \"properties\": null,\n",
      "      \"resourceType\": \"operations\"\n",
      "    },\n",
      "    {\n",
      "      \"aliases\": null,\n",
      "      \"apiVersions\": [\n",
      "        \"2017-08-01-preview\",\n",
      "        \"2017-06-01-preview\"\n",
      "      ],\n",
      "      \"locations\": [\n",
      "        \"East US 2\"\n",
      "      ],\n",
      "      \"properties\": null,\n",
      "      \"resourceType\": \"locations\"\n",
      "    },\n",
      "    {\n",
      "      \"aliases\": null,\n",
      "      \"apiVersions\": [\n",
      "        \"2017-08-01-preview\",\n",
      "        \"2017-06-01-preview\"\n",
      "      ],\n",
      "      \"locations\": [\n",
      "        \"East US 2\",\n",
      "        \"West Central US\",\n",
      "        \"Australia East\",\n",
      "        \"West Europe\",\n",
      "        \"Southeast Asia\"\n",
      "      ],\n",
      "      \"properties\": null,\n",
      "      \"resourceType\": \"locations/operations\"\n",
      "    },\n",
      "    {\n",
      "      \"aliases\": null,\n",
      "      \"apiVersions\": [\n",
      "        \"2017-08-01-preview\",\n",
      "        \"2017-06-01-preview\"\n",
      "      ],\n",
      "      \"locations\": [\n",
      "        \"East US 2\",\n",
      "        \"West Central US\",\n",
      "        \"Australia East\",\n",
      "        \"West Europe\",\n",
      "        \"Southeast Asia\"\n",
      "      ],\n",
      "      \"properties\": null,\n",
      "      \"resourceType\": \"locations/operationsStatus\"\n",
      "    }\n",
      "  ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "!az provider show -n Microsoft.MachineLearningCompute"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create names for deployment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To deploy and test your web services, provide an Azure region and a deployment name in the following cell where indicated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "LOCATION = 'Pick an Azure Region Here!! e.g. eastus2'\n",
    "DEPLOY_NAME = 'Pick a Deployment Name Here!!'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a model management account"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You need to create a [Model Management Account](https://docs.microsoft.com/en-us/azure/machine-learning/preview/model-management-overview) to deploy your models. You need to do this once per subscription, and can reuse the same account in multiple deployments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "MM_ACCOUNT = DEPLOY_NAME + 'mmacc'     \n",
    "MM_ACCOUNT_RESOURCE_GROUP = MM_ACCOUNT + 'rg'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a resource group for model management account which will be used for both local and cluster deployments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"id\": \"/subscriptions/edf507a2-6235-46c5-b560-fd463ba2e771/resourceGroups/imagetutmmaccrg\",\n",
      "  \"location\": \"eastus2\",\n",
      "  \"managedBy\": null,\n",
      "  \"name\": \"imagetutmmaccrg\",\n",
      "  \"properties\": {\n",
      "    \"provisioningState\": \"Succeeded\"\n",
      "  },\n",
      "  \"tags\": null\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "!az group create --l $LOCATION --name $MM_ACCOUNT_RESOURCE_GROUP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, create the model management account in the resource group created above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"created_on\": \"2017-11-03T17:51:11.9226449999999999Z\",\n",
      "  \"description\": \"\",\n",
      "  \"id\": \"/subscriptions/edf507a2-6235-46c5-b560-fd463ba2e771/resourceGroups/imagetutmmaccrg/providers/Microsoft.MachineLearningModelManagement/accounts/imagetutmmacc\",\n",
      "  \"location\": \"eastus2\",\n",
      "  \"model_management_swagger_location\": \"https://eastus2.modelmanagement.azureml.net/api/subscriptions/edf507a2-6235-46c5-b560-fd463ba2e771/resourceGroups/imagetutmmaccrg/accounts/imagetutmmacc/swagger.json?api-version=2017-09-01-preview\",\n",
      "  \"modified_on\": \"2017-11-03T17:51:11.9226449999999999Z\",\n",
      "  \"name\": \"imagetutmmacc\",\n",
      "  \"resource_group\": \"imagetutmmaccrg\",\n",
      "  \"sku\": {\n",
      "    \"capacity\": 1,\n",
      "    \"name\": \"S1\"\n",
      "  },\n",
      "  \"subscription\": \"edf507a2-6235-46c5-b560-fd463ba2e771\",\n",
      "  \"tags\": {},\n",
      "  \"type\": \"Microsoft.MachineLearningModelManagement/accounts\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "!az ml account modelmanagement create -l $LOCATION -n $MM_ACCOUNT -g $MM_ACCOUNT_RESOURCE_GROUP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verify that model management account is set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"created_on\": \"2017-11-03T17:51:11.9226449999999999Z\",\n",
      "  \"description\": \"\",\n",
      "  \"id\": \"/subscriptions/edf507a2-6235-46c5-b560-fd463ba2e771/resourceGroups/imagetutmmaccrg/providers/Microsoft.MachineLearningModelManagement/accounts/imagetutmmacc\",\n",
      "  \"location\": \"eastus2\",\n",
      "  \"model_management_swagger_location\": \"https://eastus2.modelmanagement.azureml.net/api/subscriptions/edf507a2-6235-46c5-b560-fd463ba2e771/resourceGroups/imagetutmmaccrg/accounts/imagetutmmacc/swagger.json?api-version=2017-09-01-preview\",\n",
      "  \"modified_on\": \"2017-11-03T17:51:11.9226449999999999Z\",\n",
      "  \"name\": \"imagetutmmacc\",\n",
      "  \"resource_group\": \"imagetutmmaccrg\",\n",
      "  \"sku\": {\n",
      "    \"capacity\": 1,\n",
      "    \"name\": \"S1\"\n",
      "  },\n",
      "  \"subscription\": \"edf507a2-6235-46c5-b560-fd463ba2e771\",\n",
      "  \"tags\": {},\n",
      "  \"type\": \"Microsoft.MachineLearningModelManagement/accounts\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "!az ml account modelmanagement show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Local environment setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following steps, you will deploy and test your web service on the local machine running Docker. If you are not running Docker locally, please jump to [the cluster environment setup](#Cluster-environment-setup) section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "LOCAL_ENV_NAME = DEPLOY_NAME + 'local'+ 'env'  \n",
    "LOCAL_RESOURCE_GROUP = LOCAL_ENV_NAME + 'rg'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set up a local environment using the following command which creates the following resources in your subscription:\n",
    "- A resource group with \"rg\" added at the end of the environment name you picked\n",
    "- A storage account\n",
    "- An Azure Container Registry (ACR)\n",
    "- An Application insights account"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating resource group imagetutlocalenvrg\n",
      "Provisioning compute resources...\n",
      "Resource creation submitted successfully.\n",
      "To see more information for your environment, run:\n",
      "  az ml env show -g imagetutlocalenvrg -n imagetutlocalenv\n",
      "You can set the new environment as your target context using:\n",
      "  az ml env set -g imagetutlocalenvrg -n imagetutlocalenv\n"
     ]
    }
   ],
   "source": [
    "!az ml env setup -l $LOCATION -n $LOCAL_ENV_NAME  --yes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can check the information on the environment with the following. You will need to wait until \"Provisioning State\" is set to \"Succeeded\" to continue with the next steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"Cluster Name\": \"imagetutlocalenv\",\n",
      "  \"Cluster Size\": \"N/A\",\n",
      "  \"Created On\": \"2017-11-06T14:34:11.309Z\",\n",
      "  \"Location\": \"eastus2\",\n",
      "  \"Provisioning State\": \"Succeeded\",\n",
      "  \"Resource Group\": \"imagetutlocalenvrg\",\n",
      "  \"Subscription\": \"edf507a2-6235-46c5-b560-fd463ba2e771\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "!az ml env show -g $LOCAL_RESOURCE_GROUP -n $LOCAL_ENV_NAME"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, set the environment to be used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Compute set to imagetutlocalenv.\n"
     ]
    }
   ],
   "source": [
    "!az ml env set -g $LOCAL_RESOURCE_GROUP -n $LOCAL_ENV_NAME"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check if your environment is set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"Cluster Name\": \"imagetutlocalenv\",\n",
      "  \"Cluster Size\": \"N/A\",\n",
      "  \"Created On\": \"2017-11-06T14:34:11.309Z\",\n",
      "  \"Current Mode\": \"local\",\n",
      "  \"Location\": \"eastus2\",\n",
      "  \"Provisioning State\": \"Succeeded\",\n",
      "  \"Resource Group\": \"imagetutlocalenvrg\",\n",
      "  \"Subscription\": \"edf507a2-6235-46c5-b560-fd463ba2e771\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "!az ml env show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deploy model locally"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can deploy your model with the following command.\n",
    "```bash\n",
    "az ml service create realtime --model-file [model file/folder path] -f [scoring file e.g. score.py] -n [your service name] -s [schema file e.g. service_schema.json] -r [runtime for the Docker container e.g. spark-py or python] -c [conda dependencies file for additional python packages]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, set the service name."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "LOCAL_SERVICE_NAME = DEPLOY_NAME + 'local' + 'srvc'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create the real time service."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " lightgbm_classifier.model\n",
      "Successfully registered model\n",
      "Id: 6328903bd0854d34808aea3832ab8299\n",
      "More information: 'az ml model show -m 6328903bd0854d34808aea3832ab8299'\n",
      "Creating new driver at C:\\Users\\fboylu\\AppData\\Local\\Temp\\tmp329rld32.py\n",
      " imgscore.py\n",
      "Successfully created manifest\n",
      "Id: 291cc2c7-6783-4294-89f4-540f2e79ee1c\n",
      "More information: 'az ml manifest show -i 291cc2c7-6783-4294-89f4-540f2e79ee1c'\n",
      "Creating image..................................................................................Done.\n",
      "Image ID: b7416cdf-f3e3-44f9-821c-d7d21e5c8960\n",
      "More details: 'az ml image show -i b7416cdf-f3e3-44f9-821c-d7d21e5c8960'\n",
      "Usage information: 'az ml image usage -i b7416cdf-f3e3-44f9-821c-d7d21e5c8960'\n",
      "[Local mode] Running docker container.\n",
      "[Local mode] Pulling the image from mlcrpacrfcc24453077f.azurecr.io/imagetutlocalsrvc:6. This may take a few minutes, depending on your connection speed...\n",
      "[Local mode] Pulling...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................\n",
      "[Local mode] Waiting for container to initialize\n",
      "[Local mode] Done\n",
      "[Local mode] Service ID: imagetutlocalsrvc\n",
      "[Local mode] Usage for cmd: az ml service run realtime -i imagetutlocalsrvc -d !! YOUR DATA HERE !!\n",
      "[Local mode] Usage for powershell: az ml service run realtime -i imagetutlocalsrvc --% -d !! YOUR DATA HERE !!\n",
      "[Local mode] Additional usage information: 'az ml service usage realtime -i imagetutlocalsrvc'\n"
     ]
    }
   ],
   "source": [
    "!az ml service create realtime --model-file lightgbm_classifier.model -f imgscore.py -n $LOCAL_SERVICE_NAME -r python -c aml_config/conda_dependencies_srvc.yml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the state of the service."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name               Id                 UpdatedAt            State\n",
      "-----------------  -----------------  -------------------  -------\n",
      "imagetutlocalsrvc  imagetutlocalsrvc  2017-11-03T20:12:12  running\n"
     ]
    }
   ],
   "source": [
    "!az ml service list realtime -o table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After the web service has been successfully deployed, you can use the following command to get the service URL and other details for calling the service endpoint. You will use the Scoring URL in the output below for calling the service in the following steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Scoring URL:\n",
      "    http://127.0.0.1:32771/score\n",
      "\n",
      "Headers:\n",
      "    Content-Type: application/json\n",
      "\n",
      "Swagger URL:\n",
      "    http://127.0.0.1:32771/swagger.json\n",
      "\n",
      "Sample CLI command:\n",
      "    Usage for cmd: az ml service run realtime -i imagetutlocalsrvc -d !! YOUR DATA HERE !!\n",
      "    Usage for powershell: az ml service run realtime -i imagetutlocalsrvc --% -d !! YOUR DATA HERE !!\n",
      "\n",
      "Sample CURL call:\n",
      "    curl -X POST -H \"Content-Type:application/json\" --data !! YOUR DATA HERE !! http://127.0.0.1:32771/score\n",
      "\n",
      "Get debug logs by calling:\n",
      "    az ml service logs realtime -i imagetutlocalsrvc\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!az ml service usage realtime -i $LOCAL_SERVICE_NAME"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, you will send a request to your real-time web service. Provide the scoring URL from the output above in the following cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = 'Your Scoring URL Here!!'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will now call the service with the sample image  you extracted from the operationalization folder. Encode the image into string representation and then serialize it into json format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "img_path = 'fail.0.jpg'\n",
    "with open(img_path, 'rb') as file:\n",
    "  encoded = base64.b64encode(file.read())\n",
    "body = json.dumps(\"{}\".format(encoded))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the following code to make a prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[0.00011646905475415675]]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "headers = {'Content-Type':'application/json'}\n",
    "response = requests.post(url, headers=headers, data=body)\n",
    "prediction = json.loads(response.content.decode('ascii'))\n",
    "prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Delete local deployment to avoid any conflicts with the cluster deployment. The service can be deleted with the following command which will delete the local docker image and the container."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Service deleted.\n"
     ]
    }
   ],
   "source": [
    "!az ml service delete realtime -i $LOCAL_SERVICE_NAME"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Delete the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Resource deletion successfully submitted.\n",
      "Resources may take 1-2 minutes to be completely deprovisioned.\n"
     ]
    }
   ],
   "source": [
    "!az ml env delete -n $LOCAL_ENV_NAME -g $LOCAL_RESOURCE_GROUP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also delete the resource group you created for the local deployment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "!az group delete --name $LOCAL_RESOURCE_GROUP --yes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following cell will display a WARNING and ERROR which is expected behaviour since we deleted the resource group which leads to desired outcome of unsetting the current environment to prepare it for cluster deployment in the next section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Unable to find env with group imagetutlocalenvrg and name imagetutlocalenv. It may have been moved or deleted, or you could have the wrong active subscription set. Unsetting current env.\n",
      "WARNING: To see available environments in the subscription, run:\n",
      "  az ml env list\n",
      "To set an active environment, run:\n",
      "  az ml env set -n <env_name> -g <env_group>\n",
      "To see available subscriptions, run:\n",
      "  az account show -o table\n",
      "To set active accout, run:\n",
      "  az account set -s <subscription_name>\n",
      "\n",
      "ERROR: ResourceGroupNotFound: Resource group 'imagetutlocalenvrg' could not be found.\n"
     ]
    }
   ],
   "source": [
    "!az ml env show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cluster environment setup "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use Cluster deployment for high-scale production scenarios. It sets up an ACS cluster with Kubernetes as the orchestrator. The ACS cluster can be scaled out to handle larger throughput for your web service calls. First, we provide a name for the cluster environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "CLUSTER_SERVICE_NAME = DEPLOY_NAME + 'clus' + 'srvc'\n",
    "CLUSTER_ENV_NAME = DEPLOY_NAME + 'clus' + 'env'\n",
    "CLUSTER_RESOURCE_GROUP = CLUSTER_ENV_NAME + 'rg'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Setup cluster environment using the following command which creates the following resources in your subscription:\n",
    "- A resource group with \"rg\" added at the end of the environment name\n",
    "- A storage account\n",
    "- An Azure Container Registry (ACR)\n",
    "- A Kubernetes deployment on an Azure Container Service (ACS) cluster\n",
    "- An Application insights account"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To deploy your web service to a production environment, first set up the environment using the following command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating resource group imagetutclusenvrg\n",
      "waiting for AAD role to propagate.done\n",
      "Provisioning compute resources...\n",
      "Resource creation submitted successfully.\n",
      "Resources may take 10-20 minutes to be completely provisioned.\n",
      "To see if your environment is ready to use, run:\n",
      "  az ml env show -g imagetutclusenvrg -n imagetutclusenv\n",
      "Once your environment has successfully provisioned, you can set it as your target context using:\n",
      "  az ml env set -g imagetutclusenvrg -n imagetutclusenv\n"
     ]
    }
   ],
   "source": [
    "!az ml env setup --cluster -l $LOCATION -n $CLUSTER_ENV_NAME  --yes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can check the information on the environment with the following. You will need to wait until \"Provisioning State\" is set to \"Succeeded\" to continue with the next steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"Cluster Name\": \"imagetutclusenv\",\n",
      "  \"Cluster Size\": 2,\n",
      "  \"Created On\": \"2017-11-06T21:08:38.636Z\",\n",
      "  \"Location\": \"eastus2\",\n",
      "  \"Provisioning State\": \"Succeeded\",\n",
      "  \"Resource Group\": \"imagetutclusenvrg\",\n",
      "  \"Subscription\": \"edf507a2-6235-46c5-b560-fd463ba2e771\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "!az ml env show -g $CLUSTER_RESOURCE_GROUP -n $CLUSTER_ENV_NAME"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next step is to set the cluster environment for deployment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Compute set to imagetutclusenv.\n"
     ]
    }
   ],
   "source": [
    "!az ml env set -g $CLUSTER_RESOURCE_GROUP -n $CLUSTER_ENV_NAME --disable-dashboard"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check that environment is set to the cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"Cluster Name\": \"imagetutclusenv\",\n",
      "  \"Cluster Size\": 2,\n",
      "  \"Created On\": \"2017-11-06T21:08:38.636Z\",\n",
      "  \"Current Mode\": \"cluster\",\n",
      "  \"Location\": \"eastus2\",\n",
      "  \"Provisioning State\": \"Succeeded\",\n",
      "  \"Resource Group\": \"imagetutclusenvrg\",\n",
      "  \"Subscription\": \"edf507a2-6235-46c5-b560-fd463ba2e771\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "!az ml env show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, set the model management account for the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"created_on\": \"2017-11-03T17:51:11.9226449999999999Z\",\n",
      "  \"description\": \"\",\n",
      "  \"id\": \"/subscriptions/edf507a2-6235-46c5-b560-fd463ba2e771/resourceGroups/imagetutmmaccrg/providers/Microsoft.MachineLearningModelManagement/accounts/imagetutmmacc\",\n",
      "  \"location\": \"eastus2\",\n",
      "  \"model_management_swagger_location\": \"https://eastus2.modelmanagement.azureml.net/api/subscriptions/edf507a2-6235-46c5-b560-fd463ba2e771/resourceGroups/imagetutmmaccrg/accounts/imagetutmmacc/swagger.json?api-version=2017-09-01-preview\",\n",
      "  \"modified_on\": \"2017-11-03T17:51:11.9226449999999999Z\",\n",
      "  \"name\": \"imagetutmmacc\",\n",
      "  \"resource_group\": \"imagetutmmaccrg\",\n",
      "  \"sku\": {\n",
      "    \"capacity\": 1,\n",
      "    \"name\": \"S1\"\n",
      "  },\n",
      "  \"subscription\": \"edf507a2-6235-46c5-b560-fd463ba2e771\",\n",
      "  \"tags\": {},\n",
      "  \"type\": \"Microsoft.MachineLearningModelManagement/accounts\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "!az ml account modelmanagement set -n $MM_ACCOUNT -g $MM_ACCOUNT_RESOURCE_GROUP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deploy model to cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print your cluster service name  for creating the service in the next steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Your Cluster Service Name: imagetutclussrvc\n"
     ]
    }
   ],
   "source": [
    "print('Your Cluster Service Name: ' + CLUSTER_SERVICE_NAME)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will need to change your directory to the current directory on Workbench CLI. Use the output in the next cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Your Working Directory: C:\\Users\\fboylu\\AppData\\Local\\Temp\\azureml_runs\\imagetutorial3_1510002303767\n"
     ]
    }
   ],
   "source": [
    "print('Your Working Directory: '+ os.getcwd())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, go to the Azure Machine Learning Workbench CLI window and change your directory using:\n",
    "```\n",
    "cd [Your Working Directory Here!!]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next step is to deploy your model to the cluster environment. However, since the command below is not available through Jupyter notebooks for cluster environment set up, we will use Azure Machine Learning Workbench CLI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This command is not available through Jupyter notebooks for cluster set up.\n",
    "#!az ml service create realtime --model-file lightgbm_classifier.model -f imgscore.py -n $CLUSTER_SERVICE_NAME -r python -c aml_config/conda_dependencies_srvc.yml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Go to the Azure Machine Learning Workbench CLI window and issue the following command by providing your cluster service name printed earlier where indicated in the following command.\n",
    "```bash\n",
    "az ml service create realtime --model-file lightgbm_classifier.model -f imgscore.py -n [Your Cluster Service Name Here!!] -r python -c aml_config/conda_dependencies_srvc.yml\n",
    "```\n",
    "You can watch the steps of the deployment from the CLI output which can take up to 20 minutes. Once completed, return back to the notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List the service you deployed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name              Id                                                 UpdatedAt            State\n",
      "----------------  -------------------------------------------------  -------------------  ---------\n",
      "imagetutclussrvc  imagetutclussrvc.imagetutclusenv-47424707.eastus2  2017-11-06T16:51:30  Succeeded\n"
     ]
    }
   ],
   "source": [
    "!az ml service list realtime -o table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print your cluster service Id in order to use it in the following steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    \"Id\": \"imagetutclussrvc.imagetutclusenv-47424707.eastus2\",\n"
     ]
    }
   ],
   "source": [
    "!az ml service list realtime | grep Id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Provide your cluster service Id printed earlier in the cell below where indicated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "CLUSTER_SERVICE_ID = 'Your cluster service Id here!!'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Scoring URL:\n",
      "    http://40.79.80.172:80/api/v1/service/imagetutclussrvc/score\n",
      "\n",
      "Headers:\n",
      "    Content-Type: application/json\n",
      "    Authorization: Bearer <key>\n",
      "        (<key> can be found by running 'az ml service keys realtime -i imagetutclussrvc.imagetutclusenv-47424707.eastus2')\n",
      "\n",
      "Swagger URL:\n",
      "    http://40.79.80.172:80/api/v1/service/imagetutclussrvc/swagger.json\n",
      "\n",
      "Sample CLI command:\n",
      "    Usage for cmd: az ml service run realtime -i imagetutclussrvc.imagetutclusenv-47424707.eastus2 -d !! YOUR DATA HERE !!\n",
      "    Usage for powershell: az ml service run realtime -i imagetutclussrvc.imagetutclusenv-47424707.eastus2 --% -d !! YOUR DATA HERE !!\n",
      "\n",
      "Sample CURL call:\n",
      "    curl -X POST -H \"Content-Type:application/json\" -H \"Authorization:Bearer <key>\" --data !! YOUR DATA HERE !! http://40.79.80.172:80/api/v1/service/imagetutclussrvc/score\n",
      "\n",
      "Get debug logs by calling:\n",
      "    az ml service logs realtime -i imagetutclussrvc.imagetutclusenv-47424707.eastus2\n",
      "\n",
      "Get STDOUT/STDERR or Request/Response logs in App Insights:\n",
      "    https://analytics.applicationinsights.io/subscriptions/edf507a2-6235-46c5-b560-fd463ba2e771/resourcegroups/imagetutclusenvrg-azureml-f1412/components/mlcrpai55c93038d316#/discover/home?apptype=Other%20(preview)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!az ml service usage realtime -i $CLUSTER_SERVICE_ID"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, you can send a request to your real-time web service. Provide the scoring URL from the output above in the following cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = 'Your Scoring URL Here!!'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For cluster deployments, an api key needs to be provided. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PrimaryKey: 62bcbaaeac7e455eabd87177607dadee\n",
      "SecondaryKey: 09fec15f3d124686894345e675d3e95b\n"
     ]
    }
   ],
   "source": [
    "!az ml service keys realtime -i $CLUSTER_SERVICE_ID"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Provide the \"PrimaryKey\" from the output above as your Api Key in the following cell where indicated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "api_key = 'Your Api Key Here!!'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will now call the service with the sample image you extracted from the operationalization folder. Encode the image into string representation and then serialize it into json format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "import base64, requests, json\n",
    "img_path = 'fail.0.jpg'\n",
    "with open(img_path, 'rb') as file:\n",
    "  encoded = base64.b64encode(file.read())\n",
    "body = json.dumps(\"{}\".format(encoded))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the following code to make a prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[0.00011646905475415675]]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "headers = {'Content-Type':'application/json', 'Authorization':('Bearer '+ api_key)}\n",
    "response = requests.post(url, headers=headers, data=body)\n",
    "prediction = json.loads(response.content.decode('ascii'))\n",
    "prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The service can be deleted with the following command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "!az ml service delete realtime -i $CLUSTER_SERVICE_ID"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Delete cluster environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Resource deletion successfully submitted.\n",
      "Resources may take 1-2 minutes to be completely deprovisioned.\n"
     ]
    }
   ],
   "source": [
    "!az ml env delete -n $CLUSTER_ENV_NAME -g $CLUSTER_RESOURCE_GROUP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Delete the resource group to clean up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "!az group delete --name $CLUSTER_RESOURCE_GROUP --yes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check that current environment is unset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Unable to find env with group imagetutclusenvrg and name imagetutclusenv. It may have been moved or deleted, or you could have the wrong active subscription set. Unsetting current env.\n",
      "WARNING: To see available environments in the subscription, run:\n",
      "  az ml env list\n",
      "To set an active environment, run:\n",
      "  az ml env set -n <env_name> -g <env_group>\n",
      "To see available subscriptions, run:\n",
      "  az account show -o table\n",
      "To set active accout, run:\n",
      "  az account set -s <subscription_name>\n",
      "\n",
      "ERROR: ResourceGroupNotFound: Resource group 'imagetutclusenvrg' could not be found.\n"
     ]
    }
   ],
   "source": [
    "!az ml env show"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "imgtutvso local",
   "language": "python",
   "name": "imgtutvso_local"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}