notebooks/02_setup_aml.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup AML"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we'll setup our Azure Machine Learning workspace as well as another storage account."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### AML Installation and configuration\n",
    "This notebook configures the notebooks in this tutorial to connect to an Azure Machine Learning (AML) Workspace.  You can use an existing workspace or create a new one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import azureml.core\n",
    "from azureml.core import Workspace\n",
    "from dotenv import set_key, get_key, find_dotenv, load_dotenv\n",
    "from pathlib import Path\n",
    "import json\n",
    "import os\n",
    "from utilities import get_auth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": [
     "install"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AML SDK Version: 1.0.17\n"
     ]
    }
   ],
   "source": [
    "print(\"AML SDK Version:\", azureml.core.VERSION) # 1.0.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set up your Azure Machine Learning workspace"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create or access an Azure ML Workspace, you will need the following information:\n",
    "\n",
    "* Your subscription id\n",
    "* A resource group name\n",
    "* A name for your workspace\n",
    "* A region for your workspace\n",
    "\n",
    "**Note**: As with other Azure services, there are limits on certain resources like cluster size associated with the Azure Machine Learning service. Please read [this article](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-manage-quotas) on the default limits and how to request more quota."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have a workspace created already, you need to get your subscription and workspace information. You can find the values for those by visiting your workspace in the [Azure portal](http://portal.azure.com). If you don't have a workspace, the create workspace command in the next section will create a resource group and a workspace using the names you provide."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Replace the values in the following cell with your information.\n",
    " \n",
    " - you can navigate to the portal to find your subscription id or enter `az account list -o table` into the cli\n",
    " - you can use `az account list-locations -o table` to find the available locations (use the values in the field 'name')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "subscription_id = \"<YOUR-SUBSCRIPTION>\"\n",
    "resource_group = \"<YOUR-RESOURCE-GROUP-NAME>\"\n",
    "workspace_name = \"<YOUR-WORKSPACE-NAME>\"\n",
    "workspace_region = \"<YOUR-REGION>\"\n",
    "storage_account_name = \"<YOUR-STORAGE-ACCOUNT-NAME>\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create and initialize a dotenv file for storing parameters used in multiple notebooks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "storage_account_name = storage_account_name.replace(\"-\",\"\")\n",
    "\n",
    "env_path = find_dotenv()\n",
    "if env_path == \"\":\n",
    "    Path(\".env\").touch()\n",
    "    env_path = find_dotenv()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": [
     "stripout"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(True, 'REGION', 'eastus')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set_key(env_path, \"SUBSCRIPTION_ID\", subscription_id) \n",
    "set_key(env_path, \"RESOURCE_GROUP\", resource_group)\n",
    "set_key(env_path, \"WORKSPACE_NAME\", workspace_name)\n",
    "set_key(env_path, \"REGION\", workspace_region)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create the workspace\n",
    "This cell will create an AML workspace for you in a subscription, provided you have the correct permissions.\n",
    "\n",
    "This will fail when:\n",
    "1. You do not have permission to create a workspace in the resource group\n",
    "2. You do not have permission to create a resource group if it's non-existing.\n",
    "2. You are not a subscription owner or contributor and no Azure ML workspaces have ever been created in this subscription\n",
    "\n",
    "If workspace creation fails, please work with your IT admin to provide you with the appropriate permissions or to provision the required resources. If this cell succeeds, you're done configuring AML!  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "create workspace"
    ]
   },
   "outputs": [],
   "source": [
    "# import the Workspace class\n",
    "ws = Workspace.create(\n",
    "    name=workspace_name,\n",
    "    subscription_id=subscription_id,\n",
    "    resource_group=resource_group,\n",
    "    location=workspace_region,\n",
    "    create_resource_group=True,\n",
    "    exist_ok=True,\n",
    "    auth=get_auth()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check the details of the workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "stripout"
    ]
   },
   "outputs": [],
   "source": [
    "ws.get_details()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's write the workspace configuration for the rest of the notebooks to connect to the workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wrote the config file config.json to: /home/mat/repos/Batch-Scoring-Deep-Learning-Models-With-AML/notebooks/aml_config/config.json\n"
     ]
    }
   ],
   "source": [
    "ws.write_config()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, you can inspect your resource group to see your active Azure Machine Learning workspace. Inside your resource group, you'll find various resources that have been generated to back the Machine Learning workspace. These resources will have a generated resource name, such as `<workspace-name>keyvaultpuvogqzk`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You are now ready to move on to the [next notebook](03_develop_pipeline.ipynb)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:batchscoringdl_aml]",
   "language": "python",
   "name": "conda-env-batchscoringdl_aml-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "source": [],
    "metadata": {
     "collapsed": false
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}