Skip to content
This repository has been archived by the owner on Nov 16, 2023. It is now read-only.

Latest commit

 

History

History

recipes

Readme.md

Getting Started with the Recipes

Prerequisites

  1. Azure subscription. This can be a free trial subscription, MSDN, or the one you use for other work.
  2. Azure Python SDK and azure-mgmt-batchai, if you like to run recipes using Python Jupyter notebook. See How to install Azure SDK.
  3. Azure CLI 2.0, if you like to run recipes using Azure CLI - See Install Azure CLI 2.0 for instructions.
  4. Azure Storage Account in East US (required for all recipes). See How to create Azure storage accounts

Preparation

Make a Local Copy of Repo.

To start, please Clone or download this repo

Install Azure CLI 2.0 and Configure Azure CLI 2.0

Please follow Azure CLI 2.0 Batch AI specific documentation to install and configure Azure CLI 2.0 for using with Batch AI.

Run Setup Wizard Script to Enable Batch AI

We provide setup wizard scripts of setup.ps1 for Windows users and setup.sh for Linux users. Please execute the script from you cloned recipe directory <your clone root>/BatchAI/recipes.

Note that Azure CLI 2.0 is required for the setup scripts. Please follow documentation to install and configure Azure CLI 2.0. For Linux Users, jq package is also required to process json data.

Manual Preparation Setup

Alternative to use setup wizard scripts. You can also configure Batch AI manually. Please follow Preparation.md for instructions.

Recipe Instructions

Use the following links for a quick navigation:

  1. Run Recipes Using Python Jupyter notebook
  2. Run Recipes Using Azure CLI 2.0

Run Recipes Using Python Jupyter notebook

Helper functions in utilities

For your convenience, we provide a collection of helper functions in BatchAI/utilities used for each recipes:

  • Read parameters from configuration file
  • Create python client object (BatchAIManagementClient) to access Azure Batch AI service
  • Create/Update resource group
  • Download file with given shared access signature (SAS)
  • Print Job/Cluster status
  • File Streaming

Install Azure Python SDK

Since all recipes utlize APIs from other Azure products (e.g, Azure storage, credentials), it is also required to install the full package of Azure Python SDK:

pip install azure

Install Azure Batch AI Management Client

Install Batch AI management client using the following command:

pip install azure-mgmt-batchai --upgrade

Install Jupyter Notebook

Please install Jupyter Notebook from https://jupyter.org/ or run

python -m pip install jupyter

Start to Run Recipes

  • Route into the root your cloned recipe directory
cd <your clone root>/BatchAI/recipes
  • Launch the Jupyter Notebook by
jupyter notebook
  • In the prompted brower brower, navigate into the recipe of interest, and start the *.ipynb file.

Run Recipes Using Azure CLI 2.0

Install Azure CLI 2.0 and Configure Azure CLI 2.0

Please follow Azure CLI 2.0 Batch AI specific documentation to install and configure Azure CLI 2.0 for using with Batch AI.

Generate Authentication Key for SSH (for Cloud Shell and GNU/Linux Users)

During Cluster and File Server creation you will need to specify a name and authentication method for administrator account which will be created on each compute node (you can use this account to ssh to the node).

You can provide a password and/or ssh public key as authentication method via --password (-p) and --ssh-public-key (-k) parameters.

GNU/Linux users (including Cloud Shell users) can generate authentication key for ssh using ssh-keygen command.

Note, GNU/Linux part of recipes expects you to have a public ssh key at ~/.ssh/id_rsa.pub, if you prefer to use different ssh key, please update -k parameter value.

Install unzip package (for GNU/Linux Users)

Training data used in recipes is compressed in zip archives and requires unzip utility to be installed on the host, please install it using your distribution package manager.

Cloud Shell has unzip already installed.

Run Recipes

Each recipe contains cli-instructions.md file which describes input data, cluster and job configuration and provides instructions for cluster and job creation.

Help or Feedback

If you have any problems or questions, you can reach the Batch AI team at [email protected] or you can create an issue on GitHub.

We also welcome your contributions of additional sample notebooks, scripts, or other examples of working with Batch AI.