forked from microsoft/SynapseML
-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: add prerequisites - openai and cognitive services resources (mi…
…crosoft#2008) * add prerequisites - openai and cognitive services resources * Update notebooks/documentation/prerequisites_azure_resources.ipynb Co-authored-by: Mark Hamilton <[email protected]> * Update notebooks/documentation/prerequisites_platforms.ipynb Co-authored-by: Mark Hamilton <[email protected]> * Update notebooks/documentation/prerequisites_azure_resources.ipynb Co-authored-by: Mark Hamilton <[email protected]> * Update notebooks/documentation/prerequisites_azure_resources.ipynb Co-authored-by: Mark Hamilton <[email protected]> * Update notebooks/documentation/prerequisites_azure_resources.ipynb Co-authored-by: Mark Hamilton <[email protected]> * Update notebooks/documentation/prerequisites_azure_resources.ipynb Co-authored-by: Mark Hamilton <[email protected]> * Update notebooks/documentation/prerequisites_platforms.ipynb Co-authored-by: Mark Hamilton <[email protected]> * Update notebooks/documentation/prerequisites_platforms.ipynb Co-authored-by: Mark Hamilton <[email protected]> * Update notebooks/documentation/prerequisites_platforms.ipynb Co-authored-by: Mark Hamilton <[email protected]> * update setting up docs * Update notebooks/documentation/prerequisites_azure_resources.ipynb Co-authored-by: Mark Hamilton <[email protected]> * Update notebooks/documentation/prerequisites_azure_resources.ipynb Co-authored-by: Mark Hamilton <[email protected]> * Apply suggestions from code review Co-authored-by: Mark Hamilton <[email protected]> --------- Co-authored-by: Mark Hamilton <[email protected]>
- Loading branch information
1 parent
2d1ee53
commit e17756e
Showing
2 changed files
with
162 additions
and
0 deletions.
There are no files selected for viewing
79 changes: 79 additions & 0 deletions
79
notebooks/documentation/prerequisites_azure_resources.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Setting up Cognitive Services and Azure OpenAI resources for SynapseML \n", | ||
| "\n", | ||
| "In order to use SynapseML's OpenAI or Cognitive Services features, specific Azure resources are required. This documentation walks you through the process of setting up these resources and acquiring the necessary credentials.\n", | ||
| "\n", | ||
| "To utilize SynapseML's OpenAI or Cognitive Services features, you need to create resources and obtain the API keys. " | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": { | ||
| "tags": [ | ||
| "azure-prerequisites" | ||
| ] | ||
| }, | ||
| "source": [ | ||
| "You need a valid Azure subscription to create resources.\n", | ||
| "* A valid Azure subscription - [Create one for free](https://azure.microsoft.com/free/cognitive-services/)." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Azure OpenAI resource\n", | ||
| "\n", | ||
| "The [Azure OpenAI service](https://azure.microsoft.com/products/cognitive-services/openai-service/) can be used to solve a large number of natural language tasks through prompting the completion API. To make it easier to scale your prompting workflows from a few examples to large datasets of examples, we have integrated the Azure OpenAI service with the distributed machine learning library SynapseML. This integration makes it easy to use the Apache Spark distributed computing framework to process millions of prompts with the OpenAI service." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": { | ||
| "tags": [ | ||
| "openai-prerequisites" | ||
| ] | ||
| }, | ||
| "source": [ | ||
| "To set up your Azure OpenAI Resource for SynapseML usage you need to: \n", | ||
| "* [Apply for access to Azure OpenAI](https://aka.ms/oai/access) if you do not already have access. \n", | ||
| "* [Create an Azure OpenAI resource](https://docs.microsoft.com/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource) \n", | ||
| "* Get your Azure OpenAI resource's key. After your resource is successfully deployed, select **Next Steps** > **Go to resource**. Once at the resource, you can get the key from **Resource Management** > **Keys and Endpoint**. Copy the key and paste it into the notebook. Store keys securely and do not share them. " | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Cognitive Services resource" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": { | ||
| "tags": [ | ||
| "cognitive-services-prerequisites" | ||
| ] | ||
| }, | ||
| "source": [ | ||
| "To set up [Cognitive Services](https://azure.microsoft.com/products/cognitive-services/) for use with SynapseML you first need to:\n", | ||
| "* [Assign yourself the Cognitive Services Contributor role](https://learn.microsoft.com/azure/role-based-access-control/role-assignments-steps) to agree to the responsible AI terms and create a resource. \n", | ||
| "* [Create an Azure Cognitive multi-service (Decision, Language, Speech, Vision) resource](https://portal.azure.com/#create/Microsoft.CognitiveServicesAllInOne). Alternatively, you can follow the steps to [create Single-service resource](https://learn.microsoft.com/en-us/azure/cognitive-services/cognitive-services-apis-create-account?tabs=decision%2Canomaly-detector%2Clanguage-service%2Ccomputer-vision%2Cwindows#create-a-new-azure-cognitive-services-resource). \n", | ||
| "* Get your Cognitive Service resource's key. After your resource is successfully deployed, select **Next Steps** > **Go to resource**. Once at the resource, you can get the key from **Resource Management** > **Keys and Endpoint**. Copy the key and paste it into the notebook. Store keys securely and do not share them. " | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "language_info": { | ||
| "name": "python" | ||
| }, | ||
| "orig_nbformat": 4 | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Setting up your computing platform for SynapseML \n", | ||
| "\n", | ||
| "SynapseML is preinstalled on Microsoft Fabric and Synapse Analytics. Follow the instructions to get started with these platforms." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Microsoft Fabric\n", | ||
| "[Microsoft Fabric](https://www.microsoft.com/microsoft-fabric/) is an all-in-one analytics solution for enterprises that covers everything from data movement to data science, Real-Time Analytics, and business intelligence. It offers a comprehensive suite of services, including data lake, data engineering, and data integration, all in one place.\n", | ||
| "\n", | ||
| "SynapseML is preinstalled on Fabric, and this guide will walk you through getting access to fabric." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": { | ||
| "tags": [ | ||
| "fabric-prerequisites" | ||
| ] | ||
| }, | ||
| "source": [ | ||
| "* [Get a Microsoft Fabric license](https://learn.microsoft.com/fabric/enterprise/licenses) or sign-up for a free [Microsoft Fabric (Preview) trial](https://learn.microsoft.com/fabric/get-started/fabric-trial).\n", | ||
| "* Sign in to [Microsoft Fabric](https://fabric.microsoft.com/)\n", | ||
| "* Go to the Data Science experience.\n", | ||
| "* [Create a new notebook](https://learn.microsoft.com/en-us/fabric/data-engineering/how-to-use-notebook#create-notebooks) or attach your notebook to a lakehouse. On the left side, select **Add** to add an existing lakehouse or [create a lakehouse](https://learn.microsoft.com/en-us/fabric/data-engineering/how-to-use-notebook#connect-lakehouses-and-notebooks)." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "SynapseML is preinstalled on Fabric, but if you want to use another version of SynapseML, follow [this guide on updating SynapseML](https://learn.microsoft.com/en-us/fabric/data-science/install-synapseml)." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Synapse Analytics\n", | ||
| "[Azure Synapse Analytics](https://azure.microsoft.com/products/synapse-analytics) is an enterprise analytics service that accelerates time to insight across data warehouses and big data systems.\n", | ||
| "\n", | ||
| "SynapseML is preinstalled on Synapse Analytics. To start with Synapse Analytics, you need:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": { | ||
| "tags": [ | ||
| "synapse-prerequisites" | ||
| ] | ||
| }, | ||
| "source": [ | ||
| "* A valid Azure subscription - [Create one for free](https://azure.microsoft.com/free/cognitive-services/).\n", | ||
| "* [Create a Synapse workspace and launch Synapse studio](https://docs.microsoft.com/en-us/azure/synapse-analytics/get-started-create-workspace)\n", | ||
| "* [Create a serverless Apache Spark pool](https://docs.microsoft.com/en-us/azure/synapse-analytics/get-started-analyze-spark#create-a-serverless-apache-spark-pool)\n", | ||
| "* Once Synapse Studio has launched, select **Develop**. Then, select the **\"+\"** icon to add a new resource. From there, select **Notebook**. A new notebook is created and opened. Alternatively, you can select **Import** to upload your notebook." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "SynapseML is preinstalled on Azure Synapse Analytics, but if you want to use another version of SynapseML, follow [this guide on updating SynapseML](https://microsoft.github.io/SynapseML/docs/getting_started/installation/#synapse)." | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "language_info": { | ||
| "name": "python" | ||
| }, | ||
| "orig_nbformat": 4 | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } |