Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds OpenAI inference service notebook #154

Merged
merged 8 commits into from
Jan 17, 2024
+396 −0
Merged

Adds OpenAI inference service notebook #154

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a61f99d
Adds openai inference service notebook.
szabosteve Jan 5, 2024
4c7ab00
Apply suggestions from code review
szabosteve Jan 5, 2024
130b25c
[DOCS] Refines notebook.
szabosteve Jan 10, 2024
127ce4f
[DOCS] Resolves conflict.
szabosteve Jan 10, 2024
26d77c6
[DOCS] Updates notebook.
szabosteve Jan 10, 2024
e23a2a3
Adds output to search.
szabosteve Jan 10, 2024
9c4cf2b
Update notebooks/search/07-inference.ipynb
szabosteve Jan 10, 2024
020a82b
Cleans up outputs.
szabosteve Jan 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
381 changes: 381 additions & 0 deletions notebooks/search/07-inference.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,381 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "7a765629",
"metadata": {},
"source": [
"# Semantic Search using the Inference API\n",
"\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/elastic/elasticsearch-labs/blob/main/notebooks/search/07-inference.ipynb)\n",
"\n",
"\n",
"Learn how to use the [Inference API](https://www.elastic.co/guide/en/elasticsearch/reference/current/inference-apis.html) for semantic search."
]
},
{
"cell_type": "markdown",
"id": "9c99b06d",
"metadata": {},
"source": [
"# 🧰 Requirements\n",
"\n",
"For this example, you will need:\n",
"\n",
"- An Elastic deployment with minimum **4GB machine learning node**\n",
szabosteve marked this conversation as resolved.
Show resolved Hide resolved
" - We'll be using [Elastic Cloud](https://www.elastic.co/guide/en/cloud/current/ec-getting-started.html) for this example (available with a [free trial](https://cloud.elastic.co/registration?utm_source=github&utm_content=elasticsearch-labs-notebook))\n",
" \n",
"- An [OpenAI account](https://openai.com/) is required to use the Inference API with \n",
"the OpenAI service. "
]
},
{
"cell_type": "markdown",
"id": "15193c10",
"metadata": {},
"source": [
"# Create Elastic Cloud deployment\n",
"\n",
"If you don't have an Elastic Cloud deployment, sign up [here](https://cloud.elastic.co/registration?utm_source=github&utm_content=elasticsearch-labs-notebook) for a free trial.\n",
"\n",
"- Go to the [Create deployment](https://cloud.elastic.co/deployments/create) page\n",
" - Under **Advanced settings**, go to **Machine Learning instances**\n",
" - You'll need at least **4GB** RAM per zone for this tutorial\n",
szabosteve marked this conversation as resolved.
Show resolved Hide resolved
" - Select **Create deployment**"
]
},
{
"cell_type": "markdown",
"id": "f27dffbf",
"metadata": {},
"source": [
"# Install packages and connect with Elasticsearch Client\n",
"\n",
"To get started, we'll need to connect to our Elastic deployment using the Python client.\n",
"Because we're using an Elastic Cloud deployment, we'll use the **Cloud ID** to identify our deployment.\n",
"\n",
"First we need to `pip` install the following packages:\n",
"\n",
"- `elasticsearch`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8c4b16bc",
"metadata": {},
"outputs": [],
"source": [
"!pip install elasticsearch"
]
},
{
"cell_type": "markdown",
"id": "41ef96b3",
"metadata": {},
"source": [
"Next, we need to import the modules we need. 🔐 NOTE: getpass enables us to securely prompt the user for credentials without echoing them to the terminal, or storing it in memory."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "690ff9af",
"metadata": {},
"outputs": [],
"source": [
"from elasticsearch import Elasticsearch, helpers\n",
"from urllib.request import urlopen\n",
"import getpass\n",
"import json\n",
"import time"
]
},
{
"cell_type": "markdown",
"id": "23fa2b6c",
"metadata": {},
"source": [
"Now we can instantiate the Python Elasticsearch client.\n",
"\n",
"First we prompt the user for their password and Cloud ID.\n",
"Then we create a `client` object that instantiates an instance of the `Elasticsearch` class."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "195cc597",
"metadata": {},
"outputs": [],
"source": [
"# Found in the 'Manage Deployment' page\n",
"CLOUD_ID = getpass.getpass('Enter Elastic Cloud ID: ')\n",
"\n",
"# Password for the 'elastic' user generated by Elasticsearch\n",
"ELASTIC_PASSWORD = getpass.getpass('Enter Elastic password: ')\n",
"\n",
"# Create the client instance\n",
"client = Elasticsearch(\n",
" cloud_id=CLOUD_ID,\n",
" basic_auth=(\"elastic\", ELASTIC_PASSWORD)\n",
")"
szabosteve marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
"cell_type": "markdown",
"id": "b1115ffb",
"metadata": {},
"source": [
"Confirm that the client has connected with this test:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cc0de5ea",
"metadata": {},
"outputs": [],
"source": [
"print(client.info())"
]
},
{
"cell_type": "markdown",
"id": "4e9e7354",
"metadata": {},
"source": [
"Refer to [the documentation](https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/connecting.html#connect-self-managed-new) to learn how to connect to a self-managed deployment.\n",
"\n",
"Read [this page](https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/connecting.html#connect-self-managed-new) to learn how to connect using API keys.\n"
]
},
{
"cell_type": "markdown",
"id": "96788aa1",
"metadata": {},
"source": [
"## Create the inference task\n",
"\n",
"Let's create the inference task by using the [Create inference API](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-inference-api.html).\n",
"\n",
"You'll need an OpenAI API key for this that you can find in your OpenAI account under the [API keys section](https://platform.openai.com/api-keys)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3e6d98af",
"metadata": {},
"outputs": [],
"source": [
"API_KEY = getpass.getpass('Enter OpenAI API key: ')\n",
"\n",
"client.inference.put_model(\n",
" task_type=\"text_embedding\",\n",
" model_id=\"openai_embeddings\",\n",
" body={\n",
" \"service\": \"openai\",\n",
" \"service_settings\": {\n",
" \"api_key\": API_KEY\n",
" },\n",
" \"task_settings\": {\n",
" \"model\": \"text-embedding-ada-002\"\n",
" }\n",
" }\n",
")"
]
},
{
"cell_type": "markdown",
"id": "e5feaf12",
"metadata": {},
"source": [
"## Create an ingest pipeline with an inference processor\n",
"\n",
"Create an ingest pipeline with an inference processor by using the [`put_pipeline`](https://www.elastic.co/guide/en/elasticsearch/reference/master/put-pipeline-api.html) method. Reference the OpenAI model created above to infer against the data that is being ingested in the pipeline."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c5897fe4",
"metadata": {},
"outputs": [],
"source": [
"client.ingest.put_pipeline(\n",
" id=\"openai_embeddings\", \n",
szabosteve marked this conversation as resolved.
Show resolved Hide resolved
" description=\"Ingest pipeline for OpenAI inference.\",\n",
" processors=[\n",
" {\n",
" \"inference\": {\n",
" \"model_id\": \"openai_embeddings\",\n",
" \"input_output\": {\n",
" \"input_field\": \"plot\",\n",
" \"output_field\": \"plot_embedding\"\n",
" }\n",
" }\n",
" }\n",
" ]\n",
")"
]
},
{
"cell_type": "markdown",
"id": "7b6dd89c",
"metadata": {},
"source": [
"Let's note a few important parameters from that API call:\n",
"\n",
"- `inference`: A processor that performs inference using a machine learning model.\n",
"- `model_id`: Specifies the ID of the machine learning model to be used. In this example, the model ID is set to `openai_embeddings`.\n",
szabosteve marked this conversation as resolved.
Show resolved Hide resolved
"- `input_output`: Specifies input and output fields.\n",
"- `input_field`: Field name from which the `dense_vector` representation is created.\n",
"- `output_field`: Field name which contains inference results. "
]
},
{
"cell_type": "markdown",
"id": "f167c8cf",
"metadata": {},
"source": [
"## Create index\n",
"\n",
"The mapping of the destination index - the index that contains the embeddings that the model will create based on your input text - must be created. The destination index must have a field with the [dense_vector](https://www.elastic.co/guide/en/elasticsearch/reference/current/dense-vector.html) field type to index the output of the OpenAI model.\n",
"\n",
"Let's create an index named `openai-movie-embeddings` with the mappings we need."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "37558907",
"metadata": {},
"outputs": [],
"source": [
"client.indices.delete(index=\"openai-movie-embeddings\", ignore_unavailable=True)\n",
"client.indices.create(\n",
" index=\"openai-movie-embeddings\",\n",
" settings={\n",
" \"index\": {\n",
" \"default_pipeline\": \"openai_embeddings\"\n",
" }\n",
" },\n",
" mappings={\n",
" \"properties\": {\n",
" \"plot_embedding\": { \n",
" \"type\": \"dense_vector\", \n",
" \"dims\": 1536, \n",
" \"element_type\": \"byte\",\n",
szabosteve marked this conversation as resolved.
Show resolved Hide resolved
" \"similarity\": \"dot_product\" \n",
" },\n",
" \"plot\": { \n",
" \"type\": \"text\" \n",
" }\n",
" }\n",
" }\n",
")"
]
},
{
"cell_type": "markdown",
"id": "e9d4bfd2",
"metadata": {},
"source": [
"## Insert Documents (option 1)\n",
"\n",
"Let's insert our example dataset of 12 movies."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cfa8eda5",
"metadata": {},
"outputs": [],
"source": [
"url = \"https://raw.githubusercontent.com/elastic/elasticsearch-labs/main/notebooks/search/movies.json\"\n",
"response = urlopen(url)\n",
"\n",
"# Load the response data into a JSON object\n",
"data_json = json.loads(response.read())\n",
"\n",
"# Prepare the documents to be indexed\n",
"documents = []\n",
"for doc in data_json:\n",
" documents.append({\n",
" \"_index\": \"openai-movie-embeddings\",\n",
" \"_source\": doc,\n",
" })\n",
"\n",
"# Use helpers.bulk to index\n",
"helpers.bulk(client, documents)\n",
"\n",
"print(\"Done indexing documents into `openai-movie-embeddings` index!\")\n",
"time.sleep(3)"
]
},
{
"cell_type": "markdown",
"id": "a68e808e",
"metadata": {},
"source": [
"## Semantic search\n",
"\n",
"After the dataset has been enriched with the embeddings, you can query the data using [semantic search](https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html#knn-semantic-search). Pass a `query_vector_builder` to the k-nearest neighbor (kNN) vector search API, and provide the query text and the model you have used to create the embeddings."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a47cdc60",
"metadata": {},
"outputs": [],
"source": [
"response = client.search(\n",
" index='openai-movie-embeddings', \n",
" size=3,\n",
" knn={\n",
" \"field\": \"plot_embedding\",\n",
" \"query_vector_builder\": {\n",
" \"text_embedding\": {\n",
" \"model_id\": \"openai_embeddings\",\n",
" \"model_text\": \"Fighting movie\"\n",
" }\n",
" },\n",
" \"k\": 10,\n",
" \"num_candidates\": 100\n",
" }\n",
")\n",
"\n",
"for hit in response['hits']['hits']:\n",
" doc_id = hit['_id']\n",
" score = hit['_score']\n",
" title = hit['_source']['title']\n",
" plot = hit['_source']['plot']\n",
" print(f\"Score: {score}\\nTitle: {title}\\nPlot: {plot}\\n\")"
szabosteve marked this conversation as resolved.
Show resolved Hide resolved
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}