tutorials/groq-gradio/groq-gradio-tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Groq and Gradio for Realtime Voice-Powered AI Applications 🚀\n",
    "\n",
    "In this tutorial, we'll build a voice-powered AI application using Groq for realtime speech recognition and text generation, Gradio for creating an interactive web interface, and Hugging Face Spaces for hosting our application.\n",
    "\n",
    "[Groq](groq.com) is known for insanely fast inference speed that is very well-suited for realtime AI applications, providing multiple Large Language Models (LLMs) and speech-to-text models via Groq API. In this tutorial, we will use the [Distil-Whisper English](https://huggingface.co/distil-whisper/distil-large-v3) and [Llama 3 70B](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct) models for speech-to-text and text-to-text. \n",
    "\n",
    "[Gradio](https://www.gradio.app/) is an open-source Python library that makes it easy to prototype and deploy interactive demos without needing to write frontend code for a nice User Interface (UI), which is great if you're a developer like me who doesn't know much about frontend Bob Ross-ery. 🖌️\n",
    "\n",
    "By combining models powered by Groq with Gradio's user-friendly interface creation, we will:\n",
    "\n",
    "- Use Distil-Whisper English powered by Groq transcribe audio input in realtime.\n",
    "- Use Llama 3 70B powered by Groq to generate instant responses based on the transcription.\n",
    "- Create a Gradio interface to handle audio input and display results on a nice UI.\n",
    "\n",
    "Let's get started!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: Create a Free GroqCloud Account and Generate Your Groq API Key\n",
    "\n",
    "If you don't already have a GroqCloud account, you can create one for free [here](https://console.groq.com) to generate a Groq API Key. We'll need the key to be able to try out the tutorial we build! "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Import Required Libraries\n",
    "\n",
    "Let's import the libraries that allow us to interact with Groq API, handle audio processing, and create the Gradio interface:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install gradio\n",
    "!pip install groq\n",
    "!pip install numpy\n",
    "!pip install soundfile\n",
    "\n",
    "import gradio as gr\n",
    "import groq\n",
    "import io\n",
    "import numpy as np\n",
    "import soundfile as sf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Implement Audio Transcription\n",
    "\n",
    "Let's build a function to take audio input and use Distil-Whisper English (`distil-whisper-large-v3-en`) powered by Groq to transcribe the audio:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def transcribe_audio(audio, api_key):\n",
    "    if audio is None:\n",
    "        return \"\"\n",
    "    \n",
    "    client = groq.Client(api_key=api_key)\n",
    "    \n",
    "    # Convert audio to the format expected by the model\n",
    "    # The model supports mp3, mp4, mpeg, mpga, m4a, wav, and webm file types \n",
    "    audio_data = audio[1]  # Get the numpy array from the tuple\n",
    "    buffer = io.BytesIO()\n",
    "    sf.write(buffer, audio_data, audio[0], format='wav')\n",
    "    buffer.seek(0)\n",
    "\n",
    "    bytes_audio = io.BytesIO()\n",
    "    np.save(bytes_audio, audio_data)\n",
    "    bytes_audio.seek(0)\n",
    "\n",
    "    try:\n",
    "        # Use Distil-Whisper English powered by Groq for transcription\n",
    "        completion = client.audio.transcriptions.create(\n",
    "            model=\"distil-whisper-large-v3-en\",\n",
    "            file=(\"audio.wav\", buffer),\n",
    "            response_format=\"text\"\n",
    "        )\n",
    "        return completion\n",
    "    except Exception as e:\n",
    "        return f\"Error in transcription: {str(e)}\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Implement Response Generation\n",
    "\n",
    "Now, let's build a function to take the transcribed text and generate a response using Llama 3 70B (`llama3-70b-8192`) powered by Groq:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_response(transcription, api_key):\n",
    "    if not transcription:\n",
    "        return \"No transcription available. Please try speaking again.\"\n",
    "    \n",
    "    client = groq.Client(api_key=api_key)\n",
    "    \n",
    "    try:\n",
    "        # Use Llama 3 70B powered by Groq for text generation\n",
    "        completion = client.chat.completions.create(\n",
    "            model=\"llama3-70b-8192\",\n",
    "            messages=[\n",
    "                {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
    "                {\"role\": \"user\", \"content\": transcription}\n",
    "            ],\n",
    "        )\n",
    "        return completion.choices[0].message.content\n",
    "    except Exception as e:\n",
    "        return f\"Error in response generation: {str(e)}\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Process Audio and Response\n",
    "\n",
    "Next, let's create a function that calls the previous two functions we built to check that a Groq API Key was provided by the user, create the transcription, and generate the response:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def process_audio(audio, api_key):\n",
    "    if not api_key:\n",
    "        return \"Please enter your Groq API key.\", \"API key is required.\"\n",
    "    transcription = transcribe_audio(audio, api_key)\n",
    "    response = generate_response(transcription, api_key)\n",
    "    return transcription, response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6: Build Web Interface with Gradio\n",
    "\n",
    "Finally, we'll use Gradio and the easy-to-use UI components that it provides for us to build out a simple interface for our project:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Custom CSS for the Groq badge and color scheme (feel free to edit however you wish)\n",
    "custom_css = \"\"\"\n",
    ".gradio-container {\n",
    "    background-color: #f5f5f5;\n",
    "}\n",
    ".gr-button-primary {\n",
    "    background-color: #f55036 !important;\n",
    "    border-color: #f55036 !important;\n",
    "}\n",
    ".gr-button-secondary {\n",
    "    color: #f55036 !important;\n",
    "    border-color: #f55036 !important;\n",
    "}\n",
    "#groq-badge {\n",
    "    position: fixed;\n",
    "    bottom: 20px;\n",
    "    right: 20px;\n",
    "    z-index: 1000;\n",
    "}\n",
    "\"\"\"\n",
    "\n",
    "with gr.Blocks(theme=gr.themes.Default()) as demo:\n",
    "    gr.Markdown(\"# 🎙️ Groq x Gradio Voice-Powered AI Assistant\")\n",
    "    \n",
    "    api_key_input = gr.Textbox(type=\"password\", label=\"Enter your Groq API Key\")\n",
    "    \n",
    "    with gr.Row():\n",
    "        audio_input = gr.Audio(label=\"Speak!\", type=\"numpy\")\n",
    "    \n",
    "    with gr.Row():\n",
    "        transcription_output = gr.Textbox(label=\"Transcription\")\n",
    "        response_output = gr.Textbox(label=\"AI Assistant Response\")\n",
    "    \n",
    "    submit_button = gr.Button(\"Process\", variant=\"primary\")\n",
    "    \n",
    "    # Add the Groq badge\n",
    "    gr.HTML(\"\"\"\n",
    "    <div id=\"groq-badge\">\n",
    "        <div style=\"color: #f55036; font-weight: bold;\">POWERED BY GROQ</div>\n",
    "    </div>\n",
    "    \"\"\")\n",
    "    \n",
    "    submit_button.click(\n",
    "        process_audio,\n",
    "        inputs=[audio_input, api_key_input],\n",
    "        outputs=[transcription_output, response_output]\n",
    "    )\n",
    "    \n",
    "    gr.Markdown(\"\"\"\n",
    "    ## How to use this app:\n",
    "    1. Enter your Groq API Key in the provided field.\n",
    "    2. Click on the microphone icon and speak your message (or forever hold your peace)! You can also provide a supported audio file. Supported audio files include mp3, mp4, mpeg, mpga, m4a, wav, and webm file types.\n",
    "    3. Click the \"Process\" button to transcribe your speech and generate a response from our AI assistant.\n",
    "    4. The transcription and AI assistant response will appear in the respective text boxes.\n",
    "    \n",
    "    \"\"\")\n",
    "\n",
    "demo.launch()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 7: Host on HuggingFace Spaces\n",
    "\n",
    "If you don't already have one, create a free Hugging Face account [here](https://huggingface.co/join). To deploy our Gradio app to Hugging Face Spaces from our browser, all we have to do is drag and drop all related files [here](https://huggingface.co/new-space). In this case, we'll create an `app.py` file as well as a `requirements.txt` file. \n",
    "\n",
    "In the `app.py` file, simply copy-paste the code. \n",
    "\n",
    "In the `requirements.txt` file, add in all the required dependencies for Hugging Face Spaces to detect and automatically install before deploying our application to a public link that anyone can access! \n",
    "\n",
    "For this project, the following dependencies were added to the `requirements.txt` file:\n",
    "\n",
    "```\n",
    "gradio==4.19.2\n",
    "groq==0.10.0\n",
    "numpy==1.26.4\n",
    "soundfile==0.12.1\n",
    "```\n",
    "\n",
    "Once the required application files are added, Hugging Face Spaces will automatically detect, build, run, and deploy our application! You can see and try this tutorial live [here](https://huggingface.co/spaces/Groq/groq-gradio-voice-assistant)! 😁"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion\n",
    "By combining Groq, Gradio, and Hugging Face Spaces, we've built and deployed a voice-powered AI assistant with just a few lines of code and learned how easy it is to create powerful, interactive AI applications! \n",
    "\n",
    "Feel free to experiment with this code, try different prompts, or extend the functionality to create your own personal project! 🤩"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}