Wyoming Faster Whisper ROCm

Quick Description

Faster whisper Running on AMD GPUs with modified CTranslate 2 Libraries served up with Wyoming protocol

System Overview

This project ties together a few projects to make Faster Whisper work with an AMD GPU. This has only been built for an AMD APU (5650G) at this time.

Host System & Hardware Requirements

• AMD GPU or APU with an architecture gfx900 or newer
  • • Check you architecture here if you don't know
• At least 100gb of disk space.
• ROCm Installed
• Docker Installed
• Docker Compose Plugin Installed

Services Required to Use this Service

• HomeAssistant
  • Wyoming Addon

Tools to Make your Life Easier

• btop
  • Great for CPU and Memory Monitoring
• AMDGPU_TOP
  • Excellent Command Line Utility for Viewing GPU utilization.

Project Components

• Wyoming
• Faster Whisper
• ROCm/Pytorch
  • Build Requires Python 3.9, the project makes use of the 20.04 image.
• CTranslate2-rocm
  • This project is essential to running Faster Whisper as the standard CTranslate2 Library does not support ROCm natively.
  • This project needs to be built to the GPU architecture specific to your hardware.

Note: The docker image required for this project is massive. On my system it is showing as 68g.4b.

Instructions:

1. Clone this repository and open the directory

git clone https://github.com/Donkey545/wyoming-faster-whisper-rocm.git
cd wyoming-faster-whisper-rocm/

2. Modify the docker-compose.yml to reflect your GPU architecture
   1. Get your architecture:

rocminfo |grep " gfx"
...
# The Response should look something like:
#  Name:                    gfx90c

2. Modify the environment section

version: "3"

services:
  whisper:
    container_name: faster-whisper-rocm
    restart: unless-stopped
    build:
      context: .
    volumes:
      - ./data:/data
    ports:
      - "10300:10300"
    devices:
      - "/dev/dri"
      - "/dev/kfd"
    environment:
      - HSA_OVERRIDE_GFX_VERSION=9.0.0 #This line can be removed if newer than VEGA

3. Run Docker Compose

docker compose up -d --build

At this point you can configure the Whisper service with Wyoming in home assistant with the <dockerhost_ip_address>:10300.

Optional Manual Build

In the docker file, comment out the build script line, and chand the entrypoint command to #ENTRYPOINT ["tail", "-f", "/dev/null"]. Exporting the Pytorch ROCm Arch as an environment variable will restrict the scope of the build to your specific architecture. This caused some problems for me on my APU, so the support may vary. Refer to the instruction on the ROCm Installation for CTranslate2-rocm.

3o. Run Docker Compose and Open an Interactive Terminal

docker compose up -d --build
...
docker exec -it faster-whisper-rocm  bash

4o. Run the build script. This will take several minutes as it is building all of the graphic architecture versions. You can scope this down with an optional environment variable.

#export PYTORCH_ROCM_ARCH=gfx1030 #optional
./src/build.sh

5o. Start the service:

./run.sh
# you can send this to background and run detached to have it operate as normal now.
# nohup run.sh  &