From cc763f5134f5f84b3020a8ea1bee409a60d15218 Mon Sep 17 00:00:00 2001 From: Dina Suehiro Jones Date: Wed, 26 Jun 2024 18:29:06 -0700 Subject: [PATCH] Update the Gaudi container example in the README (#1885) --- README.md | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index b4155dbdde7..903af342511 100644 --- a/README.md +++ b/README.md @@ -46,18 +46,13 @@ pip install "neural-compressor>=2.3" "transformers>=4.34.0" torch torchvision After successfully installing these packages, try your first quantization program. ### Weight-Only Quantization (LLMs) -Following example code demonstrates Weight-Only Quantization on LLMs, it supports Intel CPU, Intel Gauid2 AI Accelerator, Nvidia GPU, best device will be selected automatically. +Following example code demonstrates Weight-Only Quantization on LLMs, it supports Intel CPU, Intel Gaudi2 AI Accelerator, Nvidia GPU, best device will be selected automatically. To try on Intel Gaudi2, docker image with Gaudi Software Stack is recommended, please refer to following script for environment setup. More details can be found in [Gaudi Guide](https://docs.habana.ai/en/latest/Installation_Guide/Bare_Metal_Fresh_OS.html#launch-docker-image-that-was-built). ```bash +# Run a container with an interactive shell docker run -it --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none --cap-add=sys_nice --net=host --ipc=host vault.habana.ai/gaudi-docker/1.14.0/ubuntu22.04/habanalabs/pytorch-installer-2.1.1:latest -# Check the container ID -docker ps - -# Login into container -docker exec -it bash - # Install the optimum-habana pip install --upgrade-strategy eager optimum[habana]