Skip to content

Commit

Permalink
Fixing code snippet for GenAI inference on NPU
Browse files Browse the repository at this point in the history
Signed-off-by: sgolebiewski-intel <[email protected]>
  • Loading branch information
sgolebiewski-intel committed Dec 17, 2024
1 parent 919a04c commit bebc58e
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ which do not require specifying quantization parameters:
| Below is a list of such models:
* meta-llama/Meta-Llama-3-8B-Instruct
* meta-llama/Llama-3.1-8B
* microsoft/Phi-3-mini-4k-instruct
* Qwen/Qwen2-7B
* mistralai/Mistral-7B-Instruct-v0.2
Expand Down Expand Up @@ -136,6 +137,7 @@ you need to add ``do_sample=False`` **to the** ``generate()`` **method:**
ov::genai::GenerationConfig config;
config.do_sample=false;
config.max_new_tokens=100;
ov::genai::LLMPipeline pipe(models_path, "NPU");
std::cout << pipe.generate("The Sun is yellow because", config);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ make sure to :doc:`install OpenVINO with GenAI <../../get-started/install-openvi
.. dropdown:: Text-to-Image Generation

OpenVINO GenAI introduces the openvino_genai.Text2ImagePipeline for inference of text-to-image
OpenVINO GenAI introduces ``openvino_genai.Text2ImagePipeline`` for inference of text-to-image
models such as: as Stable Diffusion 1.5, 2.1, XL, LCM, Flex, and more.
See the following usage example for reference.

Expand Down Expand Up @@ -916,7 +916,7 @@ running the following code:
GenAI API
#######################################

The use case described here uses the following OpenVINO GenAI API classes:
The use case described here regards the following OpenVINO GenAI API classes:

* generation_config - defines a configuration class for text generation,
enabling customization of the generation process such as the maximum length of
Expand Down

0 comments on commit bebc58e

Please sign in to comment.