[DOCS] Adding GenAI Use Cases #27062
Conversation
sgolebiewski-intel
requested review from
andrei-kochin,
tsavina and
kblaszczak-intel
docs/articles_en/learn-openvino/llm_inference_guide/genai-guide/genai-use-cases.rst
Outdated
Show resolved
Hide resolved
docs/articles_en/learn-openvino/llm_inference_guide/genai-guide/genai-use-cases.rst
Outdated
Show resolved
Hide resolved
docs/articles_en/learn-openvino/llm_inference_guide/genai-guide/genai-use-cases.rst
Outdated
Show resolved
Hide resolved
Co-authored-by: Karol Blaszczak <[email protected]>
andrei-kochin
requested review from
Wovchena,
ilya-lavrenov,
as-suvorov and
likholat
docs/articles_en/learn-openvino/llm_inference_guide/genai-guide/genai-use-cases.rst
Show resolved
Hide resolved
docs/articles_en/learn-openvino/llm_inference_guide/genai-guide/genai-use-cases.rst
Show resolved
Hide resolved
docs/articles_en/learn-openvino/llm_inference_guide/genai-guide/genai-use-cases.rst
Outdated
Show resolved
Hide resolved
| #include "openvino/genai/text2image/pipeline.hpp" | ||
|
|
||
| #include "imwrite.hpp" | ||
|
|
||
| int32_t main(int32_t argc, char* argv[]) { | ||
| OPENVINO_ASSERT(argc >= 3 && (argc - 3) % 2 == 0, "Usage: ", argv[0], " <MODEL_DIR> '<PROMPT>' [<LORA_SAFETENSORS> <ALPHA> ...]]"); | ||
|
|
||
| const std::string models_path = argv[1], prompt = argv[2]; | ||
| const std::string device = "CPU"; // GPU, NPU can be used as well. | ||
|
|
||
| ov::genai::AdapterConfig adapter_config; | ||
| // Applying Multiple LoRA adapters simultaneously is supported. Parse them all and the corresponding alphas from cmd parameters: | ||
| for(size_t i = 0; i < (argc - 3)/2; ++i) { | ||
| ov::genai::Adapter adapter(argv[3 + 2*i]); | ||
| float alpha = std::atof(argv[3 + 2*i + 1]); | ||
| adapter_config.add(adapter, alpha); | ||
| } | ||
|
|
||
| // LoRA adapters passed to the constructor will be activated by default in the next generation. | ||
| ov::genai::Text2ImagePipeline pipe(models_path, device, ov::genai::adapters(adapter_config)); | ||
|
|
||
| std::cout << "Generating image with LoRA adapters applied, resulting image will be in lora.bmp\n"; | ||
| ov::Tensor image = pipe.generate(prompt, | ||
| ov::genai::random_generator(std::make_shared<ov::genai::CppStdGenerator>(42)), | ||
| ov::genai::width(512), | ||
| ov::genai::height(896), | ||
| ov::genai::num_inference_steps(20)); | ||
| imwrite("lora.bmp", image, true); | ||
|
|
||
| std::cout << "Generating image without LoRA adapters applied, resulting image will be in baseline.bmp\n"; | ||
| image = pipe.generate(prompt, | ||
| ov::genai::adapters(), // Passing adapters as generation overrides set in the constructor; adapters() means no adapters. | ||
| ov::genai::random_generator(std::make_shared<ov::genai::CppStdGenerator>(42)), | ||
| ov::genai::width(512), | ||
| ov::genai::height(896), | ||
| ov::genai::num_inference_steps(20)); | ||
| imwrite("baseline.bmp", image, true); | ||
|
|
||
| return EXIT_SUCCESS; | ||
| } |
There was a problem hiding this comment.
Can we prepare less bulky code snippet to be used as on-line illustration? Is there a goal to use exactly the same code as in samples? I think we can make reader life easier with a more distilled code.
|
|
||
| .. code-block:: cpp | ||
|
|
||
| int main(int argc, char* argv[]) { |
There was a problem hiding this comment.
Please use the built-in code from the code sample
There was a problem hiding this comment.
Can we reach a decision here? @slyalin I am taking into consideration your comment regarding distilling the code snippets.
There was a problem hiding this comment.
Currently, we see 2 possible solutions.
-
We can keep the whole built-in code, while commenting out some lines of code, thus leaving the part on which users can focus.
-
We keep the distilled code and provide a link to the source example.
Please, share your thoughts.
There was a problem hiding this comment.
Is there an option to embed the sample from a repo? Avoiding copy would be the best solution
There was a problem hiding this comment.
Or is it called built-in code?
There was a problem hiding this comment.
Is there an option to embed the sample from a repo? Avoiding copy would be the best solution
As far as I know, it is impossible to directly embed files outside the scope of openvino/docs directory. We can make reference to such files.
There was a problem hiding this comment.
In that case having such commented out code is weird to me, so distilled code looks is better. @andrei-kochin will take a look once more.
Would there be some mechanism ensuring the code snippets are kept up to date?
There was a problem hiding this comment.
In that case having such commented out code is weird to me, so distilled code looks is better. @andrei-kochin will take a look once more.
Would there be some mechanism ensuring the code snippets are kept up to date?
For now, I believe, these snippets will be listed under usual maintenance items to keep them up to date.
There was a problem hiding this comment.
@sgolebiewski-intel I agree to keep only inline comment related to the device. Others are not required and can be removed
docs/articles_en/learn-openvino/llm_inference_guide/genai-guide/genai-use-cases.rst
Outdated
Show resolved
Hide resolved
Creating an article with use case scenarios for using OpenVINO GenAI. This PR addresses the following JIRA ticket: CVS-153319 --------- Co-authored-by: Karol Blaszczak <[email protected]>
Creating an article with use case scenarios for using OpenVINO GenAI. This PR addresses the following
JIRA ticket: CVS-153319