Add perf metrics support for WhisperStaticPipeline #1337
Conversation
ilya-lavrenov
assigned TolyaTalamanov, evgenytalanin-intel, dmatveev and as-suvorov and unassigned evgenytalanin-intel
|
@as-suvorov could you have a look, please? |
|
I think we should add |
|
There is also non segment tokens filtering implemented for stateful pipeline. I think this should also be added to the static pipeline. |
| void infer_with_perf_metrics(ov::InferRequest& request, ov::genai::RawPerfMetrics& raw_metrics); | ||
|
|
||
| template <typename T> | ||
| void filter_by_ranges(std::vector<T>& value, size_t offset, std::vector<std::pair<size_t, size_t>>& ranges); |
There was a problem hiding this comment.
It seems filter_by_ranges used only in whisper_ustils.cpp. If you don't want to use it somewhere else it might make sense do not expose it to ov::genai::utils namespace. You can use unnamed namespace in whisper_utils.cpp (example: https://github.com/openvinotoolkit/openvino.genai/blob/master/src/cpp/src/whisper/whisper.cpp#L23).
|
|
||
| namespace { | ||
|
|
||
| template <typename T> |
There was a problem hiding this comment.
Well, in doesn't have sense to put templated function into anonym namespace, does it?
There was a problem hiding this comment.
Why not?
Example from static pipeline: https://github.com/openvinotoolkit/openvino.genai/blob/master/src/cpp/src/whisper_pipeline_static.cpp#L32
| @@ -0,0 +1,46 @@ | |||
| // Copyright (C) 2023-2024 Intel Corporation | |||
There was a problem hiding this comment.
I believe just 2024
| filter_by_ranges(raw_metrics.m_batch_sizes, offset, ranges); | ||
| } | ||
|
|
||
| } // namespace utils |
There was a problem hiding this comment.
identation is broken
eshiryae
force-pushed
the
b_perf_metrics
branch
from
0f2dd5c to
c24986f
Compare
No description provided.