Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BENCHMARK_APP] Show input command in output #13402

Merged
merged 12 commits into from
Oct 26, 2022
Merged

[BENCHMARK_APP] Show input command in output #13402

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
00bcbc5
[SAMPLES/CPP] Show input command in output
PiotrKrzem Oct 10, 2022
7346166
Update main.py
PiotrKrzem Oct 11, 2022
c09adea
Merge branch 'master' into feature/benchmark_command_in_output
PiotrKrzem Oct 11, 2022
b72a324
Merge branch 'master' into feature/benchmark_command_in_output
PiotrKrzem Oct 12, 2022
b7b9e49
Merge branch 'master' into feature/benchmark_command_in_output
PiotrKrzem Oct 12, 2022
05f35cb
Merge branch 'master' into feature/benchmark_command_in_output
PiotrKrzem Oct 18, 2022
af2ca6b
[CPP] Add Windows-specific abspath method
PiotrKrzem Oct 19, 2022
def219d
[CPP] Formatting errors
PiotrKrzem Oct 19, 2022
6772e7e
[C++] Fix naming conventions, fix incorrect buffer type to store outp…
PiotrKrzem Oct 20, 2022
5b6fd41
[C++] Formatting errors - invisible spaces remove
PiotrKrzem Oct 20, 2022
89ce3b1
[C++] Add comments, fix name of the command_string variable
PiotrKrzem Oct 20, 2022
c61ef10
[C++] Remove redefinition of symbol
PiotrKrzem Oct 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions samples/cpp/benchmark_app/main.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,42 @@
#include "remote_tensors_filling.hpp"
#include "statistics_report.hpp"
#include "utils.hpp"

#if defined(_WIN32) || defined(WIN32)
# include <windows.h>
#endif
// clang-format on

static const size_t progressBarDefaultTotalCount = 1000;

std::string get_console_command(int argc, char* argv[]) {
std::stringstream args_command;

#if defined(_WIN32) || defined(WIN32)
std::stringstream args_command;
std::string relative_path(argv[0]);
std::vector<char> buffer;

uint32_t len = 1024;
do {
buffer.resize(len);
len = GetFullPathNameA(relative_path.data(), len, buffer.data(), nullptr);
} while (len > buffer.size());

std::string full_path(buffer.begin(), buffer.end());
args_command << full_path;
#else
args_command << realpath(argv[0], nullptr);
#endif
args_command << " ";

for (int i = 1; i < argc; i++) {
args_command << argv[i] << " ";
}

return args_command.str();
}

bool parse_and_check_command_line(int argc, char* argv[]) {
// ---------------------------Parsing and validating input
// arguments--------------------------------------
Expand Down Expand Up @@ -161,12 +193,16 @@ int main(int argc, char* argv[]) {

// ----------------- 1. Parsing and validating input arguments
// -------------------------------------------------
auto args_string = get_console_command(argc, argv);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is a reason for doing it inside tool?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is easier to verify which benchmark_app is being used, C++ or Python, and from which directory. It can happen for example when the PYTHONPATH/PATH env variable is modified, or after the output alignment PR is merged, when both benchmarks' output is indistinguishable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is easier to verify which benchmark_app is being used, C++ or Python, and from which directory. It can happen for example when the PYTHONPATH/PATH env variable is modified, or after the output alignment PR is merged, when both benchmarks' output is indistinguishable.

Validation related stuff should not be included into the tool. Why this PR was merged? cc @p-durandin

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As pointed out by @helena-intel in the ticket, the idea of this change is to make the usage of the tool and sharing output logs easier for the customers and our coworkers alike, I think this is a valid addition to the tool

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PiotrKrzem I still don't understand why it is problem to share logs with initial command line. If user have ran tool, he knows what he typed in command line.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moreover for some automation scenarios we increase final log size.

t-jankowski marked this conversation as resolved.
Show resolved Hide resolved

next_step();

if (!parse_and_check_command_line(argc, argv)) {
return 0;
}

slog::info << "Input command: " << args_string << slog::endl;
t-jankowski marked this conversation as resolved.
Show resolved Hide resolved

bool isNetworkCompiled = fileExt(FLAGS_m) == "blob";
if (isNetworkCompiled) {
slog::info << "Network is compiled" << slog::endl;
Expand Down
6 changes: 6 additions & 0 deletions tools/benchmark_tool/openvino/tools/benchmark/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,15 @@ def main():
statistics = None
try:
# ------------------------------ 1. Parsing and validating input arguments ------------------------------
args_string = f"{os.path.realpath(sys.argv[0])} "
for i in range(1,len(sys.argv)):
args_string += f"{sys.argv[i]} "

next_step()
args, is_network_compiled = parse_and_check_command_line()

logger.info(f"Input command: {args_string}")

if args.number_streams is None:
logger.warning(" -nstreams default value is determined automatically for a device. "
"Although the automatic selection usually provides a reasonable performance, "
Expand Down