From f291218e2581985eaa82577d6df0a2eb4dc034af Mon Sep 17 00:00:00 2001 From: Xin Li <7219519+xin-li-67@users.noreply.github.com> Date: Wed, 12 Apr 2023 13:57:18 +0800 Subject: [PATCH] Reorg how_to.md and model_analysis.md (#2205) --- docs/en/advanced_guides/model_analysis.md | 101 ++++++++++++++++++- docs/en/how_to.md | 101 ------------------- docs/zh_cn/advanced_guides/model_analysis.md | 99 +++++++++++++++++- docs/zh_cn/how_to.md | 99 ------------------ 4 files changed, 198 insertions(+), 202 deletions(-) diff --git a/docs/en/advanced_guides/model_analysis.md b/docs/en/advanced_guides/model_analysis.md index 4b0528fd74..e10bb634a6 100644 --- a/docs/en/advanced_guides/model_analysis.md +++ b/docs/en/advanced_guides/model_analysis.md @@ -1,3 +1,102 @@ # Model Analysis -Coming soon. +## Get Model Params & FLOPs + +MMPose provides `tools/analysis_tools/get_flops.py` to get model parameters and FLOPs. + +```shell +python tools/analysis_tools/get_flops.py ${CONFIG_FILE} [--shape ${INPUT_SHAPE}] [--cfg-options ${CFG_OPTIONS}] +``` + +Description of all arguments: + +`CONFIG_FILE` : The path of a model config file. + +`--shape`: The input shape to the model. + +`--input-constructor`: If specified as batch, it will generate a batch tensor to calculate FLOPs. + +`--batch-size`：If `--input-constructor` is specified as batch, it will generate a random tensor with shape `(batch_size, 3, **input_shape)` to calculate FLOPs. + +`--cfg-options`: If specified, the key-value pair optional `cfg` will be merged into config file. + +Example: + +```shell +python tools/analysis_tools/get_flops.py configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py +``` + +We will get the following results: + +```text +============================== +Input shape: (1, 3, 256, 192) +Flops: 7.7 GFLOPs +Params: 28.54 M +============================== +``` + +```{note} +This tool is still experimental and we do not guarantee that the number is absolutely correct. Some operators are not counted into FLOPs like GN and custom operators. +``` + +## Log Analysis + +MMPose provides `tools/analysis_tools/analyze_logs.py` to analyze the training log. The log file can be either a json file or a text file. The json file is recommended, because it is more convenient to parse and visualize. + +Currently, the following functions are supported: + +- Plot loss/accuracy curves +- Calculate training time + +### Plot Loss/Accuracy Curves + +The function depends on `seaborn`, please install it first by running `pip install seaborn`. + +![log_curve](https://user-images.githubusercontent.com/87690686/188538215-5d985aaa-59f8-44cf-b6f9-10890d599e9c.png) + +```shell +python tools/analysis_tools/analyze_logs.py plot_curve ${JSON_LOGS} [--keys ${KEYS}] [--title ${TITLE}] [--legend ${LEGEND}] [--backend ${BACKEND}] [--style ${STYLE}] [--out ${OUT_FILE}] +``` + +Examples: + +- Plot loss curve + + ```shell + python tools/analysis_tools/analyze_logs.py plot_curve log.json --keys loss_kpt --legend loss_kpt + ``` + +- Plot accuracy curve and export to PDF file + + ```shell + python tools/analysis_tools/analyze_logs.py plot_curve log.json --keys acc_pose --out results.pdf + ``` + +- Plot multiple log files on the same figure + + ```shell + python tools/analysis_tools/analyze_logs.py plot_curve log1.json log2.json --keys loss_kpt --legend run1 run2 --title loss_kpt --out loss_kpt.png + ``` + +### Calculate Training Time + +```shell +python tools/analysis_tools/analyze_logs.py cal_train_time ${JSON_LOGS} [--include-outliers] +``` + +Examples: + +```shell +python tools/analysis_tools/analyze_logs.py cal_train_time log.json +``` + +The result is as follows: + +```text +-----Analyze train time of hrnet_w32_256x192.json----- +slowest epoch 56, average time is 0.6924 +fastest epoch 1, average time is 0.6502 +time std over epochs is 0.0085 +average iter time: 0.6688 s/iter +``` diff --git a/docs/en/how_to.md b/docs/en/how_to.md index fe97ee6539..a880209806 100644 --- a/docs/en/how_to.md +++ b/docs/en/how_to.md @@ -1,106 +1,5 @@ # How to -## Log Analysis - -MMPose provides `tools/analysis_tools/analyze_logs.py` to analyze the training log. The log file can be either a json file or a text file. The json file is recommended, because it is more convenient to parse and visualize. - -Currently, the following functions are supported: - -- Plot loss/accuracy curves -- Calculate training time - -### Plot Loss/Accuracy Curves - -The function depends on `seaborn`, please install it first by running `pip install seaborn`. - -![log_curve](https://user-images.githubusercontent.com/87690686/188538215-5d985aaa-59f8-44cf-b6f9-10890d599e9c.png) - -```shell -python tools/analysis_tools/analyze_logs.py plot_curve ${JSON_LOGS} [--keys ${KEYS}] [--title ${TITLE}] [--legend ${LEGEND}] [--backend ${BACKEND}] [--style ${STYLE}] [--out ${OUT_FILE}] -``` - -Examples: - -- Plot loss curve - - ```shell - python tools/analysis_tools/analyze_logs.py plot_curve log.json --keys loss_kpt --legend loss_kpt - ``` - -- Plot accuracy curve and export to PDF file - - ```shell - python tools/analysis_tools/analyze_logs.py plot_curve log.json --keys acc_pose --out results.pdf - ``` - -- Plot multiple log files on the same figure - - ```shell - python tools/analysis_tools/analyze_logs.py plot_curve log1.json log2.json --keys loss_kpt --legend run1 run2 --title loss_kpt --out loss_kpt.png - ``` - -### Calculate Training Time - -```shell -python tools/analysis_tools/analyze_logs.py cal_train_time ${JSON_LOGS} [--include-outliers] -``` - -Examples: - -```shell -python tools/analysis_tools/analyze_logs.py cal_train_time log.json -``` - -The result is as follows: - -```text ------Analyze train time of hrnet_w32_256x192.json----- -slowest epoch 56, average time is 0.6924 -fastest epoch 1, average time is 0.6502 -time std over epochs is 0.0085 -average iter time: 0.6688 s/iter -``` - -## Get Model Params & FLOPs - -MMPose provides `tools/analysis_tools/get_flops.py` to get model parameters and FLOPs. - -```shell -python tools/analysis_tools/get_flops.py ${CONFIG_FILE} [--shape ${INPUT_SHAPE}] [--cfg-options ${CFG_OPTIONS}] -``` - -Description of all arguments: - -`CONFIG_FILE` : The path of a model config file. - -`--shape`: The input shape to the model. - -`--input-constructor`: If specified as batch, it will generate a batch tensor to calculate FLOPs. - -`--batch-size`：If `--input-constructor` is specified as batch, it will generate a random tensor with shape `(batch_size, 3, **input_shape)` to calculate FLOPs. - -`--cfg-options`: If specified, the key-value pair optional `cfg` will be merged into config file. - -Example: - -```shell -python tools/analysis_tools/get_flops.py configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py -``` - -We will get the following results: - -```text -============================== -Input shape: (1, 3, 256, 192) -Flops: 7.7 GFLOPs -Params: 28.54 M -============================== -``` - -```{note} -This tool is still experimental and we do not guarantee that the number is absolutely correct. Some operators are not counted into FLOPs like GN and custom operators. -``` - ## Print Entire Config Officially provided config files inherit multiple config files, which can facilitate management and reduce redundant code. But sometimes we want to know what the default parameter values that are not explicitly written in the configuration file are. MMPose provides `tools/analysis_tools/print_config.py` to print the entire configuration information verbatim. diff --git a/docs/zh_cn/advanced_guides/model_analysis.md b/docs/zh_cn/advanced_guides/model_analysis.md index 4b0528fd74..234dc5be85 100644 --- a/docs/zh_cn/advanced_guides/model_analysis.md +++ b/docs/zh_cn/advanced_guides/model_analysis.md @@ -1,3 +1,100 @@ # Model Analysis -Coming soon. +## 统计模型参数量与计算量 + +MMPose 提供了 `tools/analysis_tools/get_flops.py` 来统计模型的参数量与计算量。 + +```shell +python tools/analysis_tools/get_flops.py ${CONFIG_FILE} [--shape ${INPUT_SHAPE}] [--cfg-options ${CFG_OPTIONS}] +``` + +参数说明： + +`CONFIG_FILE` : 模型配置文件的路径。 + +`--shape`: 模型的输入张量形状。 + +`--input-constructor`: 如果指定为 `batch`，将会生成一个 `batch tensor` 来计算 FLOPs。 + +`--batch-size`：如果 `--input-constructor` 指定为 `batch`，将会生成一个随机 `tensor`，形状为 `(batch_size, 3, **input_shape)` 来计算 FLOPs。 + +`--cfg-options`: 如果指定，可选的 `cfg` 的键值对将会被合并到配置文件中。 + +示例： + +```shell +python tools/analysis_tools/get_flops.py configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py +``` + +结果如下： + +```text +============================== +Input shape: (1, 3, 256, 192) +Flops: 7.7 GFLOPs +Params: 28.54 M +============================== +``` + +```{note} +目前该工具仍处于实验阶段，我们不能保证统计结果绝对正确，一些算子（比如 GN 或自定义算子）没有被统计到 FLOPs 中。 +``` + +## 分析训练日志 + +MMPose 提供了 `tools/analysis_tools/analyze_logs.py` 来对训练日志进行简单的分析，包括： + +- 将日志绘制成损失和精度曲线图 +- 统计训练速度 + +### 绘制损失和精度曲线图 + +该功能依赖于 `seaborn`，请先运行 `pip install seaborn` 安装依赖包。 + +![log_curve](https://user-images.githubusercontent.com/87690686/188538215-5d985aaa-59f8-44cf-b6f9-10890d599e9c.png) + +```shell +python tools/analysis_tools/analyze_logs.py plot_curve ${JSON_LOGS} [--keys ${KEYS}] [--title ${TITLE}] [--legend ${LEGEND}] [--backend ${BACKEND}] [--style ${STYLE}] [--out ${OUT_FILE}] +``` + +示例： + +- 绘制损失曲线 + + ```shell + python tools/analysis_tools/analyze_logs.py plot_curve log.json --keys loss_kpt --legend loss_kpt + ``` + +- 绘制精度曲线并导出为 PDF 文件 + + ```shell + python tools/analysis_tools/analyze_logs.py plot_curve log.json --keys acc_pose --out results.pdf + ``` + +- 将多个日志文件绘制在同一张图上 + + ```shell + python tools/analysis_tools/analyze_logs.py plot_curve log1.json log2.json --keys loss_kpt --legend run1 run2 --title loss_kpt --out loss_kpt.png + ``` + +### 统计训练速度 + +```shell +python tools/analysis_tools/analyze_logs.py cal_train_time ${JSON_LOGS} [--include-outliers] +``` + +示例： + +```shell +python tools/analysis_tools/analyze_logs.py cal_train_time log.json +``` + +结果如下： + +```text +-----Analyze train time of hrnet_w32_256x192.json----- +slowest epoch 56, average time is 0.6924 +fastest epoch 1, average time is 0.6502 +time std over epochs is 0.0085 +average iter time: 0.6688 s/iter +``` diff --git a/docs/zh_cn/how_to.md b/docs/zh_cn/how_to.md index a2824f977b..8e0b90ae02 100644 --- a/docs/zh_cn/how_to.md +++ b/docs/zh_cn/how_to.md @@ -1,104 +1,5 @@ # How to -## 分析训练日志 - -MMPose 提供了 `tools/analysis_tools/analyze_logs.py` 来对训练日志进行简单的分析，包括： - -- 将日志绘制成损失和精度曲线图 -- 统计训练速度 - -### 绘制损失和精度曲线图 - -该功能依赖于 `seaborn`，请先运行 `pip install seaborn` 安装依赖包。 - -![log_curve](https://user-images.githubusercontent.com/87690686/188538215-5d985aaa-59f8-44cf-b6f9-10890d599e9c.png) - -```shell -python tools/analysis_tools/analyze_logs.py plot_curve ${JSON_LOGS} [--keys ${KEYS}] [--title ${TITLE}] [--legend ${LEGEND}] [--backend ${BACKEND}] [--style ${STYLE}] [--out ${OUT_FILE}] -``` - -示例： - -- 绘制损失曲线 - - ```shell - python tools/analysis_tools/analyze_logs.py plot_curve log.json --keys loss_kpt --legend loss_kpt - ``` - -- 绘制精度曲线并导出为 PDF 文件 - - ```shell - python tools/analysis_tools/analyze_logs.py plot_curve log.json --keys acc_pose --out results.pdf - ``` - -- 将多个日志文件绘制在同一张图上 - - ```shell - python tools/analysis_tools/analyze_logs.py plot_curve log1.json log2.json --keys loss_kpt --legend run1 run2 --title loss_kpt --out loss_kpt.png - ``` - -### 统计训练速度 - -```shell -python tools/analysis_tools/analyze_logs.py cal_train_time ${JSON_LOGS} [--include-outliers] -``` - -示例： - -```shell -python tools/analysis_tools/analyze_logs.py cal_train_time log.json -``` - -结果如下： - -```text ------Analyze train time of hrnet_w32_256x192.json----- -slowest epoch 56, average time is 0.6924 -fastest epoch 1, average time is 0.6502 -time std over epochs is 0.0085 -average iter time: 0.6688 s/iter -``` - -## 统计模型参数量与计算量 - -MMPose 提供了 `tools/analysis_tools/get_flops.py` 来统计模型的参数量与计算量。 - -```shell -python tools/analysis_tools/get_flops.py ${CONFIG_FILE} [--shape ${INPUT_SHAPE}] [--cfg-options ${CFG_OPTIONS}] -``` - -参数说明： - -`CONFIG_FILE` : 模型配置文件的路径。 - -`--shape`: 模型的输入张量形状。 - -`--input-constructor`: 如果指定为 `batch`，将会生成一个 `batch tensor` 来计算 FLOPs。 - -`--batch-size`：如果 `--input-constructor` 指定为 `batch`，将会生成一个随机 `tensor`，形状为 `(batch_size, 3, **input_shape)` 来计算 FLOPs。 - -`--cfg-options`: 如果指定，可选的 `cfg` 的键值对将会被合并到配置文件中。 - -示例： - -```shell -python tools/analysis_tools/get_flops.py configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py -``` - -结果如下： - -```text -============================== -Input shape: (1, 3, 256, 192) -Flops: 7.7 GFLOPs -Params: 28.54 M -============================== -``` - -```{note} -目前该工具仍处于实验阶段，我们不能保证统计结果绝对正确，一些算子（比如 GN 或自定义算子）没有被统计到 FLOPs 中。 -``` - ## 打印全部配置信息 官方提供的配置文件，有时候继承了多个配置文件，这样做可以方便管理，减少冗余代码。但有时候我们希望知道配置文件中没有写明的默认参数值是什么，MMPose 提供了 `tools/analysis_tools/print_config.py` 来逐字逐句打印全部的配置信息。