Add benchmark.py #35
Add benchmark.py #35
Conversation
|
|
docs/getting_started.md
Outdated
| @@ -306,6 +306,12 @@ CUDA_VISIBLE_DEVICES=0,1,2,3 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} con | |||
| CUDA_VISIBLE_DEVICES=4,5,6,7 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config2.py ${WORK_DIR} 4 | |||
| ``` | |||
|
|
|||
| ## Benchmark | |||
| You can get average training batch for a second, we only care about the model training, not including the IO time and pre-processing time. | |||
There was a problem hiding this comment.
what does this mean: "get average training batch for a second"?
There was a problem hiding this comment.
no, our unit is xxx seconds per iter
|
need to sign the cla @xiaohangcd |
tools/benchmark.py
Outdated
| @@ -0,0 +1,74 @@ | |||
| import argparse | |||
There was a problem hiding this comment.
rename the file to benchmark_inference.py
There was a problem hiding this comment.
since this script is for inference, let's change the unit back to items/s.
docs/getting_started.md
Outdated
| @@ -306,6 +306,12 @@ CUDA_VISIBLE_DEVICES=0,1,2,3 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} con | |||
| CUDA_VISIBLE_DEVICES=4,5,6,7 ./tools/slurm_train.sh ${PARTITION} ${JOB_NAME} config2.py ${WORK_DIR} 4 | |||
| ``` | |||
|
|
|||
| ## Benchmark | |||
| You can get average training time for an iteration, we only care about the model training, not including the IO time and pre-processing time. | |||
There was a problem hiding this comment.
## Benchmark Inference Speed
You can get average inference speed using the following script. Not that it does not includes the IO time and pre-processing time.
|
as discussed in meeting, this script is not for measuring training speed. |
|
Please check linting. |
docs/getting_started.md
Outdated
| ## Benchmark | ||
| You can get average inference speed using the following script. Not that it does not includes the IO time and pre-processing time. | ||
| ```shell | ||
| python tools/benchmark.py ${MMPOSE_CONFIG_FILE} |
|
@xiaohangcd could you run this script using the settings in #40 ? namely, the |
|
the speed test report does not block merging |
No description provided.