1.x branch is much more slower than main branch in webcam_demo.py with Pose Estimation use case.

[aixiaodewugege]

I found that the webcam_demo.py in 1.x branch is much slower, even if they are using same detection and pose model. I found the only difference between them are the draw function. Do you know the reason that lead to this question? Can it be fixed?

main:

1.x:

[Ben-Louis]

Thanks for pointing out this problem! Would you mind giving the details of the config files you used in both versions? We will appreciate your help with this.

[aixiaodewugege]

I just use the default config files both in detection ,pose and animal pose model. You can reproduce results easily. Just let me know if there's anything else I can do to support you. Notify me if there is any progress for I am very curiously about it.

[aixiaodewugege]

Thanks for pointing out this problem! Would you mind giving the details of the config files you used in both versions? We will appreciate your help with this.

To be specific, I use the default config in mmpose/demo/webcam_cfg/pose_estimation.py both in main and 1.x branch.

[Ben-Louis]

You are right, the visualization node is much slower in 1.x since the visualizer uses matplotlib backend to align with mmengine, instead of opencv in the master version. We will look into this problem and fix it. Thank you again for your notification

[aixiaodewugege]

Can I just replace the visualization node to the master version to fix it?

[Ben-Louis]

I think it is okay since they have the same interface

[aixiaodewugege]

I failed with that. Could you please give me a hand?

The master version is based on follow packages:

from mmcv import color_val
from mmpose.core import imshow_bboxes, imshow_keypoints
from mmpose.datasets import DatasetInfo

even if I copy those to 1.x branch , there is error reported.

Traceback (most recent call last):
File "demo/webcam_demo.py", line 8, in
from mmpose.apis.webcam import WebcamExecutor
File "/home/wushuchen/projects/mmyolo/mmpose/mmpose/apis/init.py", line 2, in
from .inference import inference_bottomup, inference_topdown, init_model
File "/home/wushuchen/projects/mmyolo/mmpose/mmpose/apis/inference.py", line 15, in
from mmpose.datasets.datasets.utils import parse_pose_metainfo
File "/home/wushuchen/projects/mmyolo/mmpose/mmpose/datasets/init.py", line 5, in
from .samplers import MultiSourceSampler
File "/home/wushuchen/projects/mmyolo/mmpose/mmpose/datasets/samplers/init.py", line 2, in
from .distributed_sampler import DistributedSampler
File "/home/wushuchen/projects/mmyolo/mmpose/mmpose/datasets/samplers/distributed_sampler.py", line 5, in
from mmpose.core import sync_random_seed
File "/home/wushuchen/projects/mmyolo/mmpose/mmpose/core/init.py", line 3, in
from .camera import * # noqa: F401, F403
File "/home/wushuchen/projects/mmyolo/mmpose/mmpose/core/camera/init.py", line 2, in
from .camera_base import CAMERAS
File "/home/wushuchen/projects/mmyolo/mmpose/mmpose/core/camera/camera_base.py", line 4, in
from mmcv.utils import Registry
ImportError: cannot import name 'Registry' from 'mmcv.utils' (/home/wushuchen/anaconda3/envs/open-mmyolo/lib/python3.7/site-packages/mmcv/utils/init.py)

[Ben-Louis]

Instead of using import, you could try to manually copy the code of functions imshow_bboxes and imshow_keypoints directly into the script. This could potentially avoid version conflicts that may arise.

Another necessary modification is to replace

dataset_info = DatasetInfo(model_cfg.dataset_info)
keypoints = [obj['keypoints'] for obj in group]
imshow_keypoints(
    canvas,
    keypoints,
    skeleton=dataset_info.skeleton,
    kpt_score_thr=0.3,
    pose_kpt_color=dataset_info.pose_kpt_color,
    pose_link_color=dataset_info.pose_link_color,
    radius=self.radius,
    thickness=self.thickness)

with

dataset_info = model_cfg['dataset_meta']
keypoints = [obj['keypoints'] for obj in group]
imshow_keypoints(
    canvas,
    keypoints,
    skeleton=dataset_info['skeleton_links'],
    kpt_score_thr=0.3,
    pose_kpt_color=dataset_info['keypoint_colors'],
    pose_link_color=dataset_info['skeleton_link_colors'],
    radius=self.radius,
    thickness=self.thickness)

[aixiaodewugege]

Thanks for your patient reply. However It is showing another error. I think dataset_meta is not correct

Traceback (most recent call last):
File "/home/wushuchen/anaconda3/envs/open-mmyolo/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/home/wushuchen/projects/mmyolo/mmpose/mmpose/apis/webcam/nodes/node.py", line 396, in run
output_msg = self.process(input_msgs)
File "/home/wushuchen/projects/mmyolo/mmpose/mmpose/apis/webcam/nodes/base_visualizer_node.py", line 47, in process
img = self.draw(input_msg)
File "/home/wushuchen/projects/mmyolo/mmpose/mmpose/apis/webcam/nodes/visualizer_nodes/object_visualizer_node.py", line 357, in draw
canvas = self._draw_keypoint(canvas, input_msg)
File "/home/wushuchen/projects/mmyolo/mmpose/mmpose/apis/webcam/nodes/visualizer_nodes/object_visualizer_node.py", line 336, in _draw_keypoint
dataset_info = model_cfg['dataset_info']
File "/home/wushuchen/anaconda3/envs/open-mmyolo/lib/python3.7/site-packages/mmengine/config/config.py", line 897, in getitem
return self._cfg_dict.getitem(name)
File "/home/wushuchen/anaconda3/envs/open-mmyolo/lib/python3.7/site-packages/mmengine/config/config.py", line 47, in missing
raise KeyError(name)
KeyError: 'dataset_meta'

[Ben-Louis]

Sorry for misleading. I have tried the following code and it works:

dataset_info = objects[0]['dataset_meta']
keypoints = [np.concatenate((obj['keypoints'], obj['keypoint_scores'][:, None]), axis=1) for obj in group]
imshow_keypoints(
    canvas,
    keypoints,
    skeleton=dataset_info['skeleton_links'],
    kpt_score_thr=0.3,
    pose_kpt_color=dataset_info['keypoint_colors'],
    pose_link_color=dataset_info['skeleton_link_colors'],
    radius=self.radius,
    thickness=self.thickness)

[aixiaodewugege]

Thanks a lot! It works on my side too.

I truly appreciate your timely help!