Is the implementation in caffe coming soon?

[lxltaibo]

hi, we want to used the code in caffe, are you still working on caffe implementation? Is it coming soon?

[Fang-Haoshu]

Hi,

I have a script to generate stacked hourglass in caffe, do you need it?

[lxltaibo]

Yes, thank you.

[Fang-Haoshu]

Hi

Here is the script:https://drive.google.com/open?id=0BxORzoJl8N9PS0lHbFJHLWQ2RVJ1blRtYmI4b0ZpUVJMRWRZ.
The caffe version is here: https://github.com/Fang-Haoshu/RMPE

Place it under examples/rmpe/ and run the script from $RMPE_root

Currently the input format is bbox format, I also implement the centerscale version as original stacked hourglass but not tested. You can modify the param in the script.

However, the reason I do not public it now is that, it's hard to train using Caffe, always loss=nan, I only success in training it for 1 time and can not reproduce it. If you find the reason,please let me know.

Hope this may help

Best

[lxltaibo]

Hi, I use you script to generate the training prototxt, and I have some question when training.

1. The prototxt generated by the script is not the same as the Newell paper, e.g. in the papaer, elementwise adding is used (three input) between hourglasses, and in your implementation, the concat (two input) and then the elementwise adding(two input) is applied, why? (By the way, I try to generate the prototxt the same as the paper, but failed when training.)

2. I also find the loss=nan, but just in specific GPU in my PC. In detail, GPU 0, loss is ok , but when using GPU 1 , the loss is nan. The results can be reproduced.

[Fang-Haoshu]

Hi

1. In fact there are two version of Stacked hourglass. My implementation is V1 and later Newell update the second version. The performance of V1 is 89.7mAP on MPII. If possible, can you share the prototxt of the new version? I can have a look to see what's the problem.

2. Yes, that's really weird. I once guessed it's a bug in CuDNN or even GPU device.

[lxltaibo]

Thanks.
The train prototxt with two stacked hourglass modules of V2 is shown below, the main difference is:

1. There is only one residual module after pooling in the hourglass module;
2. The processing between hourglass is different;

train.txt

[Fang-Haoshu]

As far as I can see, there is no problem with the network structure... Maybe you can try to init all the conv with xavier

[lxltaibo]

Thanks for reply.
I change the version of the BN layer, and the loss is no longer nan. Maybe there is something wrong with the BatchNorm layer.

The hourglass prototxt of V2 can also obtain the right key point heatmap.

[Fang-Haoshu]

Oh yes, the official BN layer is not that perfect:)

[anshan-XR-ROB]

Hi, Fang,
I have try your caffe code about SHG. The pose estimation result is poor. The accuracy_heatmap = 0.225325 after 70000 iters on MPII dataset. What should I do to get 89.7mAP on MPII?

[Fang-Haoshu]

Hi Anshan,

Did you try the prototxt attached by lxltaibo in this issue and also change the BN version?

[anshan-XR-ROB]

Thank you for Fang. I will try @lxltaibo 's prototxt.

[anshan-XR-ROB]

Hi, Fang,
I have another problem for torch.
table.insert(out, tmpOut)
I want to know the meaning of this line. I only use tmpOut4 for output and the result is poor. Shoud l concat all tmpOut to output?

[Fang-Haoshu]

@AnshanTJU For the caffe prototxt, that's a 2 stack version. You can try to extend it to the 8 stack version

[Fang-Haoshu]

@AnshanTJU For torch, yes, it's for 8 stacked hourglass

[wutao20171222]

I try @lxltaibo 's prototxt.But I can't run successfully。Tip: The roi area of opencv is out of bounds
why？

[soulslicer]

@lxltaibo

Do you have the weights and solver for your model? I wish to try it but don't want to train from scratch

[sixsixsix666]

Thanks.
The train prototxt with two stacked hourglass modules of V2 is shown below, the main difference is:

1. There is only one residual module after pooling in the hourglass module;
2. The processing between hourglass is different;

train.txt

@lxltaibo
Thanks for sharing your training prototxt file!
I found a "EltwiseAffine" layer followed by "BatchNorm" layer in your prototxt.
Can you tell me what this layer is for? :)