What is the Back Propagation formula when fuse the pri and sec stream?

[FrancisYizhang]

Dear gkioxari

I am sorry to bother you since I want to rewrite R Star CNN on the MatConvNet. I do not know What is the Back Propagation formula when fuse the pri and sec stream at following two points.

First:
layer { name: "sum_scores" type: "Sum" bottom: "cls_score" bottom: "mil_context_cls_score" top: "sum_cls_score" }

Second:
how to fuse the following two branches
layer { name: "context_roi_pool5" type: "ROIPooling" bottom: "conv5_3" bottom: "secondary_rois" top: "context_pool5" roi_pooling_param { pooled_w: 7 pooled_h: 7 spatial_scale: 0.0625 # 1/16 } }

layer { name: "roi_pool5" type: "ROIPooling" bottom: "conv5_3" bottom: "rois" top: "pool5" roi_pooling_param { pooled_w: 7 pooled_h: 7 spatial_scale: 0.0625 # 1/16 } }

Thanks!
Francis

[gkioxari]

Hi Francis,

The gradient of the sum operations are straight forward. If the forward pass is f(a,b) = a+b then df/da = 1 and df/db = 1.

I don't know anything about MatConvNet but the source code for ROIPooling can be found here.

[FrancisYizhang]

@gkioxari
Thanks!

[FrancisYizhang]

Dear gkioxari
I am sorry to say the meaning of my question is different with your answer. I mean how to fuse two errors from GT region and max secondary region at the layer conv5_3 or layer relu5_3 into one error. Does it just average them?

layer { name: "conv5_3" type: "Convolution" bottom: "conv5_2" top: "conv5_3" param { lr_mult: 1 decay_mult: 1 } param { lr_mult: 2 decay_mult: 0 } convolution_param { num_output: 512 pad: 1 kernel_size: 3 } } layer { name: "relu5_3" type: "ReLU" bottom: "conv5_3" top: "conv5_3" }

I am sorry to ask this simple question since I don't know anything about Caffe.
Thanks a lot! @gkioxari
Francis