Max pooling and layer dimensions inside attention layer

[valillon]

• There are a couple of max_pooling2d() layers inside the attention layer sn_non_local_block_sim() which reduce the number of local features by 4 as such downsampled_num = location_num // 4. However, no downsampling step is reported in the original paper.

• Also, the first two sn_conv1x1() layers, which stand for Wg and Wf in the paper, have equal sizes C/8 x C, but the third one standing for Wh has C/2 x C shape, while should be also C/8 x C. Similarly the last conv layer.

Is there a reason for such discrepancies?

Related #8

[valillon]

Apparently, as mentioned here, max pooling inside the attention layer is just motivated by design-wise to save computation/memory overhead.