Zero contrib for MaxPool2D

[berleon]

Hi Avanti,

for my paper [1], I computed the cosine similarity using the values of self._target_contrib_vars as it includes both positive and negative contributions. However for the MaxPool2D, self._target_contrib_vars was set to zero. Is there a reason for it? Here is the code snippet:

class MaxPool2D(Pool2D):
    def _build_pos_and_neg_contribs(self):
        if (self.verbose):
            print("Heads-up: current implementation assumes maxpool layer "
                  "is followed by a linear transformation (conv/dense layer)")
        #placeholder; not used for linear layer, hence assumption above
        return tf.zeros_like(tensor=self.get_activation_vars(),
                      name="dummy_pos_cont_"+str(self.get_name())),\
               tf.zeros_like(tensor=self.get_activation_vars(),
                      name="dummy_neg_cont_"+str(self.get_name()))

As a fix, I changed it to return pos and neg contributions:

class MaxPool2D(Pool2D):
    ...
    def _build_pos_and_neg_contribs(self):
        inp_pos_contribs, inp_neg_contribs =\
            self._get_input_pos_and_neg_contribs()
        pos_contribs = self._build_activation_vars(inp_pos_contribs)
        neg_contribs = self._build_activation_vars(inp_neg_contribs)
        return pos_contribs, neg_contribs

I hope this does not break anything. What would be the correct behavior? To incorporate the pooling mask?

Cheers,
Leon

[AvantiShri]

Hi Leon,

It would not break the behavior of other layers because, as you might have guessed from the comment, the value of those tensors only matter when the layer is followed by a nonlinearity; therefore, long as the maxpooling layer is followed by a linear transformation, those tensors do not affect the calculation of the multipliers. That’s why I used placeholder values.

Yes, use of a pooling mask would be the ideal way to deal with this - however, as long as the maxpooling layer is followed by a linear layer, you will also get correct attributions if you simply treat the difference-from-reference as the pos_contribs and set the neg_contribs to zero; this is because pos_mxts=neg_mxts if the maxpooling layer were followed by a linear layer, as discussed in Section 3.4 of the DeepLIFT paper. Thus, given that _target_contrib_vars is found as pos_contribs*pos_mxts + neg_contribs*neg_mxts, and pos_mxts=neg_mxts when a layer is followed by a linear layer, I think the following fix should work:

pos_contribs = self._get_diff_from_reference_vars()
neg_contribs = tf.zeros_like(tensor=self.get_activation_vars(),
                             name="dummy_neg_cont_"+str(self.get_name()))
return pos_contribs, neg_contribs

Does that make sense?