plot_receptive_field: all questions answered

[lostanlen]

Some questions came up after the review of #49 (which closed #48):

• should the theoretical bound be T * (2**J) or half of that? (i think it's half)
• should the magnitude of the gradient be squared? (maybe not ...)
• should we aim for a ~ 1/T power law? or ~ 1/sqrt(T)? or a constant? this has implications for the way we define the Conv1D operator in between DTCWT Direct and Inverse

Let's discuss and then make the appropriate edits to the script

[xir4n]

Some questions came up after the review of #49 (which closed #48):

• should the theoretical bound be T * (2**J) or half of that? (i think it's half)

Sorry, I even didn’t notice it… Did you mean that the total theoretical receptive field should have a size of T * (2** J), but since only half of the input signal is plotted, the theoretical bound should be changed to T * (2** J)/2?

• should the magnitude of the gradient be squared? (maybe not ...)

Probably I’m wrong: I think the concept of the effective receptive field is based on the forward/backward propagation process. If so, should we also consider the effects of the following operators? (For example, for a single layer, the activation function, and for the whole model, the loss function…?)

• should we aim for a ~ 1/T power law? or ~ 1/sqrt(T)? or a constant? this has implications for the way we define the Conv1D operator in between DTCWT Direct and Inverse

Thanks! I think I further understand your intention by adding the power law curve!

Let's discuss and then make the appropriate edits to the script

Finally, maybe it’s not important: should we also operate a Conv1d on x_phi?

[xir4n]

Probably I’m wrong: I think the concept of the effective receptive field is based on the forward/backward propagation process. If so, should we also consider the effects of the following operators? (For example, for a single layer, the activation function, and for the whole model, the loss function…?)

Oh... never mind this... I'll work on MuReNNDirect.to_conv1d and also on its receptive fields' visualization!

[lostanlen]

Sorry, I even didn’t notice it… Did you mean that the total theoretical receptive field should have a size of T * (2** J), but since only half of the input signal is plotted, the theoretical bound should be changed to T * (2** J)/2?

Yes, exactly what I meant. This is because we visualize the receptive field in logscale on the time axis.

Probably I’m wrong: I think the concept of the effective receptive field is based on the forward/backward propagation process. If so, should we also consider the effects of the following operators? (For example, for a single layer, the activation function, and for the whole model, the loss function…?)

I tried to keep things simple (linear and single-layer) in my script but generalizing to multiple layers is also interesting!

Finally, maybe it’s not important: should we also operate a Conv1d on x_phi?

yes, i would welcome that.

[lostanlen]

Now that we have MuReNNDirect.to_conv1d() #56 it would be good to revisit these questions