Partially revert #519 due to performance regression & other issues

[kohr-h]

Follow-up to #519, fixing the case of scalar mean and std in normalize.

[kohr-h]

The test failure is due to this not working:

>>> torch.Tensor(torch.tensor(0.5))
RuntimeError: slice() cannot be applied to a 0-dim tensor.

I can fix the code here by calling float() on the values not calling Tensor on tensors, but I think this is a pytorch bug.

[fmassa]

I think this is getting quite messy.

Did you benchmark to see if performing a single sub followed by a div is faster than the 3 sub / div that we had before?
I think that in the other codepath we had vectorization that was enabled, and I'm not sure that's the case here.

[vfdev-5]

mean (sequence): Sequence of means for each channel.

According to the doc I have an impression that mean (same for std) is a sequence of scalars and not torch.tensors, e.g.

mean = [0.5, 0.5, 0.5]

or

mean = [0.5, ]

so in this case the previous code (ignoring the benchmark) works correctly

torch.Tensor(mean).view((n_channels, 1, 1))

[kohr-h]

Did you benchmark to see if performing a single sub followed by a div is faster than the 3 sub / div that we had before?
I think that in the other codepath we had vectorization that was enabled, and I'm not sure that's the case here.

I should have done that in the first place. The old version was actually faster 😒

According to the doc I have an impression that mean (same for std) is a sequence of scalars and not torch.tensors, e.g.

Correct if you see torch.tensor(0.5) not as a scalar, but you could also argue that anything on which you can run float() is a scalar (or complex() if necessary).