Using GPU Gaussian blur at DarkPose unbiased decoding & megvii

[HoBeom]

mmpose/mmpose/core/evaluation/top_down_eval.py

Line 266 in 9c047f6

def _gaussian_blur(heatmaps, kernel=11):

It can be process in gpu using pytorch module
https://discuss.pytorch.org/t/gaussian-kernel-layer/37619

class GaussianLayer(nn.Module):
    def __init__(self):
        super(GaussianLayer, self).__init__()
        self.seq = nn.Sequential(
            nn.ReflectionPad2d(10), 
            nn.Conv2d(3, 3, 21, stride=1, padding=0, bias=None, groups=3)
        )

        self.weights_init()
    def forward(self, x):
        return self.seq(x)

    def weights_init(self):
        n= np.zeros((21,21))
        n[10,10] = 1
        k = scipy.ndimage.gaussian_filter(n,sigma=3)
        for name, f in self.named_parameters():
            f.data.copy_(torch.from_numpy(k))

or this
https://www.programmersought.com/article/17644345494/

class GaussianBlurConv(nn.Module):
    def __init__(self, channels=3):
        super(GaussianBlurConv, self).__init__()
        self.channels = channels
        kernel = [[0.00078633, 0.00655965, 0.01330373, 0.00655965, 0.00078633],
                  [0.00655965, 0.05472157, 0.11098164, 0.05472157, 0.00655965],
                  [0.01330373, 0.11098164, 0.22508352, 0.11098164, 0.01330373],
                  [0.00655965, 0.05472157, 0.11098164, 0.05472157, 0.00655965],
                  [0.00078633, 0.00655965, 0.01330373, 0.00655965, 0.00078633]]
        kernel = torch.FloatTensor(kernel).unsqueeze(0).unsqueeze(0)
        kernel = np.repeat(kernel, self.channels, axis=0)
        self.weight = nn.Parameter(data=kernel, requires_grad=False)
 
    def __call__(self, x):
        x = F.conv2d(x.unsqueeze(0), self.weight, padding=2, groups=self.channels)

[innerlee]

Yeah many ops in cpu can be moved to gpu.
Ideally we should do a profiling and find all bottlenecks. If they are happens to be cpu ops, then we can re-implement them in gpu.

For tools of profile, ref the last comment in #73

[HoBeom]

thanks. I'll go over it using cProfile #73.
But I need some time until next week.

[HoBeom]

before gaussian blur

after using torch gpu

#378
sorry for pull requests (click miss 😄 )

[HoBeom]

It doesn't seem necessary. It is a very small performance improvement, but it requires a lot of modifications.
Thank you for comments. 👍 @innerlee

[ykk648]

@HoBeom Hey, your GaussianBlur gpu version may not proform good enough on your test, but for datasets like cocowholebody, there's 133 keypoints needs GaussianBlur to recover from heatmap, I find your codes helpful, made big model like HRNet48 dark reach batch realtime, thanks a lot !