Attention maps issue

[421psh]

Hello. I am trying to implement your code and faced weird behavior of Attention Maps (predMaskC and predMaskN). I am using your pretrained model. I also tried torchvision 0.2.0 and torch 0.4.0, but it didn't help. Do you have any suggestions?

[JiaxiongQ]

Is your python version 2.7 ?

[421psh]

Is your python version 2.7 ?

I have tried both python versions (2.7 and 3.6), same result.

My code for your model inference:

img_orig = skimage.io.imread('rgb_kitti.png')
lidar = skimage.io.imread('lidar_kitti.png').astype(np.float32)
sparse = lidar * 1.0 / 256.0
binary_mask = np.where(sparse > 0.0, 1.0, 0.0)
img = img_orig.transpose(2, 0, 1)
left = torch.FloatTensor(img.reshape(1, img.shape[0], img.shape[1], img.shape[2])).cuda()
sparse = torch.FloatTensor(sparse.reshape(1, 1, sparse.shape[0], sparse.shape[1])).cuda()
mask = torch.FloatTensor(binary_mask.reshape(1, 1, binary_mask.shape[0], binary_mask.shape[1])).cuda()

model = s2dN(1)
model = nn.DataParallel(model, device_ids=[0])
model.cuda()
state_dict = torch.load('depth_completion_KITTI.tar')['state_dict']
model.load_state_dict(state_dict)

model.eval()

with torch.no_grad():
    outC, outN, maskC, maskN = model(left, sparse, mask)
    
tempMask = torch.zeros_like(outC)
predC = outC[:,0,:,:]
predN = outN[:,0,:,:]
tempMask[:, 0, :, :] = maskC
tempMask[:, 1, :, :] = maskN
predMask = F.softmax(tempMask)
predMaskC = predMask[:,0,:,:]
predMaskN = predMask[:,1,:,:]
pred1 = predC * predMaskC + predN * predMaskN
pred = torch.squeeze(pred1)
pred = pred.data.cpu().numpy()
pred = np.where(pred <= 0.0, 0.9, pred)
pred_show = pred * 256.0
pred_show = pred_show.astype('uint16')
res_buffer = pred_show.tobytes()
img_pred = Image.new("I",pred_show.T.shape)
img_pred.frombytes(res_buffer,'raw',"I;16")
img_pred.save('img_pred.png')

[421psh]

I also have tried different images from Kitti dataset, and predMaskC is always practically empty. As a result there is no impact from Color Pathway Dense Depth in a final one.

[JiaxiongQ]

The value of pixel in the mask is from 0 to 1, you should multiply 255 for visualization.

[421psh]

The value of pixel in the mask is from 0 to 1, you should multiply 255 for visualization.

The problem is not in visualization, matplotlib allows showing images both in the range of 0 to 255 and 0 to 1. The main issue is that the Mask of Color Pathway is almost all consists of zeros, while the Mask of Normal Pathway consists of ones.

[421psh]

Did anyone face such issue?

[JiaxiongQ]

Sorry, I don't know. The code I test in my machine is actually fine.

…

On Mon, Feb 10, 2020 at 10:45 PM 421psh ***@***.***> wrote: Did anyone face such issue? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#25?email_source=notifications&email_token=AJANJRGL6BV334XK6ECFBG3RCFSAVA5CNFSM4KQ3R3S2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELIYLIQ#issuecomment-584156578>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJANJRBVBTXHD5M6KGCKMD3RCFSAVANCNFSM4KQ3R3SQ> .

[421psh]

@JiaxiongQ Thank you for your responsiveness. Finally, after neat preprocessing on the input data and following all requirements, I actually managed to get right Depth Map. Result is great, but I am wondering are you going to adapt the code to python 3 and up to date versions of torch and torchvision?

[JiaxiongQ]

Great! This is a good suggestion, but we don't have such plan in the near future.

…

On Tue, Feb 11, 2020 at 6:47 PM 421psh ***@***.***> wrote: @JiaxiongQ <https://github.com/JiaxiongQ> Thank you for your responsiveness. Finally, after neat preprocessing on the input data and following all requirements, I actually managed to get right Depth Map. Result is great, but I am wondering are you going to adapt the code to python 3 and up to date versions of torch and torchvision? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#25?email_source=notifications&email_token=AJANJRGIRYIODCPDRQNFJBTRCJ64JA5CNFSM4KQ3R3S2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELL6X2Q#issuecomment-584575978>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJANJRCLUAS5TVRVTBJDO6LRCJ64JANCNFSM4KQ3R3SQ> .

[hello7623]

Hello. I am trying to implement your code and faced weird behavior of Attention Maps (predMaskC and predMaskN). I am using your pretrained model. I also tried torchvision 0.2.0 and torch 0.4.0, but it didn't help. Do you have any suggestions?

why my result is like this

[JiaxiongQ]

The version of python should be 2.7

…

On Thu, May 14, 2020 at 8:06 PM Wen Y.K. ***@***.***> wrote: Hello. I am trying to implement your code and faced weird behavior of Attention Maps (predMaskC and predMaskN). I am using your pretrained model. I also tried torchvision 0.2.0 and torch 0.4.0, but it didn't help. Do you have any suggestions? why my result is like this [image: image] <https://user-images.githubusercontent.com/36818370/81932377-4fc95000-961e-11ea-8d4b-9f436768cf12.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJANJRFJFR2UWGVT3ECYGJLRRPNDHANCNFSM4KQ3R3SQ> .

[hello7623]

The version of python should be 2.7
…

I have used python 2.7 （a long time for config env） but it still like this
oh on