Unable to run the Code. Seems like some architectural issue. Tensor sizes seem to be mismatching.

[nximish]

Encoder selected: resnet18
Pretrained with the following strategy: imagenet
Model selected: mtbit_resnet18
Optimizer selected: adamw
Scheduler selected: step_lr
No checkpoints founded
Trainable parameters: 13198763, total parameters 13198763
Epoch: 1 - Learning rate: 0.0001
0% 0/22 [00:00<?, ?it/s]torch.Size([15, 32, 100, 100])
torch.Size([1, 32, 64, 64])
0% 0/22 [00:06<?, ?it/s]
Traceback (most recent call last):
File "/content/drive/MyDrive/3DCD/3DCD/train.py", line 203, in
out2d, out3d = net(t1, t2)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "/content/drive/MyDrive/3DCD/3DCD/models/MTBIT.py", line 207, in forward
x1 = self._forward_transformer_decoder(x1, token1)
File "/content/drive/MyDrive/3DCD/3DCD/models/MTBIT.py", line 178, in _forward_transformer_decoder
x = x + self.pos_embedding_decoder
RuntimeError: The size of tensor a (100) must match the size of tensor b (64) at non-singleton dimension 3

[VMarsocci]

Hi, can you try to run it with an image size divisible by 4?

[Varro-pepsi]

Has anyone solved this problem yet?

[rdrachmanto]

I put self.pos_embedding_decoder =nn.Parameter(torch.randn(1, 32, 100, 100)), which resolved the error in question but greeted with another error, which is related to the size of input (which is 400x400) compared to the mask (200x200). I am using the provided 3DCD dataset:

Traceback (most recent call last):
  File "/data/rdr78068/3DCD/train.py", line 212, in <module>
    loss3d = criterion3d(out3d.squeeze(dim=1), mask3d) #, exclude_zeros = exclude_zeros)
  File "/data/rdr78068/3DCD/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/rdr78068/3DCD/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/rdr78068/3DCD/venv/lib/python3.10/site-packages/torch/nn/modules/loss.py", line 608, in forward
    return F.mse_loss(input, target, reduction=self.reduction)
  File "/data/rdr78068/3DCD/venv/lib/python3.10/site-packages/torch/nn/functional.py", line 3791, in mse_loss
    expanded_input, expanded_target = torch.broadcast_tensors(input, target)
  File "/data/rdr78068/3DCD/venv/lib/python3.10/site-packages/torch/functional.py", line 76, in broadcast_tensors
    return _VF.broadcast_tensors(tensors)  # type: ignore[attr-defined]
RuntimeError: The size of tensor a (400) must match the size of tensor b (200) at non-singleton dimension 2

[rdrachmanto]

Update, after getting that error I simplified the augmentations in augmentations.py to be:

# Define the complete transformation pipeline
    train_transform = [
        # resize_transform,
        albu.Resize(200, 200),
        albu.Normalize(mean=m, std=s),
        apt.ToTensorV2(),
    ]
    return albu.Compose(train_transform, additional_targets={'t2': 'image', 'mask3d': 'mask'})

Then I updated the self.pos_embedding_decoder in models/MTBIT.py to be

nn.Parameter(torch.randn(1, 32, 50, 50))