Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perceptual Loss with mednet 3D multiple channels #7567

Closed
SomeUserName1 opened this issue Mar 22, 2024 · 0 comments · Fixed by #7568
Closed

Perceptual Loss with mednet 3D multiple channels #7567

SomeUserName1 opened this issue Mar 22, 2024 · 0 comments · Fixed by #7568

Comments

@SomeUserName1
Copy link
Contributor

Describe the bug
Perceptual loss with 3D volumes and the medicalnet network types currently works with one channel only, as MedNet3D takes single channel 3D volumes as input. However the code in MedicalNetPerceptualSimilarity suggests that it is intended to work with multiple channels.

To Reproduce
Use PerceptualLoss with spatial_dims=3, is_fake_3d=False, network_type='medicalnet_resnet50_23datasets' and 3D volumes with multiple channels, e.g. with fMRI, phase-cycle bSSFP, DTI, ... as data.

Expected behavior
MedicalNetPerceptualSimilarity passes the input channelwise and averages (the latter is already implemented).

Environment

MONAI environment

================================
Printing MONAI config...

MONAI version: 1.3.0
Numpy version: 1.26.4
Pytorch version: 2.2.1+cu121
MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
MONAI rev id: 865972f
MONAI file: /home//workspace/UNet-bSSFP/unet-venv/lib/python3.11/site-packages/monai/init.py

Optional dependencies:
Pytorch Ignite version: NOT INSTALLED or UNKNOWN VERSION.
ITK version: NOT INSTALLED or UNKNOWN VERSION.
Nibabel version: 5.2.0
scikit-image version: NOT INSTALLED or UNKNOWN VERSION.
scipy version: 1.12.0
Pillow version: 10.2.0
Tensorboard version: 2.16.2
gdown version: 4.7.3
TorchVision version: 0.17.1+cu121
tqdm version: 4.66.2
lmdb version: NOT INSTALLED or UNKNOWN VERSION.
psutil version: 5.9.8
pandas version: 2.0.3
einops version: NOT INSTALLED or UNKNOWN VERSION.
transformers version: NOT INSTALLED or UNKNOWN VERSION.
mlflow version: NOT INSTALLED or UNKNOWN VERSION.
pynrrd version: NOT INSTALLED or UNKNOWN VERSION.
clearml version: NOT INSTALLED or UNKNOWN VERSION.

For details about installing the optional dependencies, please visit:
https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies

================================
Printing system config...

System: Linux
Linux version: Arch Linux
Platform: Linux-6.7.4-zen1-1-zen-x86_64-with-glibc2.39
Processor:
Machine: x86_64
Python version: 3.11.7
Process name: python
Command: ['python', '-c', 'import monai; monai.config.print_debug_info()']
Open files: []
Num physical CPUs: 24
Num logical CPUs: 32
Num usable CPUs: 32
CPU usage (%): [93.7, 62.3, 79.0, 40.4, 61.9, 78.7, 56.0, 98.4, 96.5, 19.7, 23.0, 97.6, 96.4, 99.2, 99.6, 30.3, 99.6, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0]
CPU freq. (MHz): 3461
Load avg. in last 1, 5, 15 mins (%): [87.6, 86.7, 82.9]
Disk usage (%): 44.8
Avg. sensor temp. (Celsius): UNKNOWN for given OS
Total physical memory (GB): 62.6
Available memory (GB): 25.2
Used memory (GB): 32.0

================================
Printing GPU config...

Num GPUs: 1
Has CUDA: True
CUDA version: 12.1
cuDNN enabled: True
NVIDIA_TF32_OVERRIDE: None
TORCH_ALLOW_TF32_CUBLAS_OVERRIDE: None
cuDNN version: 8902
Current device: 0
Library compiled for CUDA architectures: ['sm_50', 'sm_60', 'sm_70', 'sm_75', 'sm_80', 'sm_86', 'sm_90']
GPU 0 Name: NVIDIA GeForce RTX 4090 Laptop GPU
GPU 0 Is integrated: False
GPU 0 Is multi GPU board: False
GPU 0 Multi processor count: 76
GPU 0 Total memory (GB): 15.7
GPU 0 CUDA capability (maj.min): 8.9

Additional context
Add any other context about the problem here.

KumoLiu added a commit that referenced this issue Apr 19, 2024
Fixes #7567 .

### Description
MedicalNetPerceptualSimilarity: Add multi-channel support for 3Dvolumes.
The current version of the code in the dev branch already largely
supports that besides the following:
medicalnet_* require inputs to have a single channel. 
This PR passes the multi-channel volume channel-wise to the networks and
concatenates the resulting feature vectors.
The existing code takes care of averaging over channels and spatially.

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [x] New tests added to cover the changes.
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [ ] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [ ] In-line docstrings updated.
- [ ] Documentation updated, tested `make html` command in the `docs/`
folder.

---------

Signed-off-by: Fabian Klopfer <[email protected]>
Co-authored-by: YunLiu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant