[Feature request] Plugin/Module to visualize MONAI Transforms

[diazandr3s]

Is your feature request related to a problem? Please describe.

There is no tool to visualize the effect of MONAI transforms on a custom dataset.

MONAI Label users will highly benefit from a tool/plugin/module that allows them to visualize the effect of the MONAI transforms applied to their dataset.

I've got lots of questions about which transforms are best to do data augmentation or to preprocess their dataset: window scale intensity, random flipping, random rotation, scaling, affine transform, etc.

The idea of this tool is to facilitate the selection of MONAI transforms that researchers can use here to do data augmentation.

Ideally, researchers can use this plugin with individual transform and/or a set of random transformations (Compose).

Describe the solution you'd like
A MONAI tool/plugin/module that allows users to visualize the effects of the MONAI transforms applied to their dataset.

[dzenanz]

You could get the feel of some data augmentation transforms by using TorchIO's Slicer plugin. Maybe MONAI could make a similar thing? The easiest thing would be to add MONAI-specific transforms to the exiting plugin code. If the author of the plugin @fepegar doesn't like that, a fork could be made.

[fepegar]

I think adding MONAI transforms to the TorchIO extension might make things confusing, so forking would possibly make more sense. But I might be wrong!

[arpinerap]

I agree with @diazandr3s. Its really hard to manage so huge amount of possibilities in transforms that Monai offer without a graphic interface. TorchIO´s Slicer plugin is a wonderfull tool but with very little options. Developing a good data augmentation strategy without being able to know what transforms exactly do with your images is really hard. Please, reconsider @diazandr3s request, no doubt I´ll be the first user. Thanks in advance!

[dzenanz]

If no one else plans to work on this, I could take a crack over the next weeks. @wyli could you assign it to me?

[Nic-Ma]

Hi @dzenanz ,

Sure, welcome contribution!

Thanks.

[wyli]

given that a monai Compose object could be created from a yaml/json config, e.g.

MONAI/tests/testing_data/transform_metatensor_cases.yaml

Lines 50 to 70 in 315d2d2

	- _target_: ScaleIntensityRangePercentilesd
	keys: "$@input_keys[0]"
	lower: 4
	upper: 95
	b_min: 1
	b_max: 10
	- _target_: RandScaleIntensityd
	keys: "$@input_keys[0]"
	prob: 1.0
	factors: [5, 10]
	- _target_: RandGaussianNoised
	keys: "$@input_keys[0]"
	prob: 1.0
	mean: 10.0
	std: 2.0
	- _target_: RandCoarseShuffled
	keys: "$@input_keys[0]"
	prob: 1.0
	holes: 2
	spatial_size: [10, 13, 18]
	max_spatial_size: [14, 30, 57]

is it possible/useful to make a generic slicer plugin visualising a set of transforms in a user-specified config? cc @SachidanandAlle any suggestions for @dzenanz to get started?

visualising multiple image patch samples according to their sampling locations is also interesting

• https://github.com/Project-MONAI/tutorials/blob/main/modules/transforms_metatensor.ipynb
• https://docs.monai.io/en/latest/transforms.html#randspatialcropsamplesd

[dzenanz]

I created my fork and I am looking at Slicer extension developing docs, so the timing on this suggestion is excellent.

[dzenanz]

I will probably need these conversion routines:
2a8c8cd#diff-ca2b9fd231511881f60bf8fc5364948993600f30797edd36c976f013d57738b5R357-R361

[wyli]

@SachidanandAlle put together an initial codebase here: https://github.com/Project-MONAI/SlicerMONAIViz

all contributions are welcomed :)

[dzenanz]

This is great! I tried to use it and immediately have some feedback. The images loaded via MonaiViz module have different direction matrix from images drag-dropped into Slicer (RAS vs LPS convention). Those buttons for editing the transform list should have mouse-over hint which explains what each button does. The dictionary flavored transforms should have one argument prefilled: keys, and it should be set to ['image', 'label']. I also have a suggestion about installation docs, but I will do that via a PR.

[dzenanz]

PR Project-MONAI/SlicerMONAIViz#1 created.

[dzenanz]

Adding tooltips was easy, too. PR: Project-MONAI/SlicerMONAIViz#2.

[wyli]

thank you @SachidanandAlle @dzenanz @diazandr3s -- closing this for now, please file follow-up tickets here https://github.com/Project-MONAI/SlicerMONAIViz/issues.

[dzenanz]

PR Project-MONAI/SlicerMONAIViz#5 is the last of my initial suggestions. The other PRs implement the other suggestions I raised in #5711 (comment).