Add SomeOf transform composer

[tuanchien]

SomeOf transform composer

Description

Adds a new transform composer allowing the user to sample "some of" the transforms from a given list of transforms. Allows sampling a fixed number, or variable number of transforms within a configurable range.
Sampling is uniform and without replacement.

The Compose class does almost all of the heavy lifting. This class just replaces the list of transforms each time with a newly sampled list of transforms on each __call__ invocation. The simple sampling entry point will allow for easy extension later in case there's a need to add support for other probability distributions.

This allows users to compose MONAI transforms in a non sequential way without needing to write their own compositor, or wrapping compositors from frameworks like Albumentations.

Maybe of interest to Issue #5312

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• Breaking change (fix or new feature that would cause existing functionality to change).
• New tests added to cover the changes.
• Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
• Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
• In-line docstrings updated.
• Documentation updated, tested make html command in the docs/ folder.

[KumoLiu]

Thanks for your contribution! Just leave some comments in line.

[ericspod]

@atbenmurray Is there anything we need to look at here relating to lazy transforms and/or random state?

[atbenmurray]

I'm taking a look now

[atbenmurray]

In general, implementations cannot dynamically modify transform lists on a call by call basis without breaking in the general case. OneOf and RandomOrder are good examples of how to implement such a function in a way that the transform list remains static.

[atbenmurray]

@tuanchien I realise I have dropped a lot of complexity on you here about all this. I am happy to discuss how you might achieve what you want with this PR using a OneOf/RandomOrder style approach. It should be quite a straightforward thing to do.

[tuanchien]

@atbenmurray thanks for your offer. Will take a stab at it first.

[tuanchien]

From my understanding, the differences between this implementation and the Albumentations version are:

This version:

• Uniformly samples transforms to use.
• Allowed for varying number of sampled transforms per call.

Albumentations:

• Takes the p's (probabilities) from the transforms, and normalises them to 1, and uses this as the sampling probability distribution.

[tuanchien]

I've added a weights parameter with a similar normalisation to OneOf to use for sampling. So now it should have feature parity with Albumentations too.

[atbenmurray]

Looking much better; a few edge cases in testing still to hit, but nearly there

[wyli]

hi @atbenmurray could you please finalize the review here?

[atbenmurray]

Taking a look now

[wyli]

/build

[atbenmurray]

@wyli @Nic-Ma @ericspod Is there an issue here with batches and differing numbers of transforms per-sample?

[wyli]

@wyli @Nic-Ma @ericspod Is there an issue here with batches and differing numbers of transforms per-sample?

good point, I think it should work by looking at the collate function:

MONAI/monai/data/utils.py

Lines 429 to 438 in b87375f

	def collate_meta_tensor(batch):
	"""collate a sequence of meta tensor sequences/dictionaries into
	a single batched metatensor or a dictionary of batched metatensor"""
	if not isinstance(batch, Sequence):
	raise NotImplementedError()
	elem_0 = first(batch)
	if isinstance(elem_0, MetaObj):
	collated = default_collate(batch)
	collated.meta = default_collate([i.meta or TraceKeys.NONE for i in batch])
	collated.applied_operations = [i.applied_operations or TraceKeys.NONE for i in batch]

@tuanchien have you tried this transform in an end-to-end training example, with monai's dataloader and dataset?

and @atbenmurray perhaps we can merge this PR now? I think this SomeOf is a good use case, if the collate function doesn't support it, we should improve the collate function...

[wyli]

/build

[tuanchien]

@wyli sorry for the late response. I have not looked at the collate function. But for what it's worth, the project I used SomeOf on was end-to-end using MONAI transforms, data loaders, and dataset classes.

[atbenmurray]

I'd like to offer our belated thanks for your contribution @tuanchien. I am sure people will find this very useful!