MetricsReloaded support #5740
Labels
Feature request
Module: metrics
metric for model quality assessments
WG: Evaluation
For the evaluation working group
Milestone
Comments
wyli
added
Feature request
WG: Evaluation
wyli
added a commit
that referenced
this issue
Jan 12, 2023
Signed-off-by: Mikael Brudfors [[email protected]](mailto:[email protected]) part of #5740 ### Description Makes the binary and categorical metrics from MetricsReloaded available in MONAI via a wrapper module (`monai/metrics/wrapper.py`). This module allows to use the MetricsReloaded metrics as, e.g.: ```py import torch from monai.metrics import MetricsReloadedBinary metric_name = "Cohens Kappa" metric = MetricsReloadedBinary(metric_name=metric_name) # first iteration # shape [batch=1, channel=1, 2, 2] y_pred = torch.tensor([[[[1.0, 0.0], [0.0, 1.0]]]]) y = torch.tensor([[[[1.0, 0.0], [1.0, 1.0]]]]) print(metric(y_pred, y)) # second iteration # shape [batch=1, channel=1, 2, 2] y_pred = torch.tensor([[[[1.0, 0.0], [0.0, 0.0]]]]) y = torch.tensor([[[[1.0, 0.0], [1.0, 1.0]]]]) print(metric(y_pred, y)) # aggregate # shape ([batch=2, channel=1]) print(metric.aggregate(reduction="none")) # tensor([[0.5], [0.2]]) # reset metric.reset() ``` Tests of all metrics are in `tests/test_metrics_reloaded.py`. Note that MetricsReloaded is an optional dependency of MONAI; so in order to use the wrapper, MONAI needs to be installed with: ```sh pip install '.[metricsreloaded]' ``` ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [x] New tests added to cover the changes. - [x] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. Signed-off-by: Mikael Brudfors <[email protected]> Signed-off-by: monai-bot <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Mikael Brudfors <[email protected]> Co-authored-by: Wenqi Li <[email protected]> Co-authored-by: monai-bot <[email protected]>
|
closing this by the other packaging/demo tasks are tracked https://github.com/Project-MONAI/MONAI/milestone/91 with separate tickets. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The MetricsReloaded package under Project-MONAI (https://github.com/Project-MONAI/MetricsReloaded), developed by @csudre, is a recommendation framework for biomedical image analysis validation. As already discussed with @wyli, @Nic-Ma, @csudre and others; being able to use this framework in MONAI would be very valuable. The aim of this issue is to integrate MetricsReloaded support into MONAI.
The text was updated successfully, but these errors were encountered: