Incorrect result in computing MulticlassRecall macro average when ignore_index is specified

[JeroenMandersloot]

🐛 Bug

Specifying the ignore_index argument in MulticlassRecall leads to incorrect results when computing the macro average.

To Reproduce

import torch
from torchmetrics.classification.precision_recall import MulticlassRecall

metric = MulticlassRecall(num_classes=2, ignore_index=0, average="macro")
y_true = torch.tensor([0, 0, 1, 1])
y_pred = torch.tensor([
    [0.9, 0.1],  # Correct
    [0.9, 0.1],  # Correct
    [0.9, 0.1],  # Incorrect
    [0.1, 0.9],  # Correct
])
metric.update(y_pred, y_true)
print(metric.compute())  # Prints 0.25, but I would expect the result to be 0.5

Expected behavior

In the toy example above there are two classes (0 and 1) and two instances of each class. I want to ignore class 0 altogether (hence ignore_index=0), so I would expect the first two predictions to be ignored entirely (since they relate to class 0 instances). Of the remaining two predictions only one is correct, so I would expect the recall score to be 0.5.

However, the actual result is 0.25. It looks as if the recall score for the ignore_index class is simply set to 0 before computing the average, causing the average to be computed as (0 + 0.5) / 2 instead of 0.5 / 1.

Environment

• TorchMetrics version: 1.3.1 (installed via pip)
• Python version: 3.10
• PyTorch version: 2.2.1
• OS: Ubuntu 22.04

[github-actions]

Hi! thanks for your contribution!, great first issue!

[ding3820]

This issue has been there for a while. After v0.9.3, average="macro" does not work with ignore_index.

[Borda]

This issue has been there for a while. After v0.9.3, average="macro" does not work with ignore_index.

thank you for the adjustment, would yo be interested in sending a PR?

[rittik9]

Pls assign it to me @Borda

[Borda]

Pls assign it to me @Borda

cc: @SkafteNicki