Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/macro recall ignore index multiclass #2737

Conversation

rittik9
Copy link
Contributor

@rittik9 rittik9 commented Sep 11, 2024

What does this PR do?

Fixes #2441

Details
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

Did you have fun?

Yes

Issue:

  • The root of the problem seems to be that the ignore_index information is not being properly propagated to the final averaging step i.e. the _adjust_weights_safe_divide function doesn't know that which class should be ignored.

Solution:

  • To address this issue, I updated the code to ensure that the ignore_index information is preserved throughout the entire process, making sure it is correctly passed through all intermediate steps up to the final averaging stage i.e. _adjust_weights_safe_divide function .
  • Updated the _adjust_weights_safe_divide function to accept an additional ignore_index parameter, which is passed through the _precision_recall_reduce function, called in the compute method of the MulticlassRecall class. This change adjusts the weights in the _adjust_weights_safe_divide function, setting the weight of the ignored class to 0.

📚 Documentation preview 📚: https://torchmetrics--2737.org.readthedocs.build/en/2737/

@rittik9 rittik9 closed this Sep 11, 2024
@rittik9 rittik9 deleted the fix/macro-recall-ignore-index-multiclass branch September 11, 2024 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect result in computing MulticlassRecall macro average when ignore_index is specified
2 participants