Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/multiclass recall macro avg ignore index #1

Merged
merged 2 commits into from
Sep 1, 2024

Conversation

rittik9
Copy link
Owner

@rittik9 rittik9 commented Sep 1, 2024

What does this PR do?

Fixes Lightning-AI#2441

Before submitting
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

Did you have fun?

Yes

Issue:

  • The root of the problem seems to be that the ignore_index information is not being properly propagated to the final averaging step i.e. the _adjust_weights_safe_divide function doesn't know that which class should be ignored.

Solution:

  • To address this issue, I updated the code to ensure that the ignore_index information is preserved throughout the entire process, making sure it is correctly passed through all intermediate steps up to the final averaging stage i.e. _adjust_weights_safe_divide function .
  • Updated the _adjust_weights_safe_divide function to accept an additional ignore_index parameter, which is passed through the _precision_recall_reduce function, called in the compute method of the MulticlassRecall class. This change adjusts the weights in the function, setting the weight of the ignored class to 0.

Copy link
Owner Author

@rittik9 rittik9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all good.

@rittik9 rittik9 merged commit acac2ff into master Sep 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect result in computing MulticlassRecall macro average when ignore_index is specified
1 participant