Skip to content
This repository has been archived by the owner on Jan 26, 2024. It is now read-only.

Add a 3rd output class, labeled UNKNOWN. #28

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Add a 3rd output class, labeled UNKNOWN. #28

wants to merge 5 commits into from

Conversation

cbaakman
Copy link
Collaborator

@cbaakman cbaakman commented Sep 27, 2022

When calculating MCC, accuracy and ROC AUC values in tensorboard, consider the following 3 class labels:

  • BENIGN: when the CNN outputs a higher number on the left side, than on the right side
  • PATHOGENIC: when the CNN outputs a higher number on the right side, than on the left side
  • UNKNOWN: when the absolute difference between the CNN output numbers is below 0.5

BENIGN and PATHOGENIC both contribute to the MCC, accuracy and ROC values. Outputs of UNKNOWN however, are left out of the equation. They don't contribute to these scores. Furthermore, loss values are still calculated in the same way as before. So this change will not affect CNN training. Only the scores are calculated in a different way. @rgayatri is this what you intended?

@cbaakman cbaakman requested a review from rgayatri September 27, 2022 14:08
@cbaakman cbaakman self-assigned this Sep 27, 2022
@coveralls
Copy link

coveralls commented Sep 27, 2022

Pull Request Test Coverage Report for Build 3149893130

  • 85 of 90 (94.44%) changed or added relevant lines in 5 files are covered.
  • 6 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-0.01%) to 75.75%

Changes Missing Coverage Covered Lines Changed/Added Lines %
deeprank/models/metrics.py 45 47 95.74%
test/tools/test_metrics_tools.py 8 11 72.73%
Files with Coverage Reduction New Missed Lines %
deeprank/models/metrics.py 6 90.18%
Totals Coverage Status
Change from base Build 2971424162: -0.01%
Covered Lines: 3030
Relevant Lines: 4000

💛 - Coveralls

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants