-
Notifications
You must be signed in to change notification settings - Fork 804
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feat: weighted average table metrics #3348
Conversation
There is one scenario we need to account for. When there is 0 tables in ground truth file, and there were some false positives. I think it could be considered as weight=1 what do you think? Now the file will be not counted right? |
good call; that makes sense |
@plutasnyy actually in the code we already filter down to only rows with non-zero "total_tables". If we intend to change that behavior it would be better we do that in a different PR since it changes the existing behavior on tallying tables |
- false negative tables gets a 0 score for table level acc metric and a weight equal to 1 table per page - false negative tables do not contribute to other table metrics since there is no ground truth to evaluate structure or content of those tables
This PR uses (number of actual table) weighted average instead of average without weights for table metrics.
table_level_acc
test
This PR updates the existing test for evaluating table metrics: