[WIP] add focal_loss and label_smoothing_loss

[rohitgr7]

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

What does this PR do?

Added FocalLoss and LabelSmoothingLoss as discussed on slack.
TODO:

1. make changes if requested
2. update docs
3. Add tests

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[pep8speaks]

Hello @rohitgr7! Thanks for updating this PR.

• In the file pl_bolts/losses/classification.py:

Line 50:121: E501 line too long (125 > 120 characters)
Line 82:121: E501 line too long (128 > 120 characters)
Line 102:1: W293 blank line contains whitespace
Line 143:1: W293 blank line contains whitespace
Line 151:121: E501 line too long (123 > 120 characters)
Line 162:5: E301 expected 1 blank line, found 0
Line 166:1: E302 expected 2 blank lines, found 1
Line 176:121: E501 line too long (125 > 120 characters)

Comment last updated at 2020-07-22 22:42:20 UTC

[codecov]

Codecov Report

Merging #121 into master will decrease coverage by 1.65%.
The diff coverage is 28.70%.

@@            Coverage Diff             @@
##           master     #121      +/-   ##
==========================================
- Coverage   92.25%   90.60%   -1.66%     
==========================================
  Files          79       80       +1     
  Lines        4031     4139     +108     
==========================================
+ Hits         3719     3750      +31     
- Misses        312      389      +77     

Flag	Coverage Δ
#unittests	90.60% <28.70%> (-1.66%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pl_bolts/losses/classification.py	28.70% <28.70%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9ef90b4...369db1a. Read the comment docs.

[rohitgr7]

cc @justusschock

[Borda]

pls add Documentation :]

[justusschock]

@rohitgr7 can you maybe also add a functional interface?

when you look at https://github.com/justusschock/dl-utils/blob/master/dlutils/losses/focal.py you can find a more detailed interface, that also works on binary/logits. Could we maybe adopt something similar here? I'm afraid I haven't tested the implementation there. The reduce and onehot functions should be available in pytorch-lightning.metrics

[rohitgr7]

@justusschock Made the changes locally with the new interface. Any resources from where I can test my implementation. All I can find online is just the implementation of binary-version which I tested and working fine. The problem is with multi-class version one. Can't find anything for that.

[rohitgr7]

Mind the docs as of now. Tested BinaryFocalLoss \ WithLogits and LabelSmoothingLoss. Added weight and ignore_index parameters to match with pytorch loss api. Can't find multi-class version of focal loss to test it. Also with weight parameter pytorch uses weighted mean when reduction='mean', but pl_metrics reduce function takes elementwise_mean which is just the basic mean. Not sure if I should keep weight parameter here.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Borda]

@rohitgr7 how is it going here, is it still WIP? 🐰

[rohitgr7]

totally forgot about this one. will get this done next week.

[Borda]

@rohitgr7 any update here? 👼