[SPARK-20348] [ML] Support squared hinge loss (L2 loss) for LinearSVC #17645
Conversation
|
Test build #75830 has finished for PR 17645 at commit
|
| * @group param | ||
| */ | ||
| @Since("2.3.0") | ||
| final val lossFunction: Param[String] = new Param(this, "lossFunction", "Specifies the loss " + |
There was a problem hiding this comment.
I'd prefer to move this out to shared params, since it can be used by other algorithms as well. Thanks.
There was a problem hiding this comment.
Sure we can do it.
But I'm thinking maybe we should conduct an integrated refactor about the common optimization parameters some time in the future, either through shared params or other trait or abstract class.
There was a problem hiding this comment.
OK, let leave as is and refactor in the future. One minor issue: What about renaming it to loss? I found the name of corresponding params in sklearn.svm.linearSVC is loss. Thanks.
|
ping @hhbyyh, where are we on this? |
|
Hi @HyukjinKwon I think this is a feature we need, but currently we are still having some discussion about optimizer interface. |
|
I took out this in the list. Though, shouldn't we maybe close this for now and reopen again when it's ready if it takes quite long? It'd be probably better than leaving this open without further updates for long time. |
|
ping @hhbyyh. WDYT? |
|
OK. I'll close it for now and try to merge it with #17862. |
What changes were proposed in this pull request?
While Hinge loss is the standard loss function for linear SVM, Squared hinge loss (a.k.a. L2 loss) is also popular in practice. L2-SVM is differentiable and imposes a bigger (quadratic vs. linear) loss for points which violate the margin. Some introduction can be found from http://mccormickml.com/2015/01/06/what-is-an-l2-svm/
Liblinear and scikit learn both offer squared hinge loss as the default loss function for linear SVM.
How was this patch tested?
strengthen existing unit test and add new unit test for comparison.