[SPARK-1327] GLM needs to check addIntercept for intercept and weights #236
Conversation
| } else { | ||
| initialWeights | ||
| } | ||
|
|
||
| val weights = optimizer.optimize(data, initialWeightsWithIntercept) | ||
| val intercept = weights(0) |
There was a problem hiding this comment.
Should check addIntercept here.
|
Merged build triggered. |
|
Merged build started. |
|
Merged build finished. |
|
One or more automated tests failed |
mark createModel protected mark predictPoint protected
|
Merged build triggered. |
|
Merged build started. |
|
Merged build finished. |
|
All automated tests passed. |
|
@tdas This PR only includes bug fix without breaking changes. Could you merge it into master if it looks good to you? I will send the combined commit to branch-0.9 after. |
|
Merged build triggered. |
|
Merged build started. |
|
Merged build finished. |
|
One or more automated tests failed |
|
Jenkins, retest this please. |
|
Merged build triggered. |
|
Merged build started. |
|
Merged build finished. |
|
All automated tests passed. |
| @@ -95,7 +95,7 @@ abstract class GeneralizedLinearAlgorithm[M <: GeneralizedLinearModel] | |||
| /** | |||
| * Create a model given the weights and intercept | |||
| */ | |||
| protected def createModel(weights: Array[Double], intercept: Double): M | |||
| def createModel(weights: Array[Double], intercept: Double): M | |||
There was a problem hiding this comment.
This should be protected as per our conversation.
|
Jenkins, test this |
|
Merged build triggered. |
|
Merged build started. |
|
Merged build finished. |
|
All automated tests passed. |
GLM needs to check addIntercept for intercept and weights. The current implementation always uses the first weight as intercept. Added a test for training without adding intercept. JIRA: https://spark-project.atlassian.net/browse/SPARK-1327 Author: Xiangrui Meng <[email protected]> Closes #236 from mengxr/glm and squashes the following commits: bcac1ac [Xiangrui Meng] add two tests to ensure {Lasso, Ridge}.setIntercept will throw an exceptions a104072 [Xiangrui Meng] remove protected to be compatible with 0.9 0e57aa4 [Xiangrui Meng] update Lasso and RidgeRegression to parse the weights correctly from GLM mark createModel protected mark predictPoint protected d7f629f [Xiangrui Meng] fix a bug in GLM when intercept is not used (cherry picked from commit d679843) Signed-off-by: Tathagata Das <[email protected]>
Adding disclaimer for shuffle file consolidation (cherry picked from commit 1b38f5f) Signed-off-by: Patrick Wendell <[email protected]>
GLM needs to check addIntercept for intercept and weights. The current implementation always uses the first weight as intercept. Added a test for training without adding intercept. JIRA: https://spark-project.atlassian.net/browse/SPARK-1327 Author: Xiangrui Meng <[email protected]> Closes apache#236 from mengxr/glm and squashes the following commits: bcac1ac [Xiangrui Meng] add two tests to ensure {Lasso, Ridge}.setIntercept will throw an exceptions a104072 [Xiangrui Meng] remove protected to be compatible with 0.9 0e57aa4 [Xiangrui Meng] update Lasso and RidgeRegression to parse the weights correctly from GLM mark createModel protected mark predictPoint protected d7f629f [Xiangrui Meng] fix a bug in GLM when intercept is not used
CodegenSupport.variablePrefix mentions RDDScansExec, so it would seem as it was expecting it to implement CodegenSupport, but it actually doesn't. It could potentially be considered to do some code sharing with RowDataSourceScanExec, which has a very similar implementation of codegen. Author: Juliusz Sompolski <[email protected]> Closes apache#236 from juliuszsompolski/SC-5913.
GLM needs to check addIntercept for intercept and weights. The current implementation always uses the first weight as intercept. Added a test for training without adding intercept.
JIRA: https://spark-project.atlassian.net/browse/SPARK-1327