Skip to content

Commit

Permalink
[SPARK-7916] [MLLIB] MLlib Python doc parity check for classification…
Browse files Browse the repository at this point in the history 
… and regression

Check then make the MLlib Python classification and regression doc to be as complete as the Scala doc.

Author: Yanbo Liang <[email protected]>

Closes #6460 from yanboliang/spark-7916 and squashes the following commits:

f8deda4 [Yanbo Liang] trigger jenkins
6dc4d99 [Yanbo Liang] address comments
ce2a43e [Yanbo Liang] truncate too long line and remove extra sparse
3eaf6ad [Yanbo Liang] MLlib Python doc parity check for classification and regression
  • Loading branch information
@yanboliang @jkbradley
yanboliang authored and jkbradley committed Jun 16, 2015
1 parent cebf241 commit ca99875
Show file tree
Hide file tree
Showing 3 changed files with 248 additions and 108 deletions.
2 changes: 1 addition & 1 deletion mllib/src/main/scala/org/apache/spark/mllib/regression/RidgeRegression.scala
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ object RidgeRegressionModel extends Loader[RidgeRegressionModel] {

/**
* Train a regression model with L2-regularization using Stochastic Gradient Descent.
* This solves the l1-regularized least squares regression formulation
* This solves the l2-regularized least squares regression formulation
* f(weights) = 1/2n ||A weights-y||^2^ + regParam/2 ||weights||^2^
* Here the data matrix has n rows, and the input RDD holds the set of rows of A, each with
* its corresponding right hand side label y.
Expand Down
187 changes: 113 additions & 74 deletions python/pyspark/mllib/classification.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@

class LinearClassificationModel(LinearModel):
"""
A private abstract class representing a multiclass classification model.
The categories are represented by int values: 0, 1, 2, etc.
A private abstract class representing a multiclass classification
model. The categories are represented by int values: 0, 1, 2, etc.
"""
def __init__(self, weights, intercept):
super(LinearClassificationModel, self).__init__(weights, intercept)
Expand All @@ -44,10 +44,11 @@ def setThreshold(self, value):
"""
.. note:: Experimental
Sets the threshold that separates positive predictions from negative
predictions. An example with prediction score greater than or equal
to this threshold is identified as an positive, and negative otherwise.
It is used for binary classification only.
Sets the threshold that separates positive predictions from
negative predictions. An example with prediction score greater
than or equal to this threshold is identified as an positive,
and negative otherwise. It is used for binary classification
only.
"""
self._threshold = value

Expand All @@ -56,31 +57,45 @@ def threshold(self):
"""
.. note:: Experimental
Returns the threshold (if any) used for converting raw prediction scores
into 0/1 predictions. It is used for binary classification only.
Returns the threshold (if any) used for converting raw
prediction scores into 0/1 predictions. It is used for
binary classification only.
"""
return self._threshold

def clearThreshold(self):
"""
.. note:: Experimental
Clears the threshold so that `predict` will output raw prediction scores.
It is used for binary classification only.
Clears the threshold so that `predict` will output raw
prediction scores. It is used for binary classification only.
"""
self._threshold = None

def predict(self, test):
"""
Predict values for a single data point or an RDD of points using
the model trained.
Predict values for a single data point or an RDD of points
using the model trained.
"""
raise NotImplementedError


class LogisticRegressionModel(LinearClassificationModel):

"""A linear binary classification model derived from logistic regression.
"""
Classification model trained using Multinomial/Binary Logistic
Regression.
:param weights: Weights computed for every feature.
:param intercept: Intercept computed for this model. (Only used
in Binary Logistic Regression. In Multinomial Logistic
Regression, the intercepts will not be a single value,
so the intercepts will be part of the weights.)
:param numFeatures: the dimension of the features.
:param numClasses: the number of possible outcomes for k classes
classification problem in Multinomial Logistic Regression.
By default, it is binary logistic regression so numClasses
will be set to 2.
>>> data = [
... LabeledPoint(0.0, [0.0, 1.0]),
Expand Down Expand Up @@ -161,8 +176,8 @@ def numClasses(self):

def predict(self, x):
"""
Predict values for a single data point or an RDD of points using
the model trained.
Predict values for a single data point or an RDD of points
using the model trained.
"""
if isinstance(x, RDD):
return x.map(lambda v: self.predict(v))
Expand Down Expand Up @@ -225,16 +240,19 @@ def train(cls, data, iterations=100, step=1.0, miniBatchFraction=1.0,
"""
Train a logistic regression model on the given data.
:param data: The training data, an RDD of LabeledPoint.
:param iterations: The number of iterations (default: 100).
:param data: The training data, an RDD of
LabeledPoint.
:param iterations: The number of iterations
(default: 100).
:param step: The step parameter used in SGD
(default: 1.0).
:param miniBatchFraction: Fraction of data to be used for each SGD
iteration.
:param miniBatchFraction: Fraction of data to be used for each
SGD iteration (default: 1.0).
:param initialWeights: The initial weights (default: None).
:param regParam: The regularizer parameter (default: 0.01).
:param regType: The type of regularizer used for training
our model.
:param regParam: The regularizer parameter
(default: 0.01).
:param regType: The type of regularizer used for
training our model.
:Allowed values:
- "l1" for using L1 regularization
Expand All @@ -243,13 +261,14 @@ def train(cls, data, iterations=100, step=1.0, miniBatchFraction=1.0,
(default: "l2")
:param intercept: Boolean parameter which indicates the use
or not of the augmented representation for
training data (i.e. whether bias features
are activated or not).
:param validateData: Boolean parameter which indicates if the
algorithm should validate data before training.
(default: True)
:param intercept: Boolean parameter which indicates the
use or not of the augmented representation
for training data (i.e. whether bias
features are activated or not,
default: False).
:param validateData: Boolean parameter which indicates if
the algorithm should validate data
before training. (default: True)
"""
def train(rdd, i):
return callMLlibFunc("trainLogisticRegressionModelWithSGD", rdd, int(iterations),
Expand All @@ -267,12 +286,15 @@ def train(cls, data, iterations=100, initialWeights=None, regParam=0.01, regType
"""
Train a logistic regression model on the given data.
:param data: The training data, an RDD of LabeledPoint.
:param iterations: The number of iterations (default: 100).
:param data: The training data, an RDD of
LabeledPoint.
:param iterations: The number of iterations
(default: 100).
:param initialWeights: The initial weights (default: None).
:param regParam: The regularizer parameter (default: 0.01).
:param regType: The type of regularizer used for training
our model.
:param regParam: The regularizer parameter
(default: 0.01).
:param regType: The type of regularizer used for
training our model.
:Allowed values:
- "l1" for using L1 regularization
Expand All @@ -281,19 +303,21 @@ def train(cls, data, iterations=100, initialWeights=None, regParam=0.01, regType
(default: "l2")
:param intercept: Boolean parameter which indicates the use
or not of the augmented representation for
training data (i.e. whether bias features
are activated or not).
:param corrections: The number of corrections used in the LBFGS
update (default: 10).
:param tolerance: The convergence tolerance of iterations for
L-BFGS (default: 1e-4).
:param intercept: Boolean parameter which indicates the
use or not of the augmented representation
for training data (i.e. whether bias
features are activated or not,
default: False).
:param corrections: The number of corrections used in the
LBFGS update (default: 10).
:param tolerance: The convergence tolerance of iterations
for L-BFGS (default: 1e-4).
:param validateData: Boolean parameter which indicates if the
algorithm should validate data before training.
(default: True)
:param numClasses: The number of classes (i.e., outcomes) a label can take
in Multinomial Logistic Regression (default: 2).
algorithm should validate data before
training. (default: True)
:param numClasses: The number of classes (i.e., outcomes) a
label can take in Multinomial Logistic
Regression (default: 2).
>>> data = [
... LabeledPoint(0.0, [0.0, 1.0]),
Expand Down Expand Up @@ -323,7 +347,11 @@ def train(rdd, i):

class SVMModel(LinearClassificationModel):

"""A support vector machine.
"""
Model for Support Vector Machines (SVMs).
:param weights: Weights computed for every feature.
:param intercept: Intercept computed for this model.
>>> data = [
... LabeledPoint(0.0, [0.0]),
Expand Down Expand Up @@ -370,8 +398,8 @@ def __init__(self, weights, intercept):

def predict(self, x):
"""
Predict values for a single data point or an RDD of points using
the model trained.
Predict values for a single data point or an RDD of points
using the model trained.
"""
if isinstance(x, RDD):
return x.map(lambda v: self.predict(v))
Expand Down Expand Up @@ -409,16 +437,19 @@ def train(cls, data, iterations=100, step=1.0, regParam=0.01,
"""
Train a support vector machine on the given data.
:param data: The training data, an RDD of LabeledPoint.
:param iterations: The number of iterations (default: 100).
:param data: The training data, an RDD of
LabeledPoint.
:param iterations: The number of iterations
(default: 100).
:param step: The step parameter used in SGD
(default: 1.0).
:param regParam: The regularizer parameter (default: 0.01).
:param miniBatchFraction: Fraction of data to be used for each SGD
iteration.
:param regParam: The regularizer parameter
(default: 0.01).
:param miniBatchFraction: Fraction of data to be used for each
SGD iteration (default: 1.0).
:param initialWeights: The initial weights (default: None).
:param regType: The type of regularizer used for training
our model.
:param regType: The type of regularizer used for
training our model.
:Allowed values:
- "l1" for using L1 regularization
Expand All @@ -427,13 +458,14 @@ def train(cls, data, iterations=100, step=1.0, regParam=0.01,
(default: "l2")
:param intercept: Boolean parameter which indicates the use
or not of the augmented representation for
training data (i.e. whether bias features
are activated or not).
:param validateData: Boolean parameter which indicates if the
algorithm should validate data before training.
(default: True)
:param intercept: Boolean parameter which indicates the
use or not of the augmented representation
for training data (i.e. whether bias
features are activated or not,
default: False).
:param validateData: Boolean parameter which indicates if
the algorithm should validate data
before training. (default: True)
"""
def train(rdd, i):
return callMLlibFunc("trainSVMModelWithSGD", rdd, int(iterations), float(step),
Expand All @@ -449,9 +481,11 @@ class NaiveBayesModel(Saveable, Loader):
"""
Model for Naive Bayes classifiers.
Contains two parameters:
- pi: vector of logs of class priors (dimension C)
- theta: matrix of logs of class conditional probabilities (CxD)
:param labels: list of labels.
:param pi: log of class priors, whose dimension is C,
number of labels.
:param theta: log of class conditional probabilities, whose
dimension is C-by-D, where D is number of features.
>>> data = [
... LabeledPoint(0.0, [0.0, 0.0]),
Expand Down Expand Up @@ -493,7 +527,10 @@ def __init__(self, labels, pi, theta):
self.theta = theta

def predict(self, x):
"""Return the most likely class for a data vector or an RDD of vectors"""
"""
Return the most likely class for a data vector
or an RDD of vectors
"""
if isinstance(x, RDD):
return x.map(lambda v: self.predict(v))
x = _convert_to_vector(x)
Expand Down Expand Up @@ -523,16 +560,18 @@ class NaiveBayes(object):
@classmethod
def train(cls, data, lambda_=1.0):
"""
Train a Naive Bayes model given an RDD of (label, features) vectors.
Train a Naive Bayes model given an RDD of (label, features)
vectors.
This is the Multinomial NB (U{http://tinyurl.com/lsdw6p}) which can
handle all kinds of discrete data. For example, by converting
documents into TF-IDF vectors, it can be used for document
classification. By making every vector a 0-1 vector, it can also be
used as Bernoulli NB (U{http://tinyurl.com/p7c96j6}).
This is the Multinomial NB (U{http://tinyurl.com/lsdw6p}) which
can handle all kinds of discrete data. For example, by
converting documents into TF-IDF vectors, it can be used for
document classification. By making every vector a 0-1 vector,
it can also be used as Bernoulli NB (U{http://tinyurl.com/p7c96j6}).
The input feature values must be nonnegative.
:param data: RDD of LabeledPoint.
:param lambda_: The smoothing parameter
:param lambda_: The smoothing parameter (default: 1.0).
"""
first = data.first()
if not isinstance(first, LabeledPoint):
Expand Down
Loading

0 comments on commit ca99875

Please sign in to comment.