RSSL (version 0.9.3)

ICLeastSquaresClassifier: Implicitly Constrained Least Squares Classifier

Description

Implementation of the Implicitly Constrained Least Squares Classifier (ICLS) of Krijthe & Loog (2015) and the projected estimator of Krijthe & Loog (2016).

Usage

ICLeastSquaresClassifier(X, y, X_u = NULL, lambda1 = 0, lambda2 = 0,
  intercept = TRUE, x_center = FALSE, scale = FALSE, method = "LBFGS",
  projection = "supervised", lambda_prior = 0, trueprob = NULL,
  eps = 1e-09, y_scale = FALSE, use_Xu_for_scaling = TRUE)

Arguments

X

Design matrix, intercept term is added within the function

y

Vector or factor with class assignments

X_u

Design matrix of the unlabeled data, intercept term is added within the function

lambda1

Regularization parameter in the unlabeled+labeled data regularized least squares

lambda2

Regularization parameter in the labeled data only regularized least squares

intercept

TRUE if an intercept should be added to the model

x_center

logical; Whether the feature vectors should be centered

scale

logical; If TRUE, apply a z-transform to all observations in X and X_u before running the regression

method

Either "LBFGS" for solving using L-BFGS-B gradient descent or "QP" for a quadratic programming based solution

projection

One of "supervised", "semisupervised" or "euclidean"

lambda_prior

numeric; prior on the deviation from the supervised mean y

trueprob

numeric; true mean y for all data

eps

numeric; Stopping criterion for the maximinimization

y_scale

logical; whether the target vector should be centered

use_Xu_for_scaling

logical; whether the unlabeled objects should be used to determine the mean and scaling for the normalization

Value

S4 object of class ICLeastSquaresClassifier with the following slots:

theta

weight vector

classnames

the names of the classes

modelform

formula object of the model used in regression

scaling

a scaling object containing the parameters of the z-transforms applied to the data

optimization

the object returned by the optim function

unlabels

the labels assigned to the unlabeled objects

Details

In Implicitly Constrained semi-supervised Least Squares (ICLS) of Krijthe & Loog (2015), we minimize the quadratic loss on the labeled objects, while enforcing that the solution has to be a solution that minimizes the quadratic loss for all objects for some (fractional) labeling of the data (the implicit constraints). The goal of this classifier is to use the unlabeled data to update the classifier, while making sure it still works well on the labeled data.

The Projected estimator of Krijthe & Loog (2016) builds on this by finding a classifier within the space of classifiers that minimize the quadratic loss on all objects for some labeling (the implicit constrained), that minimizes the distance to the supervised solution for some appropriately chosen distance measure. Using the projection="semisupervised", we get certain guarantees that this solution is always better than the supervised solution (see Krijthe & Loog (2016)), while setting projection="supervised" is equivalent to ICLS.

Both methods (ICLS and the projection) can be formulated as a quadratic programming problem and solved using either a quadratic programming solver (method="QP") or using a gradient descent approach that takes into account certain bounds on the labelings (method="LBFGS"). The latter is the preferred method.

References

Krijthe, J.H. & Loog, M., 2015. Implicitly Constrained Semi-Supervised Least Squares Classification. In E. Fromont, T. De Bie, & M. van Leeuwen, eds. 14th International Symposium on Advances in Intelligent Data Analysis XIV (Lecture Notes in Computer Science Volume 9385). Saint Etienne. France, pp. 158-169.

Krijthe, J.H. & Loog, M., 2016. Projected Estimators for Robust Semi-supervised Classification. arXiv preprint arXiv:1602.07865.

See Also

Other RSSL classifiers: EMLeastSquaresClassifier, EMLinearDiscriminantClassifier, GRFClassifier, ICLinearDiscriminantClassifier, KernelLeastSquaresClassifier, LaplacianKernelLeastSquaresClassifier(), LaplacianSVM, LeastSquaresClassifier, LinearDiscriminantClassifier, LinearSVM, LinearTSVM(), LogisticLossClassifier, LogisticRegression, MCLinearDiscriminantClassifier, MCNearestMeanClassifier, MCPLDA, MajorityClassClassifier, NearestMeanClassifier, QuadraticDiscriminantClassifier, S4VM, SVM, SelfLearning, TSVM, USMLeastSquaresClassifier, WellSVM, svmlin()

Examples

# NOT RUN {
data(testdata)
w1 <- LeastSquaresClassifier(testdata$X, testdata$y, 
                             intercept = TRUE,x_center = FALSE, scale=FALSE)
w2 <- ICLeastSquaresClassifier(testdata$X, testdata$y, 
                               testdata$X_u, intercept = TRUE, x_center = FALSE, scale=FALSE)
plot(testdata$X[,1],testdata$X[,2],col=factor(testdata$y),asp=1)
points(testdata$X_u[,1],testdata$X_u[,2],col="darkgrey",pch=16,cex=0.5)

abline(line_coefficients(w1)$intercept,
       line_coefficients(w1)$slope,lty=2)
abline(line_coefficients(w2)$intercept,
       line_coefficients(w2)$slope,lty=1)

# }