Least Angle Regression to solve LASSO-type problems
Computes the entire LASSO solution for the regression coefficients, starting from zero, to the least-squares estimates, via the Least Angle Regression (LARS) algorithm (Efron, 2004). It uses as inputs a variance matrix among predictors and a covariance vector between response and predictors.
lars2(P, v, method = c("LAR", "LAR-LASSO"), maxDF = NULL,
eps = .Machine$double.eps, scale = TRUE, verbose = FALSE)P |
(numeric matrix) Variance-covariance matrix of predictors. It can be of the "float32" type as per the 'float' R-package |
v |
(numeric vector) Covariance between response variable and predictors |
method |
(character) Either:
Default is |
maxDF |
(integer) Maximum number of predictors in the last LARS solution.
Default |
eps |
(numeric) An effective zero. Default is the machine precision |
scale |
|
verbose |
|
Finds solutions for the regression coefficients in a linear model
where yi is the response for the ith observation, xi=(xi1,...,xip)' is a vector of p predictors assumed to have unit variance, β=(β1,...,βp)' is a vector of regression coefficients, and ei is a residual.
The regression coefficients β are estimated as function of the variance matrix among predictors (P) and the covariance vector between response and predictors (v) by minimizing the penalized mean squared error function
where λ is the penalization parameter and ||β||1 = ∑|βj| is the L1-norm.
The algorithm to find solutions for each βj is fully described in Efron (2004) in which the "current correlation" between the predictor xij and the residual ei = yi - x'iβ is expressed (up-to a constant) as
where vj is the jth element of v and Pj is the jth column of the matrix P
Returns a list object with the following elements:
lambda: (vector) all the sequence of values of the LASSO penalty.
beta: (matrix) regression coefficients for each predictor (in columns) associated to each value of the penalization parameter lambda (in rows).
df: (vector) degrees of freedom, number of non-zero predictors associated to each value of lambda.
The returned object is of the class 'LASSO' for which methods fitted exist. Function plotPath can be also used
Marco Lopez-Cruz (maraloc@gmail.com) and Gustavo de los Campos. Adapted from the 'lars' function in package 'lars' (Hastie & Efron, 2013)
Efron B, Hastie T, Johnstone I, Tibshirani R (2004). Least angle regression. The Annals of Statistics, 32(2), 407–499.
Friedman J, Hastie T, Tibshirani R(2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1), 1–22.
Hastie T, Efron B (2013). lars: least angle regression, Lasso and forward stagewise. https://cran.r-project.org/package=lars.
Tibshirani R (1996). Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society B, 58(1), 267–288.
require(SFSI)
data(wheatHTP)
y = as.vector(Y[,"YLD"]) # Response variable
X = scale(WL) # Predictors
# Training and testing sets
tst = 1:ceiling(0.3*length(y))
trn = seq_along(y)[-tst]
# Calculate covariances in training set
XtX = var(X[trn,])
Xty = cov(y[trn],X[trn,])
# Run the penalized regression
fm1 = lars2(XtX,Xty,method="LAR-LASSO")
# Predicted values
yHat1 = fitted(fm1, X=X[trn,]) # training data
yHat2 = fitted(fm1, X=X[tst,]) # testing data
# Penalization vs correlation
plot(-log(fm1$lambda),cor(y[trn],yHat1)[1,], main="training")
plot(-log(fm1$lambda),cor(y[tst],yHat2)[1,], main="testing")
if(requireNamespace("float")){
# Using a 'float' type variable
XtX2 = float::fl(XtX)
fm2 = lars2(XtX2,Xty,method="LAR-LASSO")
max(abs(fm1$beta-fm2$beta)) # Check for discrepances in beta
max(abs(fm1$lambda-fm2$lambda)) # Check for discrepances in lambda
}Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.