ADMM: admm.enet – R documentation

Become an expert in R — Interactive courses, Cheat Sheets, certificates and more!

admm.enet

Elastic Net Regularization

Description

Elastic Net regularization is a combination of \ell_2 stability and \ell_1 sparsity constraint simulatenously solving the following,

\textrm{min}_x ~ \frac{1}{2}\|Ax-b\|_2^2 + λ_1 \|x\|_1 + λ_2 \|x\|_2^2

with nonnegative constraints λ_1 and λ_2. Note that if both lambda values are 0, it reduces to least-squares solution.

Usage

admm.enet(
  A,
  b,
  lambda1 = 1,
  lambda2 = 1,
  rho = 1,
  abstol = 1e-04,
  reltol = 0.01,
  maxiter = 1000
)

Arguments

`A`	an (m\times n) regressor matrix
`b`	a length-m response vector
`lambda1`	a regularization parameter for \ell_1 term
`lambda2`	a regularization parameter for \ell_2 term
`rho`	an augmented Lagrangian parameter
`abstol`	absolute tolerance stopping criterion
`reltol`	relative tolerance stopping criterion
`maxiter`	maximum number of iterations

Value

a named list containing

x: a length-n solution vector
history: dataframe recording iteration numerics. See the section for more details.

Iteration History

When you run the algorithm, output returns not only the solution, but also the iteration history recording following fields over iterates,

objval: object (cost) function value
r_norm: norm of primal residual
s_norm: norm of dual residual
eps_pri: feasibility tolerance for primal feasibility condition
eps_dual: feasibility tolerance for dual feasibility condition

In accordance with the paper, iteration stops when both r_norm and s_norm values become smaller than eps_pri and eps_dual, respectively.

Author(s)

Xiaozhi Zhu

References

Zou H, Hastie T (2005). “Regularization and variable selection via the elastic net.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2), 301–320. ISSN 1369-7412, 1467-9868, doi: 10.1111/j.1467-9868.2005.00503.x.

Examples

## generate underdetermined design matrix
m = 50
n = 100
p = 0.1   # percentange of non-zero elements

x0 = matrix(Matrix::rsparsematrix(n,1,p))
A  = matrix(rnorm(m*n),nrow=m)
for (i in 1:ncol(A)){
  A[,i] = A[,i]/sqrt(sum(A[,i]*A[,i]))
}
b = A%*%x0 + sqrt(0.001)*matrix(rnorm(m))

## run example with both regularization values = 1
output = admm.enet(A, b, lambda1=1, lambda2=1)
niter  = length(output$history$s_norm)
history = output$history

## report convergence plot
opar <- par(no.readonly=TRUE)
par(mfrow=c(1,3))
plot(1:niter, history$objval, "b", main="cost function")
plot(1:niter, history$r_norm, "b", main="primal residual")
plot(1:niter, history$s_norm, "b", main="dual residual")
par(opar)

ADMM

Algorithms using Alternating Direction Method of Multipliers

v0.3.2

GPL (>= 3)

Authors

Kisung You [aut, cre] (<https://orcid.org/0000-0002-8584-459X>), Xiaozhi Zhu [aut]

Initial release