We suppose that our data are (Yi, Xi), where Yi (i = 1, . . ., n) is a continuous variable. The LASSO solution is to the optimization problem of minimizing ∑i=1n(Yi−βTXi)2+λ∑j=1p|βj|,(2) where β = (β1, . . ., βp) and λ ≥ 0 is a penalty term. Thus, the constraint that is utilized is an L1 constraint. An alternative way of formulating (2) is to minimize ∑i=1n(Yi−βTXi)2, subject to the constraint that ∑j=1p|βj|≤t. Note that in the absence of the constraint, the solution is given by the ordinary least squares (OLS) estimator. If the usual OLS estimator satisfies the constraint, then the LASSO and OLS estimates of β coincide. However, for smaller values of t, some of the components of β are estimated to be zero. In the linear regression setting, LASSO estimation has been considered by Tibshirani [11].
