It is easy to allow different penalties λj for each of the variables. We implement this via a penalty scaling parameter γj ≥ 0. If γj > 0, then the penalty applied to βj is λj = λγj. If γj = 0, that variable does not get penalized, and always enters the model unrestricted at the first step and remains in the model. Penalty rescaling would also allow, for example, our software to be used to implement the adaptive lasso [Zou, 2006].
