In summary, one can conclude in accordance with Heidema et al. (2006), that high dimensional data should be approached by several different methods because each single method has its strengths and weaknesses: Boosting, for example, can be employed for variable selection in linear and other additive models (Bühlmann 2006; Bühlmann and Hothorn 2007, for an implementation in R). Similarly, shrinkage approaches like the LASSO (cf., e.g., Hastie et al. 2001), the elastic net (Zou and Hastie 2005) and the recent approach of Candes and Tao (2007) perform variable selection in linear models by means of penalization of the model coefficients. However, in contrast to random forests, for these methods it has to be assumed that the model is linear or additive and that the problem is sparse (meaning that only few predictor variables have an effect). For extremely small sample sizes, on the other hand, exact methods like the multivariate permutation tests described in Mielke and Berry (2001) or Good (2005) may be more suited.
