The theoretical results of Breiman (1996a) do support the fact that ensemble methods do not overfit with an increasing number of trees. However, the real data “case studies” referred to in Breiman (2001b) do not exclude the possibility that they overfit due to other reasons. For further methodological investigations of machine learning algorithms we therefore strongly suggest to employ well designed and controlled simulation experiments, rather than case studies with an unrepresentative selection of real data sets with unknown distributional properties, when analytical results are not feasible.
