Random forest versus logistic regression: a large-scale benchmark experiment.

Example of partial dependence plots. Plot of the PDP for the three simulated datasets. Each line is related to a dataset. On the left, visualization of the dataset. On the right, the partial dependence for the variable X1. First dataset: β0=1,β1=5,β2=−2 (linear), second dataset: β0=1,β1=1,β2=−1,β3=3 (interaction), third dataset β0=−2,β4=5 (non-linear)

Selection of datasets. Flowchart representing the criteria for selection of the datasets

Main results of the benchmark experiment. Boxplots of the performance for the three considered measures on the 243 considered datasets. Top: boxplot of the performance of LR (dark) and RF (white) for each performance measure. Bottom: boxplot of the difference of performances Δperf=perfRF−perfLR

Influence of n and p: subsampling experiment based on dataset ID=310. Top: Boxplot of the performance (acc) of RF (dark) and LR (white) for N=50 sub-datasets extracted from the OpenML dataset with ID=310 by randomly picking n′≤n observations and p′<p features. Bottom: Boxplot of the differences in performances Δacc=AccRF−AccLR between RF and LR. p′∈{1,2,3,4,5,6}. n′∈{5e2,1e3,5e3,1e4}. Performance is evaluated through 5-fold-cross-validation repeated 2 times

Subgroup analyses. Top: for each of the four selected meta-features n, p, p/n and Cmax, boxplots of Δacc for different thresholds as criteria for dataset selection. Bottom: distribution of the four meta-features (log scale), where the chosen thresholds are displayed as vertical lines. Note that outliers are not shown here for a more convenient visualization. For a corresponding figure including the outliers as well as the results for auc and brier, see Additional file 1

Plot of the partial dependence for the 4 considered meta-features : log(n), log(p), \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$log{\left (\frac {p}{n}\right)}$\end{document}logpn, Cmax. The log scale was chosen for 3 of the 4 features to obtain more uniform distribution (see Fig. 5 where the distribution is plotted in log scale). For each plot, the black line denotes the median of the individual partial dependences, and the lower and upper curves of the grey regions represent respectively the 25%- und 75%-quantiles. Estimated mse is 0.00382 via a 5-CV repeated 4 times