The optimal threshold (OT) for each classifier is defined after the ROC analysis is performed and consists in the score value that leads to the maximal accuracy of classification. The assessment of the statistical significance of the differences of the AUCs between two classifiers is implemented as previously described [14]. Briefly, suppose that R tests are applied on the same N individuals, which can be classified as positive or negative. Suppose that m of these individuals are actually positive and n are actually negative (m+n = N), and that positive individuals tend to have greater values than negative individuals. If we let {Xir} and {Yjr} be the sets of outcome values on the r-th test that correspond to the positive and negative individuals, respectively (i = 1,...,m; j = 1,...,n; 1 ≤ r ≤ R), the AUC for each classifier is computed with the Mann-Whitney U-statistic for comparing distributions of values from two samples, as follows:
