For further explanation on how to get these estimates, please refer to reference [14]. Accounting for correlation between ROC curves is a necessary step for paired data where two classifiers are used on the same subjects. For balanced unpaired data the off-diagonal elements of S are set to zero since the AUCs are not correlated. Consequently, the covariance matrix S is used to compute the following chi-squared statistic for testing if there is a difference between two or more classifiers:
