We compared scCODA’s performance to state-of-the-art differential compositional testing schemes from the microbiome field as well as all non-compositional tests recently applied to single-cell data (Fig. 2), all with a nominal FDR level of 0.05. In our synthetic benchmarks, we found scCODA to significantly outperform all non-Bayesian approaches in the regime of low-sample sizes across a wide variety of effects and experimental settings with an average Matthews’ correlation coefficient (MCC) of 0.64. Considering the number of replicates per group, the Bayesian models (scCODA and a standard Dirichlet-multinomial modeling approach; red lines in Fig. 2) had a considerable edge over all other methods in the common scenario with a low number of replicates per group, and increased their MCC further with the sample size (Fig. 2c). Other compositional non-Bayesian models such as ANCOM-BC13, ANCOM14, ALDEx215, and additive log-ratio (ALR) transformed proportions combined with a t test (Methods—“Model comparison”) showed similar behavior, albeit with lower MCC. Non-compositional models, such as the Beta-Binomial model16, the scDC model7, or univariate t tests, (purple lines in Fig. 2) included more false positives with increasing effect
