To the defense of W&H, one may argue that we did not do justice to their fixed effect approach by using random-effects meta-analyses in our simulations. Random-effects meta-analysis assumes a heterogeneous population effect size, but the data in all scenarios are generated from a population with a fixed effect size. However, a researcher is ignorant about the population effect size and cannot a priori determine that there is just one underlying effect in a set of studies. It is standard practice to use random-effects meta-analysis when there is evidence of effect size heterogeneity [24]. Several studies [28], [29], [30] have shown that mistakenly applying fixed-effect meta-analysis to a heterogeneous effect may lead to biased results and erroneous conclusions [31]. Those results also highlighted that applying random-effects meta-analysis when the underlying population effect is fixed does not yield bad results or erroneous conclusions. The problem with the selective publishing scenario is that the researcher does not know whether the data were generated with that scenario or rather that the data reflect true heterogeneity. Fixed-effect meta-analysis is only appropriate if the researcher
