paperKB
coga / coga-kb
Help
Sign in

Chunk #15 — Discussion

Source
Why publishing everything is more effective than selective publishing of statistically significant results.
Embedded
yes

Text

On the basis of results of their simulation, W&H concluded that, after a fixed number of publications, selective publishing yields a more accurate estimate of the true effect than publishing everything. We argue that their analysis of their results has two weaknesses. First, they use information that a researcher does not have, i.e., the standard deviation of the meta-analytic effect after many simulations; a researcher only possesses a standard error of one meta-analysis on the basis of all available results. Second, W&H analyzed their data disregarding the possible heterogeneity of effect sizes in a given meta-analysis. After re-analyzing their simulation results using random-effects meta-analysis we conclude, contrary to W&H that publishing everything yields a more accurate estimate than selective publishing. We also compared the two approaches in another scenario where the population effect is zero and scientists stop investigating the effect when they reject the null hypothesis that the population effect is at least small. The results in the latter scenario favored publishing everything even more; on average less than 4 studies were needed in the publishing everything approach, whereas