paperKB
coga / coga-kb
Help
Sign in

Chunk #9 — Method — Publishing everything is more effective (I): Re-analysis of W&H's results

Source
Why publishing everything is more effective than selective publishing of statistically significant results.
Embedded
yes

Text

To illustrate the higher precision of the meta-analytic effect in publishing everything, we ran exactly the same simulation as W&H but now recorded the sampling error of the estimate after 40 publications for each of the 5,000 simulations. We applied random-effects meta-analysis, because random-effects meta-analysis is generally recommended when the underlying population effect may be heterogeneous [24]. We used the R package metafor [25] (see Appendix S1 for the R code used in all simulations). Heterogeneity is expressed as the variance of the underlying effect [24]. Figure 1 shows the cumulative meta-analytic effect ± the standard error in both approaches as a function of publication number. Figure 1, a correction of W&H's Figure 3, mimics the development of the meta-analytic effect but shows that estimation was more precise under the publishing everything approach. The average standard error after 40 publications in the publishing everything approach was 0.023, with = 0 in 45.6% of the simulations, and an average value of equal to 0.0023. Note that the average standard error is close to both the theoretically derived = 0.02236 and the