The Caspi group that originally proposed the hypothesis16 raised concerns regarding this meta-analysis project; in particular, the decisions to exclude small studies, and to include lifetime depression as an outcome for analysis were criticized.68 As noted in our methods, although we required studies to have at least 300 participants overall, inclusion in any particular meta-analysis required only 50 of these subjects to be genotyped and have the appropriate phenotypes (ancestry, depression outcome, covariates). Although Moffitt and Caspi argue that small studies may be meticulously designed and have high quality data,68 there is a case to be made that large studies are generally likely to have better design quality than small studies.69 In addition, small studies are subject to multiple statistical issues, including publication bias (exacerbated for small studies) and the winner’s curse (which makes it likely, even if a true effect is detected, that the magnitude will be exaggerated).70 In fact, a 2013 analysis of neuroscience publications concluded that small sample size studies were undermining the reliability of neuroscience.69
