Cognitive science employs a range of methods producing a multitude of measures and rich datasets. Given this quantity of data, the potential analyses available and pressures to publish, the temptation to use arbitrary (instead of planned) statistical analyses and to mine data are substantial and may lead to questionable research practices. Even extreme cases of fraud in the cognitive sciences, such as reporting of experiments that never took place, falsifying results and confabulating data have been recently reported [3–5]. Overt fraud is probably rare, but evidence of reporting biases in multiple domains of the cognitive sciences (discussed below) raise concern about the prevalence of questionable practices that occur via motivated reasoning without any intent to be fraudulent.
