Sailing From the Seas of Chaos Into the Corridor of Stability: Practical Recommendations to Increase the Informational Value of Studies.
paper
Cited
Public
Unavailable
- Authors
- Lakens, Daniël; Evers, Ellen R K
- Year
- 2014
- Journal
- Perspectives on psychological science : a journal of the Association for Psychological Science
- PMID
- 26173264
- DOI
- 10.1177/1745691614528520
Recent events have led psychologists to acknowledge that the inherent uncertainty encapsulated in an inductive science is amplified by problematic research practices. In this article, we provide a practical introduction to recently developed statistical tools that can be used to deal with these uncertainties when performing and evaluating research. In Part 1, we discuss the importance of accurate and stable effect size estimates as well as how to design studies to reach a corridor of stability around effect size estimates. In Part 2, we explain how, given uncertain effect size estimates, well-powered studies can be designed with sequential analyses. In Part 3, we (a) explain what p values convey about the likelihood that an effect is true, (b) illustrate how the v statistic can be used to evaluate the accuracy of individual studies, and (c) show how the evidential value of multiple studies can be examined with a p-curve analysis. We end by discussing the consequences of incorporating our recommendations in terms of a reduced quantity, but increased quality, of the research output. We hope that the practical recommendations discussed in this article will provide researchers with the tools to make important steps toward a psychological science that allows researchers to differentiate among all possible truths on the basis of their likelihood.
No figures extracted from this document.
No chunks — full text not yet ingested.
No entities extracted from this document yet.
No uploaded files.
Not in any collection.
No citations found.
In this knowledge base
| Title | Year | PMID |
|---|---|---|
| Candidate gene-environment interaction research: reflections and recommendations. | 2015 | 25620996 |
External
| Title | Authors | Journal | Year | Link |
|---|---|---|---|---|
| A tutorial for calculating field-specific effect size distributions. | Glaser BD et al. | — | 2026 | → |
| How to design real-world functional near-infrared spectroscopy studies: a primer. | Jones IL et al. | — | 2026 | → |
| Reducing public stigma toward people with mental illness through the ACE intervention: a cluster-randomized waitlist-controlled trial. | Zhang Q et al. | — | 2026 | → |
| The Leiden Stroop-like Stress Task: A tool to reveal physiological dynamics using graded stress induction. | Yan J et al. | — | 2026 | → |
| Agility and the three musketeers: The interplay of epistemic curiosity, joy, and rumination. | Janani M et al. | — | 2025 | → |
| Bypassing versus correcting misinformation: Efficacy and fundamental processes. | Granados Samayoa JA et al. | — | 2025 | → |
| Novel cVEMP procedure reveals sexual dimorphism in peak to trough latency. | Gattie M et al. | — | 2025 | → |
| Registered reports in neuropsychology: Insights from the burning houses study. | Moore MJ et al. | — | 2025 | → |
| An investigation of the modulatory effects of empathic and autistic traits on emotional and facial motor responses during live social interactions. | Hsu CT et al. | — | 2024 | → |
| No evidence that belief in conspiracy theories is negatively related to attitudes toward transhumanism. | Lantian A et al. | — | 2024 | → |
| Power to Detect What? Considerations for Planning and Evaluating Sample Size. | Giner-Sorolla R et al. | — | 2024 | → |
| Sanity through Insanity: The Use of Dark Humor among United States Veterans. | Yoshimura SM et al. | — | 2024 | → |
| Unpleasant emotion inhibits attentional focus toward a peripheral target in a visual search: an ERP study. | Sanada M et al. | — | 2024 | → |
| Exercising choice over feedback schedules during practice is not advantageous for motor learning. | St Germain L et al. | — | 2023 | → |
| Extraordinary Claims in the Literature on High-Intensity Interval Training (HIIT): I. Bonafide Scientific Revolution or a Looming Crisis of Replication and Credibility? | Ekkekakis P et al. | — | 2023 | → |
| Improving the stability of bivariate correlations using informative Bayesian priors: a Monte Carlo simulation study. | Delfin C | — | 2023 | → |
| Is Open Science Neoliberal? | Uygun Tunç D et al. | — | 2023 | → |
| Measuring auditory event-related potentials at the external ear canal: A demonstrative study using a new electrode and error-feedback paradigm. | Arao H et al. | — | 2023 | → |
| Mimicry and law: Experiments in a natural setting of a law company. | Kulesza W et al. | — | 2023 | → |
| Permutation-based group sequential analyses for cognitive neuroscience. | Veillette JP et al. | — | 2023 | → |
| Publication bias, statistical power and reporting practices in the Journal of Sports Sciences: potential barriers to replicability. | Mesquida C et al. | — | 2023 | → |
| Sample size considerations in soft tissue biomechanics. | Hammer N et al. | — | 2023 | → |
| The effects of temperature on prosocial and antisocial behaviour: A review and meta-analysis. | Lynott D et al. | — | 2023 | → |
| To Lead, or to Follow? How Self-Uncertainty and the Dark Triad of Personality Influence Leadership Motivation. | Guillén L et al. | — | 2023 | → |
| An Online Experiment of NHS Information Framing on Mothers' Vaccination Intention of Children against COVID-19. | Van Hoecke AL et al. | — | 2022 | → |
| Crisis Ahead? Why Human-Robot Interaction User Studies May Have Replicability Problems and Directions for Improvement. | Leichtmann B et al. | — | 2022 | → |
| Don't dismiss gender equality: A response to Cid & Leguisamo (2021) "Gender equality does not prevent sexual abuse of women". | Reidy DE et al. | — | 2022 | → |
| Efficiency in sequential testing: Comparing the sequential probability ratio test and the sequential Bayes factor test. | Stefan AM et al. | — | 2022 | → |
| Meaningful change definitions: sample size planning for experimental intervention research. | Gruijters SLK et al. | — | 2022 | → |
| Multiverse to inform neurological research: an example using recovery outcome of neglect. | Moore MJ et al. | — | 2022 | → |
| Myths and methodologies: The use of equivalence and non-inferiority tests for interventional studies in exercise physiology and sport science. | Mazzolari R et al. | — | 2022 | → |
| Optimizing Research Output: How Can Psychological Research Methods Be Improved? | Miller J et al. | — | 2022 | → |
| The Great Porn Experiment V2.0: Sexual Arousal Reduces the Salience of Familiar Women When Heterosexual Men Judge Their Attractiveness. | Sculley J et al. | — | 2022 | → |
| The Relation of Mood and Sexual Desire: An Experience Sampling Perspective on the Dual Control Model. | van Tuijl P et al. | — | 2022 | → |
| The Spontaneous Counterfactual Inference Task: Misremembering What Might Have Been. | Smallman R et al. | — | 2022 | → |
| Arts in Education: A Systematic Review of Competency Outcomes in Quasi-Experimental and Experimental Studies. | Schneider V et al. | — | 2021 | → |
| Children's and Adults' Views of Punishment as a Path to Redemption. | Dunlea JP et al. | — | 2021 | → |
| Effects of Choice Restriction on Accuracy and User Experience in an Internet-Based Geopolitical Forecasting Task. | Widmer CL et al. | — | 2021 | → |
| Impact of past behaviour normality: meta-analysis of exceptionality effect. | Fillon A et al. | — | 2021 | → |
| Knowing how effective an intervention, treatment, or manipulation is and increasing replication rates: accuracy in parameter estimation as a partial solution to the replication crisis. | Peters GY et al. | — | 2021 | → |
| Loneliness unlocked: Associations with smartphone use and personality. | MacDonald KB et al. | — | 2021 | → |
| The absence of fans removes the home advantage associated with penalties called by National Hockey League referees. | Guérette J et al. | — | 2021 | → |
| The case for formal methodology in scientific reform. | Devezer B et al. | — | 2021 | → |
| A world beyond P: policies, strategies, tactics and advice. | Drummond G | — | 2020 | → |
| Do Children and Adults Take Social Relationship Into Account When Evaluating People's Actions? | Marshall J et al. | — | 2020 | → |
| Exploring Responses to Body Weight Criticism: Defensive Avoidance When Weight Is Seen as Controllable. | Täuber S et al. | — | 2020 | → |
| Moving Sport and Exercise Science Forward: A Call for the Adoption of More Transparent Research Practices. | Caldwell AR et al. | — | 2020 | → |
| Offloading items from memory: individual differences in cognitive offloading in a short-term memory task. | Morrison AB et al. | — | 2020 | → |
| Sans Forgetica is not desirable for learning. | Geller J et al. | — | 2020 | → |
| Attentional capture is contingent on scene region: Using surface guidance framework to explore attentional mechanisms during search. | Pereira EJ et al. | — | 2019 | → |
| A tutorial on Bayes Factor Design Analysis using an informed prior. | Stefan AM et al. | — | 2019 | → |
| Basic statistical considerations for physiology: The journal <i>Temperature</i> toolbox. | Caldwell AR et al. | — | 2019 | → |
| Differential associations of locus of control with anxiety, depression and life-events: A five-wave, nine-year study to test stability and change. | Hovenkamp-Hermelink JHM et al. | — | 2019 | → |
| Estimation for Better Inference in Neuroscience. | Calin-Jageman RJ et al. | — | 2019 | → |
| Exact replication: Foundation of science or game of chance? | Piper SK et al. | — | 2019 | → |
| Improving Time Estimation in Witness Memory. | Gasper HL et al. | — | 2019 | → |
| Quantifying the presence of evidential value and selective reporting in food-related inhibitory control training: a <i>p</i>-curve analysis. | Carbine KA et al. | — | 2019 | → |
| Recommendations for Increasing the Transparency of Analysis of Preexisting Data Sets. | Weston SJ et al. | — | 2019 | → |
| Reveal, Don't Conceal: Transforming Data Visualization to Improve Transparency. | Weissgerber TL et al. | — | 2019 | → |
| Social Transmission of Leadership Preference: Knowledge of Group Membership and Partisan Media Reporting Moderates Perceptions of Leadership Ability From Facial Cues to Competence and Dominance. | Watkins CD et al. | — | 2019 | → |
| So, It's Pricier Than Before, but Why? Price Increase Justifications Influence Risky Decision Making and Emotional Response. | Salcedo JC et al. | — | 2019 | → |
| The association between adolescent well-being and digital technology use. | Orben A et al. | — | 2019 | → |
| The impact of sleep on eyewitness identifications. | Morgan DP et al. | — | 2019 | → |
| The role of the opioid system in decision making and cognitive control: A review. | van Steenbergen H et al. | — | 2019 | → |
| Bayes factor design analysis: Planning for compelling evidence. | Schönbrodt FD et al. | — | 2018 | → |
| Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications. | Wagenmakers EJ et al. | — | 2018 | → |
| Does Twitter language reliably predict heart disease? A commentary on Eichstaedt et al. (2015a). | Brown NJL et al. | — | 2018 | → |
| fMRI connectivity of expressive language in young children and adolescents. | Youssofzadeh V et al. | — | 2018 | → |
| Imagining Autism: Feasibility of a drama-based intervention on the social, communicative and imaginative behaviour of children with autism. | Beadle-Brown J et al. | — | 2018 | → |
| Individual Differences in Uncertainty Tolerance Are not Associated With Cognitive Control Functions in the Flanker Task. | Schroeder PA et al. | — | 2018 | → |
| Mind-Wandering and Mindfulness as Mediators of the Relationship Between Online Vigilance and Well-Being. | Johannes N et al. | — | 2018 | → |
| Promoting Replicability in Developmental Research Through Meta-analyses: Insights From Language Acquisition Research. | Bergmann C et al. | — | 2018 | → |
| Testing the effect of cooperative/competitive priming on the Prisoner's Dilemma. A replication study. | Belaus A et al. | — | 2018 | → |
| The Effects of an Online Mindfulness Intervention on Perceived Stress, Depression and Anxiety in a Non-clinical Sample: A Randomised Waitlist Control Trial. | Querstret D et al. | — | 2018 | → |
| The impact of sample size on the reproducibility of voxel-based lesion-deficit mappings. | Lorca-Puls DL et al. | — | 2018 | → |
| Commentary: Mental Toughness and Individual Differences in Learning, Educational and Work Performance, Psychological Well-being, and Personality: A Systematic Review. | Gucciardi DF | — | 2017 | → |
| Daily rhythms and enrichment patterns in the transcriptome of the behavior-manipulating parasite Ophiocordyceps kimflemingiae. | de Bekker C et al. | — | 2017 | → |
| Detecting and avoiding likely false-positive findings - a practical guide. | Forstmeier W et al. | — | 2017 | → |
| Equivalent statistics and data interpretation. | Francis G | — | 2017 | → |
| Mimicry and automatic imitation are not correlated. | Genschow O et al. | — | 2017 | → |
| Null hypothesis significance testing: a guide to commonly misunderstood concepts and recommendations for good practice | Pernet C | — | 2017 | — |
| Reproducibility in Psychological Science: When Do Psychological Phenomena Exist? | Iso-Ahola SE | — | 2017 | → |
| Sample size, statistical power, and false conclusions in infant looking-time research. | Oakes LM | — | 2017 | → |
| Sex differences in discrimination reversal learning in the guppy. | Miletto Petrazzini ME et al. | — | 2017 | → |
| The Affect Misattribution Procedure. | Teige-Mocigemba S et al. | — | 2017 | → |
| The Weapons Identification Task: Recommendations for adequately powered research. | Rivers AM | — | 2017 | → |
| Do "Brain-Training" Programs Work? | Simons DJ et al. | — | 2016 | → |
| Fake feedback on pain tolerance impacts proactive versus reactive control strategies. | Rigoni D et al. | — | 2016 | → |
| Group Dynamics in Automatic Imitation. | Gleibs IH et al. | — | 2016 | → |
| Insights in Hypothesis Testing and Making Decisions in Biomedical Research. | Sacha V et al. | — | 2016 | → |
| Intentional forgetting in dysphoria: Investigating the inhibitory effects of thought substitution using independent cues. | Noreen S et al. | — | 2016 | → |
| On the reproducibility of meta-analyses: six practical recommendations. | Lakens D et al. | — | 2016 | → |
| Optimizing Research Payoff. | Miller J et al. | — | 2016 | → |
| The natural selection of bad science. | Smaldino PE et al. | — | 2016 | → |
| The testing effect for mediator final test cues and related final test cues in online and laboratory experiments. | Coppens LC et al. | — | 2016 | → |
| Candidate gene-environment interaction research: reflections and recommendations. | Dick DM et al. | — | 2015 | → |
| Null hypothesis significance testing: a short tutorial. | Pernet C | — | 2015 | → |
| On the challenges of drawing conclusions from p-values just below 0.05. | Lakens D | — | 2015 | → |
| Re: "increased levels of depressive symptoms among pregnant women in the Netherlands after the crash of flight MH17". | Lakens D | — | 2015 | → |
| Re: "is the inverse association between selenium and bladder cancer due to confounding by smoking?". | Goossens ME et al. | — | 2015 | → |
| Rumination and Performance in Dynamic, Team Sport. | Roy MM et al. | — | 2015 | → |
| The illusion of nonmediation in telecommunication: voice intensity biases distance judgments to a communication partner. | Zhang C et al. | — | 2015 | → |
| The influence of motivation on stress: is it stressful not to fit? | Schwab S et al. | — | 2015 | → |
| The significance fallacy in inferential statistics. | Kühberger A et al. | — | 2015 | → |
| An Ethical Approach to Peeking at Data. | Sagarin BJ et al. | — | 2014 | → |
| Community-Augmented Meta-Analyses: Toward Cumulative Data Assessment. | Tsuji S et al. | — | 2014 | → |
| Continuously Cumulating Meta-Analysis and Replicability. | Braver SL et al. | — | 2014 | → |
| Electrifying the motor engram: effects of tDCS on motor learning and control. | Orban de Xivry JJ et al. | — | 2014 | → |
| Introduction to the Special Section on Advancing Our Methods and Practices. | Ledgerwood A | — | 2014 | → |
| Introduction to the Special Section on Moving Toward a Cumulative Science: Maximizing What Our Research Can Tell Us. | Ledgerwood A | — | 2014 | → |
| Let's Put Our Money Where Our Mouth Is: If Authors Are to Change Their Ways, Reviewers (and Editors) Must Change With Them. | Maner JK | — | 2014 | → |
| Safeguard Power as a Protection Against Imprecise Power Estimates. | Perugini M et al. | — | 2014 | → |