Are Small Effects the Indispensable Foundation for a Cumulative Psychological Science? A Reply to Götz et al. (2022)

Primbs, M.A., Pennington, C.R., Lakens, D., Silan, M.A.A., Lieck, D.S.N., Forscher, P.S., Buchanan, E.M. and Westwood, S. 2023. Are Small Effects the Indispensable Foundation for a Cumulative Psychological Science? A Reply to Götz et al. (2022). Perspectives on Psychological Science. 18 (2), pp. 508-512. https://doi.org/10.1177/17456916221100420

TitleAre Small Effects the Indispensable Foundation for a Cumulative Psychological Science? A Reply to Götz et al. (2022)
TypeJournal article
AuthorsPrimbs, M.A., Pennington, C.R., Lakens, D., Silan, M.A.A., Lieck, D.S.N., Forscher, P.S., Buchanan, E.M. and Westwood, S.
Abstract

In the January 2022 issue of Perspectives, Götz et al. argued that small effects are “the indispensable foundation for a cumulative psychological science.” They supported their argument by claiming that (a) psychology, like genetics, consists of complex phenomena explained by additive small effects; (b) psychological-research culture rewards large effects, which means small effects are being ignored; and (c) small effects become meaningful at scale and over time. We rebut these claims with three objections: First, the analogy between genetics and psychology is misleading; second, p values are the main currency for publication in psychology, meaning that any biases in the literature are (currently) caused by pressure to publish statistically significant results and not large effects; and third, claims regarding small effects as important and consequential must be supported by empirical evidence or, at least, a falsifiable line of reasoning. If accepted uncritically, we believe the arguments of Götz et al. could be used as a blanket justification for the importance of any and all “small” effects, thereby undermining best practices in effect-size interpretation. We end with guidance on evaluating effect sizes in relative, not absolute, terms.

JournalPerspectives on Psychological Science
Journal citation18 (2), pp. 508-512
ISSN1745-6924
Year2023
PublisherSage
Publisher's version
License
CC BY 4.0
File Access Level
Open (open metadata and files)
Digital Object Identifier (DOI)https://doi.org/10.1177/17456916221100420
PubMed ID36126652
Web address (URL)https://doi.org/10.1177/17456916221100420
Publication dates
Published online20 Sep 2022
Published in print2023
ProjectEffect Size Working Group
FunderNederlandse Organisatie voor Wetenschappelijk Onderzoek

Related outputs

Clinical and cognitive effects of external trigeminal nerve stimulation (eTNS) in neurological and psychiatric disorders: a systematic review and meta-analysis.
Westwood, Samuel, Conti, Aldo Alberto, Tang, Wanjie, Xue, Shuang, Cortese, Samuele and Rubia, Katja 2023. Clinical and cognitive effects of external trigeminal nerve stimulation (eTNS) in neurological and psychiatric disorders: a systematic review and meta-analysis. Molecular Psychiatry. 28, pp. 4025-4043. https://doi.org/10.1038/s41380-023-02227-4

Computerized cognitive training in attention-deficit/hyperactivity disorder (ADHD): a meta-analysis of randomized controlled trials with blinded and objective outcomes
Samuel J. Westwood, Valeria Parlatini, Katya Rubia, Samuele Cortese, Edmund J. S. Sonuga-Barke, T. Banaschewski, D. Baeyens, S. Bölte, D. Brandeis, J. Buitelaar, S. Carucci, D. Coghill, D. Daley, M. Döpfner, M. Ferrin, C. Galera, C. Hollis, M. Holtmann, D. Purper-Ouakil, P. Nagy, P. Santosh, E. Simonoff, E. J. Sonuga-Barke, C. A. Soutullo, A. Stringaris, A. Thapar, S. van der Oord, B. J. van den Hoofdakker and A. Zuddas 2023. Computerized cognitive training in attention-deficit/hyperactivity disorder (ADHD): a meta-analysis of randomized controlled trials with blinded and objective outcomes. Molecular Psychiatry. 28, p. 1402–1414. https://doi.org/10.1038/s41380-023-02000-7

Effects of transcranial direct current stimulation in children and young people with psychiatric disorders: a systematic review
Gallop, Lucy, Westwood, Samuel J, Lewis, Yael, Campbell, Iain C and Schmidt, Ulrike 2023. Effects of transcranial direct current stimulation in children and young people with psychiatric disorders: a systematic review. European child & adolescent psychiatry. Advanced online publication. https://doi.org/10.1007/s00787-023-02157-0

Double-Blind, Sham-Controlled Randomized Trial Testing the Efficacy of fMRI Neurofeedback on Clinical and Cognitive Measures in Children With ADHD
Lam, Sheut-Ling, Criaud, Marion, Lukito, Steve, Westwood, Samuel J., Agbedjro, Deborah, Kowalczyk, Olivia S., Curran, Sarah, Barret, Nadia, Abbott, Chris, Liang, Holan, Simonoff, Emily, Barker, Gareth J., Giampietro, Vincent and Rubia, Katya 2022. Double-Blind, Sham-Controlled Randomized Trial Testing the Efficacy of fMRI Neurofeedback on Clinical and Cognitive Measures in Children With ADHD. American Journal of Psychiatry. 179 (12), pp. 947-958. https://doi.org/10.1176/appi.ajp.21100999

What senior academics can do to support reproducible and open research: a short, three-step guide
Kowalczyk, O., Lautarescu, A., Blok, E., Dall’Aglio, L. and Westwood, S. 2022. What senior academics can do to support reproducible and open research: a short, three-step guide. BMC Research Notes. 15 116. https://doi.org/10.1186/s13104-022-05999-0

Towards a culture of open scholarship: The role of pedagogical communities
Azevedo, F., Liu, M., Pennington, C.R., Pownall, M., Evans, T.R., Parsons, S., Elsherif, M.M., Micheli, L., Westwood, S.J. and Framework for Open, Reproducible Research Training (FORRT) 2022. Towards a culture of open scholarship: The role of pedagogical communities. BMC Research Notes. 15 75. https://doi.org/10.1186/s13104-022-05944-1

Towards a culture of open scholarship: the role of pedagogical communities.
Azevedo, F., Liu, Meng, Pennington, Charlotte R, Pownall, Madeleine, Evans, Thomas Rhys, Parsons, Sam, Elsherif, Mahmoud Medhat, Micheli, Leticia, Westwood, Samuel J and Framework for Open, Reproducible Research Training (FORRT) 2022. Towards a culture of open scholarship: the role of pedagogical communities. BMC Research Notes. 15 (1) 75. https://doi.org/10.1186/s13104-022-05944-1

The effect of transcranial direct current stimulation (tDCS) combined with cognitive training on EEG spectral power in adolescent boys with ADHD
Westwood, S., Bozhilova, N., Criaud, M., Lam, Sheut-Ling, Lukito, S.D.A.W., Wallace-Hanlon, S., Kowalczyk, O.S., Kostara, A., Mathew, J., Wexler, B.E., Kadosh, R.C., Asherson, P. and Rubia, K. 2022. The effect of transcranial direct current stimulation (tDCS) combined with cognitive training on EEG spectral power in adolescent boys with ADHD. IBRO Neuroscience Reports. 12, pp. 55-64. https://doi.org/10.1016/j.ibneur.2021.12.005

What senior academics can do to support reproducible and open research: a short, three-step guide
Kowalczyk, O.S., Lautarescu, A., Blok, E., Dall’Aglio, L. and Westwood, S.J. 2022. What senior academics can do to support reproducible and open research: a short, three-step guide. BMC Research Notes. 15 116. https://doi.org/10.1186/s13104-022-05999-0

Systematic Review and Meta-analysis: Screening Tools for Attention-Deficit/Hyperactivity Disorder in Children and Adolescents.
Mulraney, Melissa, Arrondo, Gonzalo, Musullulu, Hande, Iturmendi-Sabater, Iciar, Cortese, Samuele, Westwood, Samuel J, Donno, Federica, Banaschewski, Tobias, Simonoff, Emily, Zuddas, Alessandro, Döpfner, Manfred, Hinshaw, Stephen P and Coghill, David 2021. Systematic Review and Meta-analysis: Screening Tools for Attention-Deficit/Hyperactivity Disorder in Children and Adolescents. Journal of the American Academy of Child and Adolescent Psychiatry. 61 (8), pp. 982-996. https://doi.org/10.1016/j.jaac.2021.11.031

Limits to tDCS effects in language: Failures to modulate word production in healthy participants with frontal or temporal tDCS
Westwood, S., Olson, A., Miall, C., Nappo, R. and Romani, C. 2017. Limits to tDCS effects in language: Failures to modulate word production in healthy participants with frontal or temporal tDCS. Cortex. 86, pp. 64-82. https://doi.org/10.1016/j.cortex.2016.10.016

Permalink - https://westminsterresearch.westminster.ac.uk/item/vx86w/are-small-effects-the-indispensable-foundation-for-a-cumulative-psychological-science-a-reply-to-g-tz-et-al-2022


Share this

Usage statistics

50 total views
24 total downloads
These values cover views and downloads from WestminsterResearch and are for the period from September 2nd 2018, when this repository was created.