2024-08-15 Bonn Melbourne - Cognitive biases researchers and the research process

We study human psyche and behavior, yet we do not often reflect on how our findings relate to us researchers and the way we conduct research. When we do, we sometimes tend to see ourselves as less biased than others, be it compared to the average person, journal reviewers/editors, university administration, our colleagues, our students, etc. That, alone, is an interesting reflection of the classic “bias blind spot”, in our inability to fully appreciate our own biases as researchers and understand the consequences in how these are hindering research.
I will briefly summarize our large-scale collaboration of completing over 150 replications and extensions of classic findings on classic heuristics and biases in judgment and decision-‎making, and will present an ongoing research program to examine cognitive biases in researchers and the scientific academic process.
Researchers are humans and are constrained by cognitive limitations, and there is increasing evidence that these gravely impact all stages of the research life-cycle. From literature review, formation of research questions and ‎study design, through data analysis and the presentation of results, to journal submissions for ‎publication, the review process, and public dissemination and media engagement, researchers engage in a wide ‎array of practices that are possibly distorted by biases, hindering progress towards more credible accurate unbiased science.‎
I will touch on some of our work and evidence regarding 8 specific biases as a demonstration of implications for science: Self-interest is over-estimated, Bias Blind Spot, Outcome Bias, Hindsight Bias, Confirmation Bias, Omission Bias, Default Bias, and Status Quo Bias.
Finally, I will briefly mention potential remedies, mostly in the context of the ongoing science reform movement, the emerging practice of "Registered Reports", and the "Peer Community in Registered Reports" revolution.

本研究探讨人类心理与行为，然而，我们往往未能深思我们的发现如何与研究者自身及其研究方法相联系。当我们反思时，有时我们倾向于将自己视为他人，无论是与普通人、期刊审稿人/编辑、大学管理层、同事、学生等相比，都显得较少偏见。这一点，本身便是对经典‘偏见盲点’的有趣反映，即我们作为研究者无法充分认识到自身的偏见，并理解这些偏见如何阻碍研究的进展。我们将简要概述我们完成超过150项经典判断与决策偏见研究的复制和扩展的大规模合作，并呈现一个持续进行的研究计划，以检验研究者和科学学术过程中的认知偏见。研究者作为人类，受到认知局限性的约束，越来越多的证据表明这些局限严重影响了研究生命周期的各个阶段。从文献综述、研究问题形成和实验设计，到数据分析、结果展示，再到期刊投稿、审稿过程、公共传播和媒体互动，研究者参与了一系列可能受到偏见扭曲的实践，阻碍了向更可信、更准确、无偏见的科学进步。我将简要提及我们关于8种特定偏见的部分工作和证据，以展示其对科学的影响：自我利益高估、偏见盲点、结果偏见、事后诸葛亮偏见、确认偏见、遗漏偏见、默认偏见和现状偏见。最后，我将简要介绍一些可能的补救措施，主要是在持续的科学改革运动、新兴的‘注册报告’实践以及‘注册报告中的同行社区’革命背景下。