2024-10-29 Lingnan University - Credible science: Understanding and overcoming challenges

Do we know how to assess the credibility of information and scientific evidence? It’s a routine task we face in our everyday lives and in our work as scientists, and yet in recent years we’ve increasingly become aware of our limitations – we are not sure how and not very good at assessing credibility. This has immense implications, ranging from making simple decisions, to addressing urgent global challenges such as pandemics and climate change. In every step of the way our cognitive limitations, bounded rationality, and social influences hinder us in noticing, processing, searching, evaluating, interpreting, comparing, remembering, retrieving, and communicating information.

I will address these challenges from the perspective of experiments from the field of judgment and decision making, and social psychology. In these fields we are now coping with the immense task of understanding and addressing the growing crisis of misinformation and the increasing prevalence and impact of AI tools. Yet, in our very own academic circles we have yet to come to a consensus as a scientific community in how to best assess the credibility of scientific evidence, in how we should conduct and evaluate research. A recent example is surrounding the so-called “science crisis” identifying challenges regarding the replicability, reproducibility, theory, measurement, and sharing of scientific evidence. This led to a science reform movement calling for major overhaul in the way that we do science and the need for an implementation of “open science”, routinely evaluating science using scientific principles using “meta science”, and moving towards collaborative large-scale “team science”.

I will provide a brief overview of my understanding of these challenges, my changes to my own science process, and the research that I do that aims to improve on and overcome some of these challenges.

我们是否知晓如何评估信息的可信度和科学证据？这既是我们在日常生活中，也是作为科学家在工作过程中所面临的一项常规任务。然而，近年来，我们越来越意识到自身的局限性——我们不确定如何以及并不擅长于评估可信度。这一现象具有深远的影响，从做出简单的决策，到应对诸如流行病和气候变化等紧迫的全球性挑战，无不受到影响。在信息处理的每一个环节，我们的认知局限性、有限理性以及社会影响都阻碍了我们对于信息的感知、处理、搜索、评估、解读、比较、记忆、检索和沟通。