Contextual Changes, Credible Conclusions? A Direct and Conceptual Replication of Shen et al.'s (2019) Study on Online Image Credibility

Alarmingly low replication rates have been highlighted across disciplines. In communication science, rates are unknown but replications have some tradition. However, due to the field’s need to adapt to rapidly changing research contexts, replications have largely been conceptual rather than direct. We conduct two in-depth replications to systematically evaluate both forms. For that, we consider online information credibility as rapidly changing research context, serving as signpost for people to regulate cognitive capacities and navigate available information. In one study, Shen and colleagues examined US participants’ credibility of manually fabricated news images. We assess the robustness and generalizability of their findings through a direct (identical stimuli, US sample) and a conceptual (AI-generated fake images, German sample) replication, successfully replicating 71% of the original study’s empirical claims through our direct and 86% through our conceptual replication. Our reflection echoes recent calls for more open science and dedicated data editors.