Replication Data For: Mapping quality judgment in International Relations: Cognitive dimensions and sociological correlates

Research quality assessment is a cornerstone of academic practice, yet the criteria
that inform such judgments are often assumed rather than critically examined through
empirical research. This article draws on a global survey of International Relations
scholars (N = 820) to analyze the cognitive dimensions underlying research quality
evaluation and their variation across sociological and epistemological factors. We
identify seven distinct quality factors: theoretical significance, logical style and
structure, practical significance, methodological rigor, contribution to future research,
interest and topicality, and challenge to existing knowledge. Our results indicate that,
while personal preferences, disciplinary norms, and professional practices—shaped by
variables such as gender, nationality, and political orientation—influence evaluations,
research quality judgments are ultimately grounded in shared cognitive frameworks.
This study offers robust evidence that quality assessments, though subject to
sociological variation, reflect deeper, common cognitive structures within the academic
community. In light of this, we also consider how these insights might affect scholars
positioned both at the margins and in positions of power.