Validity and reliability evidence for the science diagram comprehension test

College science students are expected to learn from and with diagrams. Although a substantial body of research exists regarding efforts to improve students’ diagram comprehension, little attention has been paid to this comprehension ability as an individual difference affecting learning and performance in science courses. This lack of attention can be at least partially attributed to a lack of tools available to measure diagram comprehension ability. This paper introduces and evaluates a new measure, the Science Diagram Comprehension Test (SDCT), to address this need. The SDCT is a multiple-choice test, intended for undergraduates, that includes a variety of diagram types from a variety of introductory science courses. This paper presents evidence for the reliability and validity of SDCT scores across four phases. Scale-level descriptive statistics, correlations, IRT, and EFA demonstrate that the SDCT has acceptable psychometric properties, holds expected relationships with other variables (e.g. major and GPA), and has a single factor structure. Expert review confirms that all SDCT diagrams are similar to those used in college science courses. The final version of the SDCT contains 27 items corresponding to 14 different diagrams. Results indicate that the SDCT captures diagram comprehension ability and is appropriate for use in research.