Can state test scores be used to provide usable feedback to professional learning communities? a formative evaluation of the Comprehensive Accountability Profile

The Comprehensive Accountability Profile (CAP) is an annual measurement of subject matter teacher team performance in California schools; it is derived from publicly available California Standardized Test (CST) data. CAP is presented using two different scores: a Success Score, which is the percentage of students who score Basic, Proficient, or Advanced on the CST in a given year, and an Adjusted Score, which adjusts a department’s Success Score for 16 non‐malleable similar school characteristics. The purpose of this formative evaluation was to determine teacher response to and perceived utility of their respective CAP data, particularly as it pertains to department collaboration. Data were collected in three stages in order to complete this study. Stage 1 involved administering a survey on teacher beliefs, perceived utility, and value for the use of student standardized test score data in teacher evaluation models; this survey was administered to an entire teaching staff of a public charter high school in California. Stage 2 required interviewing seven teachers from this group. During the interviews, teachers were introduced to their respective CAP data, and responses were recorded. Stage 3 required that the interviewed teachers complete a follow‐up survey about the CAP model. Findings indicated that 88% of respondents agreed that CAP would be a tool they would use during department collaboration, and 86% believed they could use CAP data to facilitate goal‐setting in department meetings; however, 57% disagreed to its use in their summative performance evaluations. Based on the generated data, a communication protocol for introducing CAP data to new school sites was developed. Being a formative evaluation, strengths and weaknesses of the CAP model were identified, but results were inconclusive as to whether or not the CAP model changes teacher beliefs about the use of student standardized test score data in PLC performance evaluation.

综合问责概况（Comprehensive Accountability Profile，CAP）是一项针对加州学校学科教师团队绩效的年度测评工具，其数据源自公开可得的加州标准化考试（California Standardized Test，CST）结果。CAP通过两类分数呈现：一是达标分数（Success Score），即某一学年内加州标准化考试成绩达到基础级、熟练级或高级的学生占比；二是调整后分数（Adjusted Score），该分数基于16项不可变更的同类学校特征对院系达标分数进行校准。
本项形成性评价的目标在于探究教师对各自所属CAP数据的反馈与感知效用，尤其是其在院系协作场景中的应用情况。
本研究通过三个阶段收集数据：第一阶段为发放调查问卷，内容围绕教师对在教师评价模型中使用学生标准化考试分数数据的看法、感知效用与价值；本次调查面向加州某公立特许高中的全体教职员工开展。第二阶段为从该群体中选取7名教师进行访谈，访谈期间研究人员向教师展示其所属院系的CAP数据，并记录教师的反馈。第三阶段要求参与访谈的教师完成关于CAP模型的后续调查问卷。
研究结果显示，88%的受访者认可CAP可作为院系协作期间的使用工具，86%的受访者认为可借助CAP数据在院系会议中推进目标设定；但有57%的受访者反对将CAP用于教师终结性绩效评价。基于本次研究产生的数据，本研究开发了一套面向新校区的CAP数据介绍沟通方案。
作为一项形成性评价，本研究明确了CAP模型的优势与局限，但目前尚无定论，CAP模型是否会改变教师对在专业学习共同体（Professional Learning Community，PLC）绩效评价中使用学生标准化考试分数数据的看法。