California under the microscope: an evaluation of the practical effect of different approaches to accountability

Restricted until 2 April 2009. The purpose of this study was to examine five practical approaches to educational accountability that are utilized or are within the current constraints of California's accountability system. Various approaches were compared and analyzed as potential comparative assessments for elementary schools that serve a large population of English language learners (ELL). The study specifically compared and analyzed the interpretability, reliability, fairness, and the validity of: (a) State decile ranks; (b) Similar-schools rank; (c) Similar-schools difference scores; (d) Academic Performance Index (API) for English learners; and (e) Residualized difference scores.; The population included elementary schools that had API Base and Growth data for any of the 2003-2005 school years with at least 130 students with valid test scores. For the API for English learners and the Residualized difference score approaches, the sample was inclusive of schools that had a separate ELL API in 2005 and 2006.; Results of the study found state decile ranks, which are utilized in California, to be strongly correlated to socioeconomic status and thus were of limited utility as measures of accountability for schools that have high a proportion of ELLs. Similar-schools rank, which is the state's recommended method in evaluating schools against those that are similar in demographic characteristics, was a more equitable method for evaluating schools that serve ELLs. An analysis of the API for English learners, which was recently added by the state, showed that it was a promising method for comparing schools with a large population of ELLs.; Two other comparative assessments within the current constraints of California's accountability system were examined (Similar-schools difference scores and Residualized difference scores). Results of the evaluation of comparative assessments suggest Residualized difference scores to be the best method in evaluating schools that serve a large number of ELLs.; This study only examined approaches to accountability that are currently being utilized or are feasible within the constraints of California's accountability system. Further research is still necessary to enhance the accuracy of school-level-achievement data especially in schools that serve a large population of English learners.