Online Credit Recovery Study: Effects on High School Students' Proximal and Distal Outcomes

The American Institutes for Research conducted a multisite randomized study that tested an online learning model for
credit recovery at 24 high schools in Los Angeles, California in 2018 and 2019.
The study focused on first-year high school students who failed Algebra 1 or
English 9 (their ninth-grade English course) and retook the course during the
summer before their second year of high school. Within each participating
school, we used a lottery to determine whether each student was placed in either
the school’s typical teacher-directed class (business-as-usual control condition) or a
class that used an online learning model (treatment condition). For the online
learning model, an online provider supplied the main course content, and the
school provided a subject-appropriate, credentialed in-class teacher who could
supplement the digital content with additional instruction.<br><br>The study compared outcomes of students assigned to the treatment condition to outcomes of students assigned to the
control condition. Analyses focused both on proximal outcomes (ex: student course
experiences, content knowledge, and credit recovery rates) and distal outcomes
(ex: on-time graduation and cumulative credits earned by the end of the 4th year of
high school). We estimated average treatment effects for the intent-to-treat
sample using regression models that control for student characteristics and
randomization blocks. We conducted separate analyses for students who failed
Algebra 1 and students who failed at least one semester of their English 9
course.<br><br>This ICPSR data deposit includes our final analytical dataset and three supplemental files. Data come from three sources: (1) extant district data on
student information and academic outcomes, (2) end-of-course surveys of
students’ and teachers’ experiences, and (3) end-of-course test of students’
content knowledge. Data fields include:<br>Sample information: term, school (anonymized), teacher (anonymized), course, randomization block, student cohort, treatment statusDemographics: sex, race/ethnicity, National School Lunch Program status, inclusion in the Gifted/Talented program, Special Education status, and English language learner statusPre-treatment information (treatment group only): 9th grade GPA, 9th grade attendance rate, number of 9th grade courses failed, 8th grade test scoresOnline course engagement information: percentage of online course completed, average score on online activities, minutes spent in online platformStudent survey data: responses a survey administered at the end of the course for treatment and control students. Questions cover degree of student engagement with the course, perceptions of teacher support and course difficulty, and clarity of course expectations.End-of-course test data: answers and scores on an end-of-course assessment administered to treatment and control students to evaluate content knowledge (Algebra 1 or English 9). The test did not count towards the final course grade and included 17-20 multiple choice questions.Academic outcomes: grade in credit recovery course, credits attempted/earned in each year of high school, GPA in each year of high school, credits/GPA in math and ELA in each year of high school, indicator for on-time high school graduation, 10th grade PSAT scoresTeacher survey and logs: teacher-reported logs on the use of different instructional activities and responses to surveys about course pacing, content, goals, and degree of student support