From Argument to Algorithm: L2 Teachers’ cognitive bootstrapping in Validity Argument in Writing Assessment

Efficient focus on potential loss in the validity argument of writing assessment, such as writing subjectivity, content coverage, criteria vagueness, and raters' incompetence, has been found to positively develop teachers' overall writing assessment competence (AC). In this study, we propose a computational bootstrapping model of validity argument in L2 writing assessment, comparing it to argument-based models such as Practical Reasoning (PR), Assessment Use Argument (AUA), and Rubric Use Argument (RUA). Specifically, this computational model gradually enhances the validity argument by capturing subtle deficiencies from previous assessment competence and constructing a bootstrapping of the validity argument. We collected data from the Chinese English Teachers' Writing Assessment Competence Corpus (CETWACC), which includes texts from a total of Chinese L2 teachers in higher education. The corpus includes six levels and 60 items of information on how these teachers perform: (i) construction, (ii) reflection, (iii) externalization, (iv) internalization, (v) enhancement, and (vi) reconstruction. The findings indicate that the cognitive bootstrapping model (CBM) model leads to a higher enhancement of teachers' assessment competence with reasoning argument and impacts the more scientific measures of validity argument with computing algorithm. Overall, this study focuses on the computational evidence of validity argument and delves into the subtle process of micro changes in L2 writing assessment, transitioning from the argument to the algorithm. The results of the study have implications for discussions about the roles of validity argument bootstrapping in present writing assessment, providing a universally applicable and operationally feasible model for its validity argument.