University Classroom Teaching Quality Evaluation via Domain Pretraining and Reinforcement Learning

Overview This repository contains the implementation of a novel framework for University Classroom Teaching Quality Evaluation based on Large Language Models (LLMs), Domain-Specific Pretraining, and Reinforcement Learning. Unlike traditional teaching evaluation methods that rely on questionnaires, expert observation, or final examination scores, this project analyzes teacher–student classroom discourse to provide interpretable and evidence-based assessments of instructional quality. The proposed framework first learns classroom interaction structures through domain pretraining and then optimizes structured teaching-quality predictions using reinforcement learning with customized reward functions. Features 📚 Domain-specific language model pretraining for classroom discourse 🗣️ Dialogue Structure Modeling (DSM) 🔍 Interaction-Oriented Discrimination (IOD) 🎯 Reinforcement Learning optimization using GRPO 📈 Structured teaching quality assessment 💡 Interpretable rationale generation 👨‍🏫 Teacher–student interaction analysis 📊 Robust discourse move prediction Framework The proposed method consists of two stages: Stage 1: Domain Pretraining The model is adapted to educational dialogue using: Dialogue Structure Modeling Interaction-Oriented Discrimination The objective is to learn: Speaker-role relationships Turn-level dependencies Question–response–feedback chains Teacher and student discourse moves Interaction consistency Stage 2: Reinforcement Learning After supervised fine-tuning, the model is optimized with Group Relative Policy Optimization (GRPO) using three reward functions: Move Accuracy Reward Interaction Quality Reward Robustness Constraint Reward This stage improves: discourse prediction accuracy rationale quality output consistency robustness under context perturbations Task Definition Input The model receives: Classroom transcript Speaker roles Target utterance Dialogue context Optional metadata Output The model predicts: Teacher Move Student Move Dialogic Move Interaction Quality Natural Language Rationale Datasets The framework is designed using two public educational datasets: Dataset Purpose TalkMoves Domain pretraining NCTE Transcripts Supervised fine-tuning and Reinforcement Learning Model The default backbone is Qwen2.5-7B-Instruct Other supported baselines include LLaMA-3.1-8B-Instruct DeepSeek-R1-Distill-Qwen-7B InternLM2.5-7B-Chat Baichuan2-7B-Chat Yi-1.5-9B-Chat Repository Structure ├── data/ │ ├── TalkMoves/ │ └── NCTE/ │ ├── models/ │ ├── backbone/ │ ├── pretraining/ │ └── rl/ │ ├── training/ │ ├── pretrain.py │ ├── sft.py │ └── grpo.py │ ├── evaluation/ │ ├── metrics.py │ └── inference.py │ ├── configs/ ├── scripts/ ├── figures/ └── README.md Evaluation Metrics The framework is evaluated using: Dialogue Structure Modeling (DSM) Move-F1 Interaction Quality Score (IQS) Hard-F1 Generalization Gap Response Stability Applications This project can be applied to University teaching quality evaluation Classroom discourse analysis Educational NLP Teacher professional development Learning analytics Intelligent classroom observation systems Citation If you use this work in your research, please cite: @article{zhao2025classroom, title={University Classroom Teaching Quality Evaluation via Domain Pretraining and Reinforcement Learning}, author={Benhong Zhao}, journal={Under Review}, year={2025} }