Athlete Training Action Quality Assessment via Domain Pretraining and Reinforcement Learning

Athlete Training Action Quality Assessment via Domain Pretraining and Reinforcement Learning Overview This repository presents a vision-language framework for Athlete Training Action Quality Assessment (AQA). Unlike conventional action recognition, which focuses on identifying action categories, our approach evaluates how well an athlete performs an action and generates structured coaching-oriented feedback. The proposed framework combines: Domain Pretraining on FineDiving for learning procedural sports knowledge Procedure Structure Modeling (PSM) for temporal action understanding Quality-Oriented Discrimination (QOD) for quality-sensitive representation learning Reinforcement Learning (RL) with reward-guided policy optimization Vision-Language Models (VLMs) for structured score prediction and coaching feedback generation The final model produces: Overall quality score Technical score Phase-level quality assessment Training-oriented feedback Framework Our method consists of two stages: Stage I: Domain Pretraining Using the FineDiving dataset, the model learns: Procedure Structure Modeling (PSM) Quality-Oriented Discrimination (QOD) This stage enables the VLM to understand: action-stage ordering temporal dependencies sport-specific execution patterns quality-sensitive representations Stage II: Reinforcement Learning Optimization The pretrained model is optimized on MTL-AQA through reward-guided policy learning. The reward function consists of: Score Reward Feedback Quality Reward Robustness Constraint Reward The optimized policy generates structured assessment reports suitable for athlete training. Features ✅ Procedure-aware Action Quality Assessment ✅ Vision-Language based Structured Evaluation ✅ Domain-specific Pretraining ✅ Reinforcement Learning Optimization ✅ Phase-level Technical Analysis ✅ Training-oriented Feedback Generation ✅ Robust Cross-domain Generalization Repository Structure . ├── configs/ # Training configurations ├── datasets/ # Dataset preprocessing ├── models/ # Vision-language models ├── pretraining/ # Domain pretraining ├── reinforcement_learning/ # RL optimization ├── evaluation/ # Evaluation scripts ├── utils/ ├── scripts/ ├── checkpoints/ └── README.md Supported Models The framework can be built upon multiple open-source vision-language models, including: Qwen2.5-VL InternVL LLaVA-OneVision MiniCPM-V DeepSeek-VL GLM-4.1V Datasets FineDiving Used for domain pretraining: Procedure annotations Temporal boundaries Sub-action labels Official scores MTL-AQA Used for: Supervised fine-tuning Reinforcement learning Final evaluation Method The framework formulates athlete training assessment as a structured prediction task. Input: Video + Optional Phase Information + Optional Reference Action Output: Overall Quality Score Technical Score Phase-level Assessment Training Feedback The total optimization objective combines: L = L_pretraining + L_RL where L_pretraining = Procedure Structure Modeling + Quality-Oriented Discrimination and RL optimization maximizes: Reward = Score Reward + Feedback Quality Reward + Robustness Reward Experimental Evaluation The proposed framework is evaluated on: Procedure Structure Modeling Score Prediction Feedback Quality Hard-case Robustness Cross-domain Generalization Response Stability Metrics include: Spearman Correlation PSM FBScore Hard-Spearman GenGap R-Stab Installation git clone https://github.com/yourname/Athlete-Training-AQA.git cd Athlete-Training-AQA conda create -n athlete_aqa python=3.10 conda activate athlete_aqa pip install -r requirements.txt Training Domain Pretraining bash scripts/pretrain.sh Reinforcement Learning bash scripts/train_rl.sh Evaluation bash scripts/evaluate.sh Citation If you find this project useful, please consider citing: @article{li2025athlete, title={Athlete Training Action Quality Assessment via Domain Pretraining and Reinforcement Learning}, author={Li, Yingjia and Zhang, Shufeng and Wang, Jialing}, journal={Frontiers}, year={2025} } Acknowledgements This work builds upon several outstanding open-source projects and datasets, including: FineDiving MTL-AQA Qwen2.5-VL InternVL LLaVA MiniCPM-V DeepSeek-VL We sincerely thank the authors for making their resources publicly available.