Impact of Chatbots on Chinese Speaking Skills Among International Students in China and Its Mechanism

This research conducted an 8-week randomized controlled trial to investigate the impact of the AI chatbot "Doubao" on Chinese oral proficiency among international students in China and explore its underlying mechanisms. The experimental group (30 participants) engaged in high-frequency interactive dialogue training using "Doubao," while the control group (30 participants) performed traditional oral exercises. Results showed that the experimental group achieved significant improvements in vocabulary (mean improvement of 3.3 points, p < 0.001) and meaningful expression (2.8 points, p < 0.001), with a total score increase of 12.2 points—substantially exceeding the control group's 3.4-point gain (p < 0.001). Large effect sizes were observed for vocabulary (Cohen’s d = 1.1) and meaningful expression (d = 1.22). ANCOVA analysis confirmed the experimental group's superiority in fluency and syntax (e.g., F = 36.42, partial η² = 0.39), though pronunciation improvement remained limited (η² = 0.06), likely due to speech recognition technology constraints.
Textual analysis revealed three core mechanisms: contextualized input (e.g., simulated shopping/medical scenarios enhancing vocabulary application), immediate feedback internalization (29.3% increase in grammatical error correction rates), and emotional incentives (68.4% learner-initiated topic rate). A case study of Student P23 demonstrated substantial growth in linguistic complexity (50.8% increase in complex sentence usage) and error correction rates (from 63.5% to 82.1%). The findings validate that AI chatbots address critical gaps in traditional pedagogy—such as authentic language contexts and personalized guidance—through high-frequency interactions and precision feedback, offering empirical support for digital transformation in international Chinese education. The study also highlights the need to optimize pronunciation training modules for comprehensive oral proficiency enhancement.