From GPT-4 to GPT-5 The Impact of LLM Generation on Spanish Chat Interactions

This repository contains the data, configurations, and statistical analysis outputs for the empirical study on how different LLM generations impact human-AI interaction in foreign language learning # 01_Technical_SetupThis folder contains the technical configurations for the Conversational Agents exported as JSON files from the Flowise platform.- System Prompts: Full metaprompt instructions for the personas Carla, Samir, and Ramirez.- Model Parameters: API settings including specific version bindings for the research.- Model Versions: Configurations for OpenAI gpt-4-0613, gpt-4-turbo-2024-04-09, and gpt-5-2025-08-07. # 02a_Raw_Chatlogs.xlsxThis file contains the anonymized chat corpus used for linguistic analysis.* Scope: 159 conversations with a total of 2,007 messages.* Data Privacy: All participant names and identifying details have been pseudonymized. # 02b_User_Ratings.xlsxThis file contains the subjective evaluations provided by the participants.* Sample Size: 144 complete evaluations from 48 participants.* Scale: Scores range from 1 (very good) to 5 (very bad). # 03_Analysis_OutputsThis folder contains the results of the Mixed Model Analysis (MLA) exported from SPSS.* Formal Metrics (RQ1): Frequency analysis for emojis, exclamation marks, and question marks.* Behavioral Engagement (RQ2): Analysis of conversation duration, response latency, and character counts.* Perceived Quality (RQ3): Statistical comparisons of user scores across model generations.Files: - conv_duration.pdf = coun_characters_message_LLM.pdf - count_characters_message_user.pdf - count_emoji_LLM.pdf - count_emoji_user.pdf - count_exclamation_marks_LLM.pdf - count_exclamation_marks_user.pdf - count_question_marks_LLM.pdf - count_question_marks_user.pdf - rating_result.pdf ## Study Methodology * Framework: Task-Based Language Teaching (TBLT) using an "opinion-gap" negotiation scenario.* Design: Within-subject design where each participant interacted with all three LLM versions.* Population: Students from German secondary schools (8th & 12th grade) and university-level learners (B1/B2). ## License and CitationThese data are provided for replication and transparency purposes in Open Science.* Citation: Please cite the original paper: "Fahnroth, Felix (2027) "From GPT-4 to GPT-5: The Impact of LLM Generation on Spanish Chat Interactions."本仓库包含针对「不同大语言模型（LLM）代际对外语学习中人机交互的影响」开展的实证研究所需的数据、配置文件及统计分析结果。 # 01_Technical_Setup 本文件夹包含从Flowise平台导出的对话智能体（Conversational Agents）技术配置JSON文件。 - 系统提示（System Prompts）：面向Carla、Samir与Ramirez三个角色的完整元提示指令。 - 模型参数（Model Parameters）：包含本次研究所用模型版本绑定信息的API设置。 - 模型版本（Model Versions）：针对OpenAI gpt-4-0613、gpt-4-turbo-2024-04-09及gpt-5-2025-08-07的配置文件。 # 02a_Raw_Chatlogs.xlsx 本文件包含用于语言学分析的匿名化聊天语料库。 * 覆盖范围：共计159段对话，含2007条消息。 * 数据隐私：所有参与者姓名及身份识别信息均已完成假名化处理。 # 02b_User_Ratings.xlsx 本文件包含参与者提供的主观评估结果。 * 样本量：来自48名参与者的144份完整有效评估。 * 评分量表：分值区间为1分（极佳）至5分（极差）。 # 03_Analysis_Outputs 本文件夹包含从SPSS导出的混合效应模型分析（Mixed Model Analysis, MLA）结果。 * 核心指标（研究问题1，RQ1）：针对表情符号、感叹号及问号的频次分析。 * 行为参与度（研究问题2，RQ2）：针对对话时长、响应延迟及字符数的分析。 * 感知质量（研究问题3，RQ3）：不同大语言模型代际下用户评分的统计学对比。 所含文件： - conv_duration.pdf = coun_characters_message_LLM.pdf - count_characters_message_user.pdf - count_emoji_LLM.pdf - count_emoji_user.pdf - count_exclamation_marks_LLM.pdf - count_exclamation_marks_user.pdf - count_question_marks_LLM.pdf - count_question_marks_user.pdf - rating_result.pdf ## 研究方法论 * 理论框架：采用「意见差异型」协商场景的任务型语言教学（Task-Based Language Teaching, TBLT）框架。 * 实验设计：被试内设计，每名参与者均与全部三款大语言模型版本进行交互。 * 研究对象：德国中学八年级与十二年级学生，以及CEFR B1/B2级别的高校学习者。 ## 许可与引用说明 本数据集用于开放科学（Open Science）框架下的研究复现与成果透明化。 * 引用要求：请引用原论文：Fahnroth, Felix (2027) 《从GPT-4到GPT-5：大语言模型代际对西班牙语聊天交互的影响》。