"A Design-Theoretic Taxonomy for Deconstructing Engagement AI Audit Annotation Results Data Table"

"This dataset contains two complementary components for analyzing player engagement in interactive game systems: (1) a large-scale corpus of 14,717 Steam user reviews annotated with a custom 28-dimensional binary activation framework, and (2) a smaller reliability audit subset of 201 reviews (67 annotation tasks) designed to validate cross-model consistency.The smaller audit subset (N=201) consists of 67 targeted annotation tasks evaluated by three architecturally distinct large language models (Qwen-2.5-7B, Llama-3.1-8B, Gemma-2-9B) via a triangular consensus protocol. This component computes reliability metrics including Prevalence-Adjusted Bias-Adjusted Kappa (PABAK) and triple-match rates to validate the robustness of the automated coding scheme, ensuring the main corpus annotations reflect consistent, design-theoretic signals rather than model-specific artifacts."

本数据集包含两个互补组件，用于分析交互式游戏系统中的玩家参与度：(1) 规模为14717条的Steam用户评论文语料库，该语料库采用自定义的28维二元激活框架完成标注；(2) 包含201条评论的小型可靠性审核子集（共67项标注任务），用于验证跨模型一致性。该小型审核子集（样本量N=201）包含67项针对性标注任务，由3款架构迥异的大语言模型（Large Language Model，LLM）——Qwen-2.5-7B、Llama-3.1-8B、Gemma-2-9B——通过三角共识协议进行评估。本组件可计算包括患病率校正偏差校正Kappa值（Prevalence-Adjusted Bias-Adjusted Kappa，PABAK）与三重匹配率在内的可靠性指标，以验证自动编码方案的鲁棒性，确保主语料库的标注结果所反映的是符合设计理论的一致性信号，而非模型特异性伪影。