Dataset for "From Prediction to Explanation: Temporal SHAP for Perceived Difficulty in Online Learning"

# Difficulty–Danmaku Dataset ## DescriptionThis dataset combines **teacher segment-level ratings** with **student bullet-screen comments (danmaku)** collected from Bilibili online courses, enabling research on instructional quality, cognitive dynamics, and perceived difficulty. Eight courses across multiple domains are included: `finance`, `freud_paragraphs`, `if_else_paragraphs`, `jane_eyre_paragraphs`, `literature`, `math`, `mayuan1`, and `physics`. - **Teacher ratings (CSV):** Each course segment is independently rated by four teachers. Files follow the naming pattern `<course>.csv`, `<course>2.csv`, `<course>3.csv`, `<course>4.csv`. - **Danmaku annotations (XLSX):** Bullet-screen comments are annotated with interaction type, emotion target, fine-grained emotion, cognitive level, and perceived difficulty. - **Derived outputs:** Running the accompanying code produces SHAP-based time-series explanations (`shap_timeseries*.csv`) that quantify the temporal contribution of instructional features. - **Reliability summaries:** Files such as `kappa_overall_summary.xlsx` report inter-rater agreement. ## File structure- **CSV (teacher ratings):** Segment boundaries, transcript snippets, and 10 instructional feature ratings (e.g., explaining skill, interaction skill, feedback timeliness, clarity, difficulty). - **XLSX (danmaku annotations):** Timestamped comments annotated along interaction, emotion, cognition, and difficulty dimensions. - **Generated SHAP files:** Feature-level attributions for model interpretability. ## Use casesThis dataset supports:- Analysis of **instructional features and student perception** in online learning. - Research on **temporal dynamics** of cognition, emotion, and perceived difficulty. - Benchmarking of **multi-task learning and explainable AI** methods in educational contexts. ## Ethics & ComplianceAll data have been de-identified and anonymized. Bullet-screen comments contain no personal identifiers. Data use complies with Bilibili’s non-commercial academic use policy. This dataset is provided strictly for **academic, non-commercial research**. ## LicenseDataset released under **CC BY 4.0**. Please cite if used. ## CitationPlease cite as: *Authors*. **Difficulty–Danmaku Dataset** (Version 1.0). Zenodo. DOI: [to be added after upload]