"I Thought It Was a Real Person": Distinguishing AI-Generated from Human-Written Comments, Evaluative Bias, and Trust Concerns on Social Media

# Dataset: Distinguishing AI-Generated from Human-Written Social Media Comments This repository contains the **anonymized quantitative data** for the study: > *"I Thought It Was a Real Person": Distinguishing AI-Generated from Human-Written> Comments, Evaluative Bias, and Trust Concerns on Social Media.* Submitted to the *International Journal of Human–Computer Interaction* (IJHCI),Special Issue "Trust and Mistrust in Artificial Intelligence." ## Study summary A blind within-subjects experiment (N = 60) tested whether social media users candistinguish AI-generated from human-written comments on Weibo. Each participant judgeda balanced set of comments (half AI-generated, half human-written). For every commentthey (a) decided whether it was AI-generated and (b) rated it on five evaluativedimensions using 5-point scales. Comments spanned multiple topic contexts. ## Files | File | Level | Rows | Description ||------|-------|------|-------------|| `participants.csv` | one row per participant | 60 | Demographics and total response time || `responses.csv` | one row per participant × comment | 3,120 | AI-detection responses and five evaluative ratings || `codebook.csv` | — | — | Variable dictionary for all fields | ## Privacy / ethics This package contains **only de-identified quantitative data**. No names, free-textresponses, exact timestamps, IP addresses, contact details, or interview transcriptsare included. Participants are referenced solely by an arbitrary numeric ID. Thequalitative interview data described in the manuscript are **not** shared in order toprotect participant confidentiality and are available from the corresponding authorupon reasonable request, subject to ethical approval. ## Key variables (see `codebook.csv` for full detail) - `true_source` — ground truth of the comment: `AI` or `human`.- `detection_choice` — participant's 1–5 judgment (lower = "more likely AI", higher = "more likely human").- `correct` — whether the participant's AI/human judgment matched the ground truth (`1` = correct, `0` = incorrect, `0.5` = midpoint/undecided response).- Five evaluative ratings, each on a 1–5 scale (higher = more positive): `readability`, `fluency`, `emotional_authenticity`, `innovativeness`, `aesthetics`. ## Notes on coding - Ratings and detection responses are on 5-point scales as administered.- `item_label` (e.g., `1-3`) is an internal stimulus identifier (topic group – item index); the underlying comment text and images are not redistributed.- The reported overall correct-recognition rate (~46.6%) can be reproduced by averaging the `correct` column. ## License Recommended: Creative Commons Attribution 4.0 (CC BY 4.0). Update if your repositoryor institution requires a different license. ## Citation Please cite the associated article (full reference to be added upon publication).