Raw dataset for: Manipulating Evaluation Bias in Reasoning LLMs for Idea Evaluation: Approaching Human Criteria

This dataset accompanies the study “Manipulating Evaluation Bias in Reasoning LLMs for Idea Evaluation: Approaching Human Criteria” (Furukawa, 2025).

It provides the raw data, processed results, and supplementary materials used to analyze and align evaluation biases between human participants and a reasoning large language model (GPT-5). The dataset is released to support reproducibility and transparency of the reported analyses.

The materials include:- Raw evaluation data from human participants- LMM-adjusted human evaluation results and estimated bias values- Raw GPT-5 evaluation results under systematically manipulated bias prompt patterns- LMM-adjusted GPT-5 evaluation scores- Derived similarity and agreement metrics comparing human and GPT-5 evaluations (correlation, MAE, RMSE, with confidence intervals)- Results of statistical analyses, including ART-ANOVA, Tukey post-hoc tests, and estimated marginal means- Experimental materials and metadata, including evaluated ideas (Japanese text), prompt templates, and raw GPT-5 output files- README.md providing detailed descriptions of all files and variables

All human evaluation data are pseudonymized, and no personally identifiable information is included. Ethical approval and informed consent procedures are described in the associated paper.