Leveraging LLM-Respondents for Item Evaluation: a Psychometric Analysis

This dataset contains 1,165 rows, each corresponding to a respondent (including LLM-generated respondents) in our study. It contains 21 columns. The first column, "Generating Model," specifies the model or source (e.g., "Human") that generated the responses. The remaining 20 columns (Q1 to Q20) indicate the correctness of answers to 20 college algebra questions for each respondent. <b>"TRUE"</b> means the respondent answered correctly, <b>"FALSE"</b> indicates an incorrect answer, and <b>N/A</b> represents missing data (i.e., no response). The dataset includes responses from seven different generating models:<b>Human</b>: 265 responses<b>GPT-4</b>: 150 responses<b>GPT-3.5</b>: 150 responses<b>Llama 3</b>: 150 responses<b>Llama 2</b>: 150 responses<b>Gemini</b>: 150 responses<b>Cohere</b>: 150 responses