LLM-Assisted Content Analysis (LACA): Coded data and model reasons

Description This resources consists of calibrations sets (N=100) for 4 publically available datasets using the LLM-Assisted Content Analysis method (LACA). Each dataset contains the following columns: <br> <strong>text_id: </strong> Unique ID for each text document <strong>code_id: </strong> Unique ID for each code category <strong>text: </strong> Document text that's been coded <strong>original_code:</strong> Coded response from the original datasets <strong>replicated_code:</strong> Coded response from independent coding exercise from our study team <strong>model_code:</strong> Coded response generated from the LLM (GPT-3.5-turbo) <strong>reason:</strong> LLM generated reason for coding decision <br> Additional details on methods and definitions of individual code categories are available in the following paper: <br> Chew, R., Bollenbacher, J., Speer, J., Wenger., M, Kim., A. (2023) LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding. <br> Trump Tweets <strong>Citation</strong>: Coe, Kevin, Berger, Julia, Blumling, Allison , Brooks, Katelyn , Giorgi, Elizabeth , Jackson, Jennifer , … Wellman, Mariah . Quantitative Content Analysis of Donald Trump’s Twitter, 2017-2019. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2020-04-01. https://doi.org/10.3886/E118603V1 <strong>Source</strong>: https://www.openicpsr.org/openicpsr/project/118603/version/V1/view <br> BBC News <strong>Citation</strong>: Greene, D., &amp; Cunningham, P. (2006). Practical solutions to the problem of diagonal dominance in kernel document clustering. In <em>Proceedings of the 23rd international conference on Machine learning</em> (pp. 377-384). <strong>Source</strong>: https://www.kaggle.com/datasets/shivamkushwaha/bbc-full-text-document-classification <br> Contrarian Claims <strong>Citation</strong>: Coan, T. G., Boussalis, C., Cook, J., &amp; Nanko, M. O. (2021). Computer-assisted classification of contrarian claims about climate change. Scientific reports, 11(1), 22320. <strong>Source</strong>: https://socialanalytics.ex.ac.uk/cards/data.zip <br> Ukraine Water Problems <strong>Citation</strong>: Afanasyev S, N. B, Bodnarchuk T, S. V, M. V, T. V, Yu V, K. G, V. D, Konovalenko O, O. K, E. K, Lietytska O, O. L, V. M, Marushevska O, Mokin V, K. M, Osadcha N, O. I (2013) River Basin Management Plan for Pivdenny Bug: river basin analysis and measures <strong>Source</strong>: https://www.kaggle.com/datasets/vbmokin/nlp-reports-news-classification