Mitigating Biases in Collective Decision-Making: Enhancing Performance in the Face of Fake News

Data supporting "Mitigating Biases in Collective Decision-Making: Enhancing Performance in the Face of Fake News". If you use this dataset in your own research, please cite this paper: ```@misc{abels2024mitigating, title={Mitigating Biases in Collective Decision-Making: Enhancing Performance in the Face of Fake News}, author={Axel Abels and Elias Fernandez Domingos and Ann Nowé and Tom Lenaerts}, year={2024}, eprint={2403.08829}, archivePrefix={arXiv}, primaryClass={cs.HC}}``` column name description treatment identifier for the set of headlines presented to the participant trial trial/round in which the headline was presented arm which "arm" the headline was presented as (0=left, 1=middle, 2=right) advice the participant's response (0=very unlikely, 0.25=unlikely, 0.5=undecided, 0.75=likely, 1=very likely) genuine whether the headline was genuine (1) or altered (0) headline the headline as shown to the participant original the headline before a possible alteration expert_id participant's identifier sentiment whether the headline reported a negative (-1) or positive (1) outcome expert:ethnicity the participant's ethnicity expert:sex the participant's sex expert:age the participant's age outcome:white, outcome:black, outcome:young, outcome:old, outcome:male, outcome:female whether the headline reported a negative (-1) or positive (1) or neutral (0) outcome for the specified group trial_time how long the participant took to respond to the trial/round abstractIndividual and social biases undermine the effectiveness of human advisers by inducing judgment errors which can disadvantage protected groups. In this paper, we study the influence these biases can have in the pervasive problem of fake news by evaluating human participants' capacity to identify false headlines. By focusing on headlines involving sensitive characteristics, we gather a comprehensive dataset to explore how human responses are shaped by their biases. Our analysis reveals recurring individual biases and their permeation into collective decisions. We show that demographic factors, headline categories, and the manner in which information is presented significantly influence errors in human judgment. We then use our collected data as a benchmark problem on which we evaluate the efficacy of adaptive aggregation algorithms. In addition to their improved accuracy, our results highlight the interactions between the emergence of collective intelligence and the mitigation of participant biases.