Mitigating Biases in Collective Decision-Making: Enhancing Performance in the Face of Fake News

Data supporting "Mitigating Biases in Collective Decision-Making: Enhancing Performance in the Face of Fake News". 

 If you use this dataset in your own research, please cite this paper:

```@misc{abels2024mitigating,      title={Mitigating Biases in Collective Decision-Making: Enhancing Performance in the Face of Fake News},       author={Axel Abels and Elias Fernandez Domingos and Ann Nowé and Tom Lenaerts},      year={2024},      eprint={2403.08829},      archivePrefix={arXiv},      primaryClass={cs.HC}}```

column name
description

treatment
identifier for the set of headlines presented to the participant

trial
trial/round in which the headline was presented 

arm
which "arm" the headline was presented as (0=left, 1=middle, 2=right)

advice
the participant's response (0=very unlikely, 0.25=unlikely, 0.5=undecided, 0.75=likely, 1=very likely)

genuine
whether the headline was genuine (1) or altered (0)

headline
the headline as shown to the participant

original
the headline before a possible alteration

expert_id
participant's identifier

sentiment
whether the headline reported a negative (-1) or positive (1) outcome

expert:ethnicity
the participant's ethnicity

expert:sex
the participant's sex

expert:age
the participant's age

outcome:white, outcome:black, outcome:young, outcome:old, outcome:male, outcome:female
whether the headline reported a negative (-1) or positive (1) or neutral (0) outcome for the specified group

trial_time
how long the participant took to respond to the trial/round

abstractIndividual and social biases undermine the effectiveness of human advisers by inducing judgment errors which can disadvantage protected groups. In this paper, we study the influence these biases can have in the pervasive problem of fake news by evaluating human participants' capacity to identify false headlines. By focusing on headlines involving sensitive characteristics, we gather a comprehensive dataset to explore how human responses are shaped by their biases. Our analysis reveals recurring individual biases and their permeation into collective decisions. We show that demographic factors, headline categories, and the manner in which information is presented significantly influence errors in human judgment. We then use our collected data as a benchmark problem on which we evaluate the efficacy of adaptive aggregation algorithms. In addition to their improved accuracy, our results highlight the interactions between the emergence of collective intelligence and the mitigation of participant biases.