fdata-03-00003-i0001_The Datafication of Hate: Expectations and Challenges in Automated Hate Speech Monitoring.tif

Hate speech has been identified as a pressing problem in society and several automated approaches have been designed to detect and prevent it. This paper reports and reflects upon an action research setting consisting of multi-organizational collaboration conducted during Finnish municipal elections in 2017, wherein a technical infrastructure was designed to automatically monitor candidates' social media updates for hate speech. The setting allowed us to engage in a 2-fold investigation. First, the collaboration offered a unique view for exploring how hate speech emerges as a technical problem. The project developed an adequately well-working algorithmic solution using supervised machine learning. We tested the performance of various feature extraction and machine learning methods and ended up using a combination of Bag-of-Words feature extraction with Support-Vector Machines. However, an automated approach required heavy simplification, such as using rudimentary scales for classifying hate speech and a reliance on word-based approaches, while in reality hate speech is a linguistic and social phenomenon with various tones and forms. Second, the action-research-oriented setting allowed us to observe affective responses, such as the hopes, dreams, and fears related to machine learning technology. Based on participatory observations, project artifacts and documents, interviews with project participants, and online reactions to the detection project, we identified participants' aspirations for effective automation as well as the level of neutrality and objectivity introduced by an algorithmic system. However, the participants expressed more critical views toward the system after the monitoring process. Our findings highlight how the powerful expectations related to technology can easily end up dominating a project dealing with a contested, topical social issue. We conclude by discussing the problematic aspects of datafying hate and suggesting some practical implications for hate speech recognition.

仇恨言论已被识别为社会中一个紧迫的问题，并且已经设计出多种自动化方法以检测和预防之。本文报告并反思了2017年芬兰市政选举期间多机构协作进行的一项行动研究设置。在此设置中，设计了一套技术基础设施，以自动监控候选人的社交媒体更新中的仇恨言论。该设置为我们提供了一个独特的视角，用以探究仇恨言论如何作为一个技术问题浮现。项目开发了一种运行良好的算法解决方案，利用监督机器学习进行测试。我们测试了各种特征提取和机器学习方法，最终采用了一种结合词袋模型特征提取与支持向量机的组合。然而，自动化方法需要大量的简化，例如使用基础尺度对仇恨言论进行分类，以及依赖于基于单词的方法，而实际上，仇恨言论是一种具有多种语气和形式的语言和社会现象。其次，以行动研究为导向的设置使我们能够观察情感反应，如与机器学习技术相关的希望、梦想和恐惧。基于参与性观察、项目成果和文件、项目参与者的访谈以及针对检测项目的在线反应，我们识别出了参与者对有效自动化技术的期望，以及算法系统引入的中立性和客观性水平。然而，在监控过程之后，参与者对该系统表达了更为批判的观点。我们的发现突显了与科技相关的强大期望如何轻易地主导一个处理有争议的、时事性社会问题的项目。我们最后讨论了将仇恨数据化的问题，并提出了仇恨言论识别的一些实际应用建议。