On the Effectiveness of Text and Image Embeddings in Multimodal Hate Speech Detection

Additional resources for the paper:

On the Effectiveness of Text and Image Embeddings in Multimodal Hate Speech Detection.

Lewis, N., Cavalcante, C. C., Boukouvalas, Z., & Corizzo, R.

2024 IEEE International Conference on Big Data (BigData) (pp. 3277-3281). IEEE.

MMHS150K [1] is a manually labeled multimodal dataset that contains $150000$ tweets with two modalities: text, and  corresponding image. Tweets are collected from September 2018 until February 2019 and are labeled according to different types of hate speech: no attacks to any community, racist, sexist, homophobic, religion-based attacks, or attacks to other communities. 

We extract vector embeddings leveraging different text (BERT, OpenAI) and image (ResNet, PVT, ViT) modele backbones and assess their effectiveness in the hate speech detection task.

Citation:

@inproceedings{lewis2024effectiveness,
title={On the Effectiveness of Text and Image Embeddings in Multimodal Hate Speech Detection},
author={Lewis, Nora and Cavalcante, Charles C and Boukouvalas, Zois and Corizzo, Roberto},
booktitle={2024 IEEE International Conference on Big Data (BigData)},
pages={3277--3281},
year={2024},
organization={IEEE}
}