Twitter corpus of Resource-Scarce Languages for Sentiment Analysis and Multilingual Emoji Prediction

This dataset is created by leveraging the social media platforms such as twitter for developing corpus across multiple languages. The corpus creation methodology is applicable for resource-scarce languages provided the speakers of that particular language are active users on social media platforms. We present an approach to extract social media microblogs such as tweets (Twitter). We created corpus for multilingual sentiment analysis and emoji prediction in Hindi, Bengali and Telugu. Further, we perform and analyze multiple NLP tasks utilizing the corpus to get interesting observations.

本数据集依托Twitter等社交媒体平台采集构建，旨在生成多语言语料库。该语料库构建方法适用于低资源语言，前提是该语言的母语使用者为社交媒体平台的活跃用户。我们提出了一种提取Twitter等社交媒体平台推文内容的方法。我们针对印地语、孟加拉语与泰卢固语，构建了用于多语言情感分析及表情符号预测的语料库。此外，我们利用该语料库开展并分析了多项自然语言处理（Natural Language Processing, NLP）任务，得到了若干值得关注的观测结果。