CMDSA-24

A code-mixed dataset (CMDSA-24) has been created for sentiment analysis in Roman Urdu, Roman Punjabi, and English. This dataset may be served as a benchmark for future researchers to evaluate sentiment analysis in low-resource, code-mixed languages. Data collection was conducted from diverse online social media platforms, including YouTube and WhatsApp, to ensure a varied dataset. Specifically, data from popular YouTube channels was selected for scraping using the YouTube V3 API. We developed a web scraping script to collect comments associated with each music post. For every post, we extracted the unique video ID, which was then input into the script to retrieve the corresponding comments. In addition to YouTube, we manually gathered WhatsApp message data from various users to create a corpus of code-mixed text based on SMS communications.  

为开展罗马乌尔都语、罗马旁遮普语与英语的情感分析任务，我们构建了语码混合数据集（code-mixed dataset）CMDSA-24。该数据集可作为基准数据集，供后续研究者开展低资源语码混合语言的情感分析评估工作。为保障数据集的多样性，我们从YouTube、WhatsApp等多类主流在线社交媒体平台采集数据。具体而言，我们依托YouTube V3 API对热门YouTube频道的相关数据进行爬取。我们开发了网络爬虫脚本，用于采集每条音乐帖文对应的评论数据。针对每条帖文，我们提取其唯一的视频ID，并将该ID输入爬虫脚本以获取对应评论内容。除YouTube平台外，我们还通过人工方式从多位用户处收集WhatsApp消息数据，以此构建基于短信通信的语码混合文本语料库。