Analyzing Mentions of Death in Covid-19 Tweets

Dataset preparation and annotation The dataset is a subset of the TBCOV dataset collected at QCRI filtered for mentions of personally related COVID-19 deaths. The filtering was done using regular expressions such as my * passed, my * died, my * succumbed & lost * battle. A sample of the dataset was annotated on Appen. Please see 'annotation-instructions.txt' for the full instructions provided to the annotators. Dataset description The "classifier_filtered_english.csv" file contains 33k deduplicated and classifier-filtered tweets (following X's content redistribution policy). for the 6 countries (Australia, Canada, India, Italy, United Kingdom, and United States) from March 2020 to March 2021 with classifier-labeled death labels, regular expression-filtered gender and relationship labels, and the user device label. The full 57k regex-filtered collection of tweets can be made available on special cases for Academics and Researchers. date: the date of the tweet country_name: the country name from Nominatim API tweet_id: the ID of the tweet url: the full URL of the tweet full_text: the full-text content of the tweet (also includes the URL of any media attached) does_the_tweet_refer_to_the_covidrelated_death_of_one_or_more_individuals_personally_known_to_the_tweets_author: the classifier predicted label for the death (also includes the original labels for the annotated samples) what_is_the_relationship_between_the_tweets_author_and_the_victim_mentioned: the annotated relationship labels relative_to_the_time_of_the_tweet_when_did_the_mentioned_death_occur: the annotated relative time labels user_is_verified: if the user is verified or not user_gender: the gender of the Twitter user (from the user profile) user_device: the Twitter client the user uses has_media: if the tweet has any attached media has_url: if the tweet text contains a URL matched_device: the device (Apple or Android) based on the Twitter client regex_gender: the gender inferred from regular expression-based filtering regex_relationship: the relationship label from regular expression-based filtering Inferring gender using regular expressions We first determine the mapping between different relationship labels mentioned in the tweet to the gender. We do not use any relationship like "cousin" from which we cannot easily infer the gender. Male relationships: 'father', 'dad', 'daddy', 'papa', 'pop', 'pa', 'son', 'brother', 'uncle', 'nephew', 'grandfather', 'grandpa', 'gramps', 'husband', 'boyfriend', 'fiancé', 'groom', 'partner', 'beau', 'friend', 'buddy', 'pal', 'mate', 'companion', 'boy', 'gentleman', 'man', 'father-in-law', 'brother-in-law', 'stepfather', 'stepbrother' Female relationships: 'mother', 'mom', 'mama', 'mum', 'ma', 'daughter', 'sister', 'aunt', 'niece', 'grandmother', 'grandma', 'granny', 'wife', 'girlfriend', 'fiancée', 'bride', 'partner', 'girl', 'lady', 'woman', 'miss', 'mother-in-law', 'sister-in-law', 'stepmother', 'stepsister' Based on these mappings, we used the following regex for each gender label to determine the gender of the deceased mentioned in the tweet. "[m|M]y\s(" + "|".join([r + "s?" for r in relationships]) + ")\s(died|succumbed|deceased)" Age groups from relationship labels First, we get the relationship labels using regex filtering, and then we group them into different age-group categories as shown in the following table. The UK and the US use different age groups because of the different age group definitions in the official data. Category Relationship (from tweets) Age Group (UK) Age Group (US) Grandparents grandfather, grandmother 65+ 65+ Parents father, mother, uncle, aunt 45-64 35-64 Siblings brother, sister, cousin 15-44 15-34 Children son, daughter, nephew, niece 0-14 0-14 Training the classifier The 'english-training.csv' file contains about 13k deduplicated human-annotated tweets. We use a random seed (42) to create the train/test split. The model Covid-Bert-V2 was fine-tuned on the training set for 2 epochs with the following hyperparameters (obtained using 10-fold CV): random_seed: 42, batch_size: 32, dropout: 0.1. We obtained a F1-score of 0.81 on the test set. We used about 5% (671) of the combined and deduplicated annotated tweets as the test set, about 2% (255) as the validation set, and the remaining 12,494 tweets were used for fine-tuning the model. The tweets were preprocessed to replace mentions, URLs, emojis, etc with generic keywords. The model was trained on a system with a single Nvidia A4000 16GB GPU. The fine-tuned model is also available as the 'model.bin' file. The code for finetuning the model as well as reproducing the experiments are available in this GitHub repository. Datasheet We also include a datasheet for the dataset following the recommendation of "Datasheets for Datasets" (Gebru et. al.) which provides more information about how the dataset was created and how it can be used. Please see "Datasheet.pdf". NOTE: We recommend that researchers try to rehydrate the individual tweets to ensure that the user has not deleted the tweet since posting. This gives users a mechanism to opt out of having their data analyzed. Please only use your institutional email when requesting the dataset as anything else (like gmail.com) will be rejected. The dataset will only be made available on reasonable request for Academics and Researchers. Please mention why you need the dataset and how you plan to use the dataset when making a request.数据集准备与标注 本数据集为QCRI采集的TBCOV数据集的子集，经筛选后仅保留提及与推文作者个人相关的新冠死亡事件的样本。筛选过程采用正则表达式实现，匹配模式包括"my * passed""my * died""my * succumbed"及"lost * battle"。本数据集的部分样本通过安彭（Appen）平台完成标注。完整的标注指南详见向标注人员提供的`annotation-instructions.txt`文件。 数据集说明 `classifier_filtered_english.csv`文件包含3.3万条经去重与分类器筛选的推文（符合X平台内容再分发政策），覆盖2020年3月至2021年3月间澳大利亚、加拿大、印度、意大利、英国、美国6个国家的相关数据，包含分类器标注的死亡标签、正则表达式筛选得到的性别与亲属关系标签，以及用户设备标签。完整的5.7万条经正则表达式筛选的推文集合，仅针对学术人员与研究者的特殊申请提供。 以下为各字段说明： - 日期：推文发布日期 - 国家名称：通过Nominatim API获取的国家名称 - 推文ID：推文的唯一标识符 - 推文链接：推文的完整URL - 完整文本：推文的完整内容（同时包含所附媒体的URL） - 推文是否提及作者个人认识的一名或多名个体的新冠相关死亡：分类器预测的死亡标签（同时包含标注样本的原始标签） - 推文作者与提及受害者的亲属关系：标注得到的亲属关系标签 - 提及的死亡事件相对推文发布的时间：标注得到的相对时间标签 - 用户认证状态：该Twitter用户是否经过认证 - 用户性别：从用户主页信息推断的Twitter用户性别 - 用户设备：用户使用的Twitter客户端 - 是否包含媒体：该推文是否附有媒体内容 - 是否包含链接：推文文本中是否包含URL - 匹配设备：根据Twitter客户端推断的设备类型（苹果或安卓） - 正则推断性别：基于正则表达式筛选得到的推断性别 - 正则推断亲属关系：基于正则表达式筛选得到的亲属关系标签 基于正则表达式的性别推断 首先建立推文中提及的各类亲属关系标签与性别的对应映射，对于无法直接推断性别的亲属关系（如"表亲/堂亲"）则不予采用。 男性亲属关系标签：'father', 'dad', 'daddy', 'papa', 'pop', 'pa', 'son', 'brother', 'uncle', 'nephew', 'grandfather', 'grandpa', 'gramps', 'husband', 'boyfriend', 'fiancé', 'groom', 'partner', 'beau', 'friend', 'buddy', 'pal', 'mate', 'companion', 'boy', 'gentleman', 'man', 'father-in-law', 'brother-in-law', 'stepfather', 'stepbrother' 女性亲属关系标签：'mother', 'mom', 'mama', 'mum', 'ma', 'daughter', 'sister', 'aunt', 'niece', 'grandmother', 'grandma', 'granny', 'wife', 'girlfriend', 'fiancée', 'bride', 'partner', 'girl', 'lady', 'woman', 'miss', 'mother-in-law', 'sister-in-law', 'stepmother', 'stepsister' 基于上述映射，我们针对每类性别标签采用以下正则表达式，以推断推文中提及的逝者性别：`[m|M]ys(` + `|`.join([r + `s?` for r in relationships]) + `)s(died|succumbed|deceased)`。 基于亲属关系标签的年龄分组 首先通过正则表达式筛选获取亲属关系标签，随后将其划分为不同的年龄分组，详见下表。由于官方数据中的年龄组定义存在差异，英国与美国采用了不同的年龄分组标准。 | 分类 | 推文中的亲属关系 | 英国年龄组 | 美国年龄组 | |------------|--------------------------------|------------|------------| | 祖父母辈 | grandfather, grandmother | 65+ | 65+ | | 父母辈 | father, mother, uncle, aunt | 45-64 | 35-64 | | 兄弟姐妹辈 | brother, sister, cousin | 15-44 | 15-34 | | 子女辈 | son, daughter, nephew, niece | 0-14 | 0-14 | 分类器训练 `english-training.csv`文件包含约1.3万条经去重的人工标注推文。我们采用随机种子（42）进行训练集与测试集的划分，并基于训练集对Covid-Bert-V2模型进行了2个训练轮次的微调，所用超参数通过10折交叉验证得到，具体为：随机种子（random_seed）：42，批次大小（batch_size）：32，丢弃率（dropout）：0.1。我们在测试集上得到了0.81的F1分数。具体拆分比例为：将合并去重后的标注推文的约5%（671条）作为测试集，约2%（255条）作为验证集，剩余的12494条推文用于模型微调。 推文预处理阶段会将提及对象、URL、表情符号等替换为通用关键词。模型训练基于搭载单张Nvidia A4000 16GB GPU的计算平台完成。微调后的模型以`model.bin`文件形式提供。模型微调代码与实验复现代码均可在本GitHub仓库中获取。 数据集文档表 我们遵循《数据集文档表（Datasheets for Datasets）》（Gebru等人）的建议，附带了本数据集的文档表，其中包含数据集构建与使用方式的更多细节，详见`Datasheet.pdf`文件。 注意：我们建议研究人员尝试对单条推文进行数据还原，以确认该推文的作者尚未在发布后删除该推文，此举可为用户提供退出数据被分析的途径。 申请数据集时仅可使用机构邮箱，非机构邮箱（如gmail.com）的申请将被拒绝。本数据集仅会针对学术人员与研究者的合理申请提供。申请时请说明您的数据集用途与使用计划。