linagora/linto-dataset-text-ar-tn

LinTO Textual Dataset in Arabic Tunisian是一个用于语言建模的突尼斯方言文本集合，主要用于训练具有代码切换能力的突尼斯方言自动语音识别系统（ASR）。该数据集包含多种文本来源，如音乐（如RAP歌词）、纪录片、名字、故事等。数据集的组成包括从Hugging Face、GitHub和网站等多个来源收集的文本内容。数据集还经过了词规范化、数字转换和标点符号去除等处理。数据集的内容类型包括音乐、情感分析、纪录片、推文、课程等，语言和方言主要为突尼斯阿拉伯语，也包括一些现代标准阿拉伯语（MSA）、法语和英语的代码切换。

The LinTO Textual Dataset in Arabic Tunisian is a diverse collection of text sources, including music (e.g., RAP lyrics), documentaries, names, stories, and more. This dataset was curated to support the development of a language model for the Tunisian Arabic Automatic Speech Recognition (ASR) system. The dataset is composed of multiple subsets, each with its own source and characteristics. The README also includes information about the data processing steps, content types, languages and dialects covered, and example usage in Python. The dataset is licensed under the Apache 2.0 license, and citations are provided for proper acknowledgment of its use.