FreCDo

FreCDo是由布加勒斯特大学和艾克斯-马赛大学创建的大型法语跨领域方言识别数据集，包含413,522个法语文本样本，来源于比利时、加拿大、法国和瑞士的公共新闻网站。数据集设计旨在消除与主题、写作风格和出版来源相关的潜在偏见，适用于训练和评估现代深度语言模型。创建过程中，数据被分为训练、验证和测试集，确保不同子集间主题和出版来源的重叠最小化。该数据集主要用于解决法语方言识别问题，有助于分析和理解语言随时间和地理区域的演变。

FreCDo is a large-scale French cross-domain dialect identification dataset developed by the University of Bucharest and Aix-Marseille University. It contains 413,522 French text samples sourced from public news websites of Belgium, Canada, France and Switzerland. The dataset is designed to eliminate potential biases related to topics, writing styles and publication sources, and is applicable to the training and evaluation of modern deep language models. During the creation process, the data was divided into training, validation and test sets, ensuring minimal overlap of topics and publication sources across different subsets. This dataset is mainly used to tackle the French dialect identification task, and helps analyze and understand the evolution of language across time and geographic regions.