FrancophonIA/French_GEC

该数据集是通过Wikipedia的编辑历史创建的，目的是找到对句子的修正并创建X,y句子对的数据集。数据集包含45百万个X,y句子对，这些句子对几乎涵盖了整个法语Wikipedia。CSV文件包含五个列：X（源句子）、y（目标句子）、title（文章标题）、timestamps（两个日期，分别是源句子和目标句子的创建时间）、comments（编辑的评论，如果有的话）。数据集在用于语法错误纠正（GEC）任务时存在一个问题，即许多句子对的修正并不属于GEC任务的范畴，而是句子的重新表述、简化或澄清。为了解决这个问题，建议使用迁移学习创建一个分类模型来过滤掉这些“不良”句子。

This dataset was created using the edit histories of the French Wikipedia, containing 45 million X,y sentence pairs extracted from almost the entire French Wikipedia. The datasets CSV files include four columns: X (the source sentence), y (the target sentence), title (the title of the article from which the sentence came from), timestamps (the two dates when the source sentence and target sentence were created), and comments (the comment of the edit if specified). A major issue with this dataset is that many of the extracted sentence pairs are not within the scope of the GEC (Grammar Error Correction) task, as many Wikipedia edits involve reformulations, synthetizations, or clarifications rather than correcting grammar, typos, or syntax. Therefore, it is suggested to create a classification model using transfer learning to filter out these bad sentences.