UMCU/DutchMedicalTextV3

DutchMedicalText v2是一个荷兰语医疗文本数据集，由PMC/Pubmed、Apollo/Meditron现有数据集、多种荷兰语来源以及经过医学筛选的fineweb和finepdf文本的翻译版本混合而成。数据集通过神经机器翻译生成，可能存在虚假重复的问题，但提供了Python代码示例来缓解这一问题。使用了多种翻译工具（如MariaNMT/NLLB200、GeminiFlash 1.5/2.0、GPT4o-mini）和开发库（如PubScience）进行处理，所有文本都使用DEDUCE进行了近似去标识化处理。适用于文本生成和填充掩码任务，专注于荷兰语医疗领域，大小在10M到100M之间。

DutchMedicalText v2 is a Dutch medical text dataset consisting of a translated mix of PMC/Pubmed, existing data sets from Apollo/Meditron, various Dutch sources, and medically filtered texts from fineweb and finepdf. The dataset may contain spurious repetitions due to neural machine translation, but provides Python code examples to mitigate this issue. It utilizes various translation tools (e.g., MariaNMT/NLLB200, GeminiFlash 1.5/2.0, GPT4o-mini) and development libraries (e.g., PubScience), with all texts approximately de-identified using DEDUCE. Suitable for text-generation and fill-mask tasks, focusing on the Dutch medical domain, with a size between 10M and 100M.