Corpus for Knowledge-Enhanced Language Model Pre-training (KELM) 专注于将知识图谱三元组（主语、关系、宾语）转换为自然语言句子，用于数据到文本的生成任务。该语料库包含约1800万个句子，涵盖约4500万个三元组和约1500个不同的关系，数据来源于英文维基百科及其相关的知识图谱。数据集包括训练集、验证集和测试集，并采用CC BY-SA
Recent advances in scaling large language models (LLMs) has resulted in significant improvements over a number of natural language processing benchmarks. There has been some work to pretrain these lan
