1213d/CROP-dataset

--- license: cc-by-nc-4.0 --- ## Introduction Crop-dataset is a large-scale open-source instruction fine-tuning dataset for LLMs in crop science, which includes over 210K high-quality question-answer pairs in Chinese and English. <div style="text-align: center;"> <img src="./Figures/dataset_pre_distribution.png" alt="Dataset Framework" width="100%"/> </div> ## Basic Information Currently, Crop-dataset primarily includes two types of grains: rice and corn. The dataset contains a sufficient amount of single-turn and multi-turn question-answer pairs. ### Composition of the Single-round Dialogue Dataset | Cereal | Type | Task | English QA | Chinese QA | Total | |--------|------|-----------------------------|------------|------------|--------| | Rice | Closed QA | In-domain knowledge QA | 42951 | 83396 | 126347 | | | Open QA | In-domain knowledge QA | 2430 | 2037 | 4467 | | | | Event extraction | 1891 | 1030 | 2921 | | | | Named Entity Recognition| 2003 | 1604 | 3607 | | | | Summary | 1586 | 1628 | 3214 | | Corn | Closed QA | In-domain knowledge QA | 25259 | 27667 | 52926 | | | Open QA | In-domain knowledge QA | 3202 | 3047 | 6249 | | | | Event extraction | 2245 | 1322 | 3567 | | | | Named Entity Recognition| 2008 | 1316 | 3324 | | | | Summary | 1559 | 1857 | 3416 | | Others*| --- | --- | --- | --- | <1000 | | Overall| --- | --- | 85134 | 124904 | 210038 | *The 'Others' category includes data from other grains like wheat, which is less than 1000 QA pairs in total. Please note that despite our data-cleaning efforts, the final QA pairs inevitably contain a small amount of data (<0.5%) from other grains like wheat. Because we think this data does not dominantly influence the fine-tuning results, it is included in the rice and maize QA pairs, and we have listed it separately to avoid any misleading counts. ### Composition of the Multi-round Dialogue Dataset | Cereal | Scenario | Task | English QA | Chinese QA | Total | |--------|---------------------------|--------------------------|--------------------------------|-----------------------------|-------| | Rice | Problem Solving | Pest Control | 14 (3-turn) + 71 (4-turn) | 8 (3-turn) + 37 (4-turn) | 130 | | | | Nutrient Supplementation | 19 (3-turn) + 93 (4-turn) | 2 (3-turn) + 90 (4-turn) + 1 (5-turn) | 205 | | | | Disease Containment | 19 (3-turn) + 60 (4-turn) | 4 (3-turn) + 39 (4-turn) | 122 | | | Personalized Recommendation | Crop Variety Selection | 12 (3-turn) + 53 (4-turn) | 9 (3-turn) + 9 (4-turn) | 83 | | | | Resource Management | 4 (3-turn) + 110 (4-turn) + 1 (5-turn) | 5 (3-turn) + 50 (4-turn) | 170 | | | Knowledge Interpretation | Research Interpretation | 3 (3-turn) + 125 (4-turn) + 1 (5-turn) | 8 (3-turn) + 85 (4-turn) | 222 | | Corn | Problem Solving | Pest Control | 20 (3-turn) + 84 (4-turn) | 7 (3-turn) + 77 (4-turn) | 188 | | | | Nutrient Supplementation | 24 (3-turn) + 56 (4-turn) | 8 (3-turn) + 30 (4-turn) | 118 | | | | Disease Containment | 21 (3-turn) + 64 (4-turn) | 2 (3-turn) + 19 (4-turn) + 1 (5-turn) | 107 | | | Personalized Recommendation| Crop Variety Selection | 19 (3-turn) + 75 (4-turn) | 46 (3-turn) + 47 (4-turn) | 187 | | | | Resource Management | 8 (3-turn) + 94 (4-turn) | 1 (3-turn) + 69 (4-turn) | 172 | | | Knowledge Interpretation | Research Interpretation | 5 (3-turn) + 94 (4-turn) + 1 (5-turn) | 6 (3-turn) + 61 (4-turn) | 167 | | Overall| --- | --- | 1150 | 721 | 1871 | ## How to Use We have released our dataset in Alpaca format. All question-answer pairs are organized by task. The codes and prompts related to this dataset are released at https://github.com/RenqiChen/The_Crop. ## BibTeX & Citation If you find our codes and datasets useful, please consider citing our work: ```bibtex @inproceedings{zhangempowering, title={Empowering and Assessing the Utility of Large Language Models in Crop Science}, author={Zhang, Hang and Sun, Jiawei and Chen, Renqi and Liu, Wei and Yuan, Zhonghang and Zheng, Xinzhe and Wang, Zhefan and Yang, Zhiyuan and Yan, Hang and Zhong, Han-Sen and others}, booktitle={The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track} } ```license: CC BY-NC 4.0（知识共享署名-非商业性使用4.0国际许可协议） ## 简介 作物数据集（Crop-dataset）是面向作物科学领域大语言模型（Large Language Model, LLM）的大规模开源指令微调数据集，包含超过21万条高质量中英双语问答对。 <div style="text-align: center;"> <img src="./Figures/dataset_pre_distribution.png" alt="数据集框架" width="100%"/> </div> ## 基本信息 目前，本数据集主要涵盖水稻与玉米两大类谷物作物，包含足量的单轮与多轮问答对。 ### 单轮对话数据集构成 | 谷物类别 | 类型 | 任务类型 | 英文问答对数量 | 中文问答对数量 | 总计 | |--------|------|-----------------------------|------------|------------|--------| | 水稻 | 封闭域问答 | 域内知识问答 | 42951 | 83396 | 126347 | | | 开放域问答 | 域内知识问答 | 2430 | 2037 | 4467 | | | | 事件抽取 | 1891 | 1030 | 2921 | | | | 命名实体识别| 2003 | 1604 | 3607 | | | | 文本摘要 | 1586 | 1628 | 3214 | | 玉米 | 封闭域问答 | 域内知识问答 | 25259 | 27667 | 52926 | | | 开放域问答 | 域内知识问答 | 3202 | 3047 | 6249 | | | | 事件抽取 | 2245 | 1322 | 3567 | | | | 命名实体识别| 2008 | 1316 | 3324 | | | | 文本摘要 | 1559 | 1857 | 3416 | | 其他类别*| --- | --- | --- | --- | <1000 | | 总计| --- | --- | 85134 | 124904 | 210038 | *「其他类别」涵盖小麦等其他谷物作物的相关数据，总计问答对数量不足1000条。 请注意，尽管经过了数据清洗流程，最终的问答对中仍不可避免地包含少量（<0.5%）小麦等其他谷物作物的相关数据。由于我们认为此类数据不会对微调结果产生显著影响，因此将其归入水稻与玉米的问答对中，并单独列出以避免统计结果产生误导。 ### 多轮对话数据集构成 | 谷物类别 | 应用场景 | 任务类型 | 英文问答对数量 | 中文问答对数量 | 总计 | |--------|---------------------------|--------------------------|--------------------------------|-----------------------------|-------| | 水稻 | 问题解决 | 病虫害防治 | 14（3轮） + 71（4轮） | 8（3轮） + 37（4轮） | 130 | | | | 养分补给 | 19（3轮） + 93（4轮） | 2（3轮） + 90（4轮） + 1（5轮） | 205 | | | | 病害防控 | 19（3轮） + 60（4轮） | 4（3轮） + 39（4轮） | 122 | | | 个性化推荐 | 作物品种选择 | 12（3轮） + 53（4轮） | 9（3轮） + 9（4轮） | 83 | | | | 资源管理 | 4（3轮） + 110（4轮） + 1（5轮） | 5（3轮） + 50（4轮） | 170 | | | 知识解读 | 科研解读 | 3（3轮） + 125（4轮） + 1（5轮） | 8（3轮） + 85（4轮） | 222 | | 玉米 | 问题解决 | 病虫害防治 | 20（3轮） + 84（4轮） | 7（3轮） + 77（4轮） | 188 | | | | 养分补给 | 24（3轮） + 56（4轮） | 8（3轮） + 30（4轮） | 118 | | | | 病害防控 | 21（3轮） + 64（4轮） | 2（3轮） + 19（4轮） + 1（5轮） | 107 | | | 个性化推荐| 作物品种选择 | 19（3轮） + 75（4轮） | 46（3轮） + 47（4轮） | 187 | | | | 资源管理 | 8（3轮） + 94（4轮） | 1（3轮） + 69（4轮） | 172 | | | 知识解读 | 科研解读 | 5（3轮） + 94（4轮） + 1（5轮） | 6（3轮） + 61（4轮） | 167 | | 总计| --- | --- | 1150 | 721 | 1871 | ## 使用方式 本数据集已以Alpaca格式发布，所有问答对均按任务类型进行组织。与本数据集相关的代码与提示词已发布于：https://github.com/RenqiChen/The_Crop。 ## 参考文献与引用 若您认为本代码与数据集对您的研究有所帮助，请考虑引用我们的工作： bibtex @inproceedings{zhangempowering, title={Empowering and Assessing the Utility of Large Language Models in Crop Science}, author={Zhang, Hang and Sun, Jiawei and Chen, Renqi and Liu, Wei and Yuan, Zhonghang and Zheng, Xinzhe and Wang, Zhefan and Yang, Zhiyuan and Yan, Hang and Zhong, Han-Sen and others}, booktitle={The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track} }