Instruction-Tuning Dataset and LoRA Adapter for the Climatology-specific Large Language Model

This repository provides the instruction-tuning dataset and LoRA adapter used in our study, aiming to ensure transparency and reproducibility in fine-tuning large language models (LLMs) on recent academic papers and assessment reports concerning climate change.

Project Overview

This project aims to develop an AI-based framework that enables local governments to conduct scientifically sound climate risk assessments and design adaptation strategies, even in the absence of specialized expertise or significant resources, based on recent academic papers and assessment reports.

Contents

Directory
Description

dataset
Question-Answer pairs in JSONL format (English and Japanese)

lora-adapter
LoRA adapter for Llama-3.3-Swallow-70B-Instruct-v0.4<br>(Compatible with Hugging Face)

Dataset Overview

The dataset used in this study consists of question–answer pairs based on given contexts.

Each sample requires the model to either generate an open-ended textual answer or select one of four multiple-choice options.

The data are organized by language (English or Japanese) and answer format (textual or multiple-choice), resulting in four subsets:

`en_default`: English, open-ended textual answers  

`en_multichoice`: English, multiple-choice questions  

`jp_default`: Japanese, open-ended textual answers  

`jp_multichoice`: Japanese, multiple-choice questions  

Each subset is divided into training, validation, and test sets in an 8:1:1 ratio. The number of samples in each split is shown below. In total, the dataset contains approximately 190,000 samples, ensuring sufficient coverage across both languages and answer types. This design allows models to learn variations in linguistic structures and response formats, enabling effective cross-lingual and cross-format generalization.

Dataset
Train (train.json)
Validation (val.json)
Test (test.json)
Total (dataset.json)

en_default
38,745
4,843
4,844
48,432

en_multichoice
37,903
4,738
4,738
47,379

jp_default
38,336
4,792
4,792
47,920

jp_multichoice
37,111
4,639
4,639
46,389

all data
152,095
19,012
19,013
190,120

Sources of Instruction-Tuning Data

The dataset was constructed from research papers and assessment reports relevant to climate change.

Ministry of Education, Culture, Sports, Science and Technology (MEXT)  Advanced Studies of Climate Change Projection (SENTAN Program)

National Institute for Environmental Studies (NIES)  Climate Change Adaptation Information Platform (A-PLAT) 

3. Intergovernmental Panel on Climate Change (IPCC)  AR6 Synthesis Report: Climate Change 2023

Dataset Construction and Fine-Tuning Environment

Component
Tool / Model
Version / Source

PDF text extraction
Docling 
2.26.0

Instruction generation model
OpenGVLab/InternVL2_5-78B-MPO
78B MPO model

Fine-tuning framework
ms-swift
3.3.0

Base model for fine-tuning
tokyotech-llm/Llama-3.3-Swallow-70B-Instruct-v0.4
f99e99588303e8a52b88076d3a5f24db19f757a6

Intended Use

This dataset and LoRA adapter are released for research and educational purposes only. Recommended use cases include:

Summarization and explanation of scientific and assessment reports on climate change  

Knowledge extraction and question answering in climate science and policy domains  

Development of AI assistants for science communication and knowledge dissemination

License Information

Please note that the dataset and the LoRA adapter provided in this repository are distributed under different licenses. By downloading or using these resources, you agree to abide by their respective terms.

Dataset:

The QA dataset covering climate science and global change is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0).View CC BY-NC-SA 4.0 License details

LoRA Adapter:The LoRA adapter was fine-tuned on the Llama 3.3 Swallow 70B Instruct base model. Therefore, the use of these model weights is subject to the Llama 3.3 Community License Agreement, as well as the Gemma Terms of Use.View Llama 3.3 Community LicenseView Gemma Terms of Use

Notes

The dataset contains texts derived from research papers and assessment reports published by the above institutions. Copyright remains with the original authors and organizations.  

Commercial use of the dataset is strictly prohibited in accordance with the CC BY-NC-SA 4.0 license. For the commercial use of the LoRA adapter, users must refer to and comply with the specific terms of the Llama 3.3 Community License and Gemma Terms of Use.

Users should verify and evaluate model outputs before citation or publication.

Acknowledgments
We are grateful to Drs. Takero Yoshida, Masuo Nakano, Takashi Hosono, Yoichi Ishikawa, Noriko Ishizaki, Yuya Takane, Yasutaka Wakazuki, Makoto Tamura, Takashi Hamada, Masatoshi Kuribayashi, Shinnosuke Furuya for their advice on the model development and application. This work was supported by NEDO GENIAC (Grant No. 24036962); Environment Research and Technology Development Fund of the ERCA (Grant No. JPMEERF25S12433); DIAS (Grant No. JPMXD0721453504) and SENTAN (Grant No. JPMXD0722680734) of MEXT; and JSPS KAKENHI (Grant No. JP22H01316). This work was conducted using the Earth Simulator under the support of JAMSTEC and computational resources provided by Amazon Web Services (AWS) under the support of Classmethod, Inc.

Copyright
Copyright (c) 2026 Japan Agency for Marine-Earth Science and Technology (JAMSTEC). All rights reserved.

(Except for the source texts derived from MEXT, NIES, and IPCC reports, where copyrights remain with their respective original authors and organizations.)

Related Links

SENTAN Program

A-PLAT

IPCC AR6 Reports

Docling Project

OpenGVLab/InternVL2_5-78B-MPO

ms-swift Framework

tokyotech-llm/Llama-3.3-Swallow-70B-Instruct-v0.4