projudge-173k

# ProJudge: A Multi-Modal Multi-Discipline Benchmark and Instruction-Tuning Dataset for MLLM-based Process Judges

**ProJudge-173k** is the first large-scale instruction tuning dataset specifically designed for process evaluation with fine-grained step-level annotations.

It features:
- Multi-Modal: Various modalities, including pure text, single image, and multi-image interleaved content;
- Multi-Discipline: 4 scientific disciplines: mathematics, physics, chemistry, and biology;
- Multi-Difficulty: Diverse difficulty levels ranging from primary school to competition-levels.

# An Example to load the data

```python
# To load the entire dataset:
from datasets import load_dataset
dataset=load_dataset("julyai/ProJudge-173k", split="train")
print(dataset[0])

# To load different subset:
camel_dataset=load_dataset("julyai/ProJudge-173k/Camel-AI", split="train")
print(camel_dataset[0])
k12_dataset=load_dataset("julyai/ProJudge-173k/k12", split="train")
print(k12_dataset[0])
OlympiadBench_dataset=load_dataset("julyai/ProJudge-173k/OlympiadBench", split="train")
print(OlympiadBench_dataset[0])
```

More details on loading and using the data are at our [github page](https://github.com/jiaxin-ai/ProJudge).

If you do find our code helpful or use our benchmark dataset, please citing our paper.

```
@article{ai2025projudge,
title={ProJudge: A Multi-Modal Multi-Discipline Benchmark and Instruction-Tuning Dataset for MLLM-based Process Judges},
author={Jiaxin Ai and Pengfei Zhou and Zhaopan Xu and Ming Li and Fanrui Zhang and Zizhen Li and Jianwen Sun and Yukang Feng and Baojin Huang and Zhongyuan Wang and Kaipeng Zhang},
journal={arXiv preprint arXiv:2503.06553},
year={2025}
}
```

# ProJudge：面向基于多模态大语言模型（MLLM）的流程评判器的多模态多学科基准与指令微调数据集

**ProJudge-173k** 是首个专为流程评估打造的大规模指令微调数据集，带有细粒度的步骤级标注。

其特性如下：
- 多模态：涵盖纯文本、单图像、多图像交错内容等多种模态；
- 多学科：覆盖数学、物理、化学、生物4大学科领域；
- 多难度：包含从小学到竞赛级别的多样化难度层级。

# 数据加载示例
python
# 加载完整数据集：
from datasets import load_dataset
dataset=load_dataset("julyai/ProJudge-173k", split="train")
print(dataset[0])

# 加载不同子集：
camel_dataset=load_dataset("julyai/ProJudge-173k/Camel-AI", split="train")
print(camel_dataset[0])
k12_dataset=load_dataset("julyai/ProJudge-173k/k12", split="train")
print(k12_dataset[0])
OlympiadBench_dataset=load_dataset("julyai/ProJudge-173k/OlympiadBench", split="train")
print(OlympiadBench_dataset[0])

更多数据加载与使用细节，请参阅我们的[GitHub页面](https://github.com/jiaxin-ai/ProJudge)。

若您的工作中用到了本代码或基准数据集，请引用我们的论文：

@article{ai2025projudge,
title={ProJudge: A Multi-Modal Multi-Discipline Benchmark and Instruction-Tuning Dataset for MLLM-based Process Judges},
author={Jiaxin Ai and Pengfei Zhou and Zhaopan Xu and Ming Li and Fanrui Zhang and Zizhen Li and Jianwen Sun and Yukang Feng and Baojin Huang and Zhongyuan Wang and Kaipeng Zhang},
journal={arXiv preprint arXiv:2503.06553},
year={2025}
}