GEM/references

# GEM References

## What is it?

This repository contains all the reference datasets that are used for running evaluation on the GEM benchmark. Some of these datasets were originally hosted as a [GitHub release](https://github.com/GEM-benchmark/GEM-metrics/releases) on the [`GEM-metrics`](https://github.com/GEM-benchmark/GEM-metrics) repository, but have been migrated to the Hugging Face Hub.

## Converting datasets to JSON

We provide a `convert_dataset_to_json.py` conversion script that converts the datasets in the GEM organisation to the JSON format expected by the `GEM-metrics` library. To run the script, first install [`jq`](https://stedolan.github.io/jq/download/) and then install the script's Python dependencies:

```
python -m pip install -r requirements.txt
```

You can then run the script as follows:

```python
python generate_evaluation_datasets.py
```

This script will:

* Download and convert the datasets under the GEM organisation to JSON format
* Validate that the each dataset has the expected columns of `gem_id`, `target`, and `references`

原始信息汇总

GEM References 数据集概述

数据集用途

该数据集用于GEM基准的评估运行，包含所有参考数据集。

数据集格式转换

提供了一个名为convert_dataset_to_json.py的转换脚本，用于将GEM组织下的数据集转换为GEM-metrics库预期的JSON格式。

转换步骤

1. 安装jq工具。

2. 安装脚本的Python依赖：

   python -m pip install -r requirements.txt

3. 运行脚本： python python generate_evaluation_datasets.py

脚本功能

• 下载并转换GEM组织下的数据集至JSON格式。
• 验证每个数据集是否包含预期的gem_id, target, 和 references列。