lewtun/gem-sub-03

---
benchmark: gem
type: prediction
submission_name: T5-base (Baseline)
---

# GEM submissions for gem-sub-03

## Submitting to the benchmark

FILL ME IN

### Submission file format

Please follow this format for your `submission.json` file:

```json
{
"submission_name": "An identifying name of your system",
"param_count": 123, # the number of parameters your system has.
"description": "An optional brief description of the system that will be shown on the website",
"tasks":
{
"dataset_identifier": {
"values": ["output1", "output2", "..."], # A list of system outputs
# Optionally, you can add the keys which are part of an example to ensure that there is no shuffling mistakes.
"keys": ["key-0", "key-1", ...]
}
}
}
```

In this case, `dataset_identifier` is the identifier of the dataset followed by an identifier of the set the outputs were created from, for example `_validation` or `_test`. That means, the `mlsum_de` test set would have the identifier `mlsum_de_test`.

The `keys` field can be set to avoid accidental shuffling to impact your metrics. Simply add a list of the `gem_id` for each output example in the same order as your values.

### Validate your submission

To ensure that your submission files are correctly formatted, run the following command from the root of the repository:

```
python cli.py validate
```

If everything is correct, you should see the following message:

```
All submission files validated! ✨ 🚀 ✨
Now you can make a submission 🤗
```

### Push your submission to the Hugging Face Hub!

The final step is to commit your files and push them to the Hub:

```
python cli.py submit
```

If there are no errors, you should see the following message:

```
Submission successful! 🎉 🥳 🎉
Your submission will be evaulated on Sunday 05 September 2021 ⏳
```

where the evaluation is run every Sunday and your results will be visible on the leaderboard.

原始信息汇总

GEM submissions for gem-sub-03

提交文件格式

请按照以下格式准备您的 submission.json 文件：

json { "submission_name": "系统的标识名称", "param_count": 123, # 系统参数的数量 "description": "网站上显示的系统简短描述（可选）", "tasks": { "dataset_identifier": { "values": ["output1", "output2", "..."], # 系统输出的列表 "keys": ["key-0", "key-1", ...] # 可选，用于避免意外的顺序错误 } } }

其中，dataset_identifier 是数据集的标识符，后面跟着输出集的标识符，例如 _validation 或 _test。这意味着，mlsum_de 测试集的标识符将是 mlsum_de_test。

keys 字段可以设置以避免意外的顺序错误。只需添加与您的值顺序相同的每个输出示例的 gem_id 列表。

验证提交文件

为了确保您的提交文件格式正确，请从仓库的根目录运行以下命令：

bash python cli.py validate

如果一切正确，您应该会看到以下消息：

所有提交文件验证通过！✨ 🚀 ✨ 现在您可以进行提交 🤗

提交到 Hugging Face Hub

最后一步是提交您的文件并推送到 Hub：

bash python cli.py submit

如果没有错误，您应该会看到以下消息：

提交成功！🎉 🥳 🎉 您的提交将在2021年9月5日星期日进行评估 ⏳

评估每周日运行，您的结果将在排行榜上可见。