CodeTheory/demo

---
size_categories: n<1K
tags:
- rlfh
- argilla
- human-feedback
---

# Dataset Card for demo

This dataset has been created with [Argilla](https://docs.argilla.io).

As shown in the sections below, this dataset can be loaded into Argilla as explained in [Load with Argilla](#load-with-argilla), or used directly with the `datasets` library in [Load with `datasets`](#load-with-datasets).

## Dataset Description

- **Homepage:** https://argilla.io
- **Repository:** https://github.com/argilla-io/argilla
- **Paper:**
- **Leaderboard:**
- **Point of Contact:**

### Dataset Summary

This dataset contains:

* A dataset configuration file conforming to the Argilla dataset format named `argilla.yaml`. This configuration file will be used to configure the dataset when using the `FeedbackDataset.from_huggingface` method in Argilla.

* Dataset records in a format compatible with HuggingFace `datasets`. These records will be loaded automatically when using `FeedbackDataset.from_huggingface` and can be loaded independently using the `datasets` library via `load_dataset`.

* The [annotation guidelines](#annotation-guidelines) that have been used for building and curating the dataset, if they've been defined in Argilla.

### Load with Argilla

To load with Argilla, you'll just need to install Argilla as `pip install argilla --upgrade` and then use the following code:

```python
import argilla as rg

ds = rg.FeedbackDataset.from_huggingface("CodeTheory/demo")
```

### Load with `datasets`

To load this dataset with `datasets`, you'll just need to install `datasets` as `pip install datasets --upgrade` and then use the following code:

```python
from datasets import load_dataset

ds = load_dataset("CodeTheory/demo")
```

### Supported Tasks and Leaderboards

This dataset can contain [multiple fields, questions and responses](https://docs.argilla.io/en/latest/conceptual_guides/data_model.html#feedback-dataset) so it can be used for different NLP tasks, depending on the configuration. The dataset structure is described in the [Dataset Structure section](#dataset-structure).

There are no leaderboards associated with this dataset.

### Languages

[More Information Needed]

## Dataset Structure

### Data in Argilla

The dataset is created in Argilla with: **fields**, **questions**, **suggestions**, **metadata**, **vectors**, and **guidelines**.

The **fields** are the dataset records themselves, for the moment just text fields are supported. These are the ones that will be used to provide responses to the questions.

| Field Name | Title | Type | Required | Markdown |
| ---------- | ----- | ---- | -------- | -------- |
| instruction | 指令 | text | True | False |
| input | 输入 | text | True | False |
| output | 输出 | text | True | False |

The **questions** are the questions that will be asked to the annotators. They can be of different types, such as rating, text, label_selection, multi_label_selection, or ranking.

| Question Name | Title | Type | Required | Description | Values/Labels |
| ------------- | ----- | ---- | -------- | ----------- | ------------- |
| question-rating | 对提问进行评分 | rating | False | N/A | [1, 2, 3, 4, 5, 6, 7, 8, 9] |
| context-rating | 对回复进行评分 | rating | True | N/A | [1, 2, 3, 4, 5, 6, 7, 8, 9] |
| preference | 哪个回复最好？按照从好到坏的顺序进行排序 | ranking | True | N/A | ['A', 'B', 'C'] |
| suggestion | 建议 | text | True | N/A | N/A |
| topics | 选择提问的主题 | multi_label_selection | True | N/A | ['数学问题', '逻辑问题', '地理知识', '文学知识', '居家知识', '行业知识'] |
| sentiment | 对话的感情 | label_selection | True | N/A | ['积极', '中立', '消极'] |

The **suggestions** are human or machine generated recommendations for each question to assist the annotator during the annotation process, so those are always linked to the existing questions, and named appending "-suggestion" and "-suggestion-metadata" to those, containing the value/s of the suggestion and its metadata, respectively. So on, the possible values are the same as in the table above, but the column name is appended with "-suggestion" and the metadata is appended with "-suggestion-metadata".

The **metadata** is a dictionary that can be used to provide additional information about the dataset record. This can be useful to provide additional context to the annotators, or to provide additional information about the dataset record itself. For example, you can use this to provide a link to the original source of the dataset record, or to provide additional information about the dataset record itself, such as the author, the date, or the source. The metadata is always optional, and can be potentially linked to the `metadata_properties` defined in the dataset configuration file in `argilla.yaml`.

| Metadata Name | Title | Type | Values | Visible for Annotators |
| ------------- | ----- | ---- | ------ | ---------------------- |

The **guidelines**, are optional as well, and are just a plain string that can be used to provide instructions to the annotators. Find those in the [annotation guidelines](#annotation-guidelines) section.

### Data Instances

An example of a dataset instance in Argilla looks as follows:

```json
{
"external_id": null,
"fields": {
"input": "",
"instruction": "\u4fdd\u6301\u5065\u5eb7\u7684\u4e09\u4e2a\u63d0\u793a\u3002",
"output": "\u4ee5\u4e0b\u662f\u4fdd\u6301\u5065\u5eb7\u7684\u4e09\u4e2a\u63d0\u793a\uff1a\n\n1. \u4fdd\u6301\u8eab\u4f53\u6d3b\u52a8\u3002\u6bcf\u5929\u505a\u9002\u5f53\u7684\u8eab\u4f53\u8fd0\u52a8\uff0c\u5982\u6563\u6b65\u3001\u8dd1\u6b65\u6216\u6e38\u6cf3\uff0c\u80fd\u4fc3\u8fdb\u5fc3\u8840\u7ba1\u5065\u5eb7\uff0c\u589e\u5f3a\u808c\u8089\u529b\u91cf\uff0c\u5e76\u6709\u52a9\u4e8e\u51cf\u5c11\u4f53\u91cd\u3002\n\n2. \u5747\u8861\u996e\u98df\u3002\u6bcf\u5929\u98df\u7528\u65b0\u9c9c\u7684\u852c\u83dc\u3001\u6c34\u679c\u3001\u5168\u8c37\u7269\u548c\u8102\u80aa\u542b\u91cf\u4f4e\u7684\u86cb\u767d\u8d28\u98df\u7269\uff0c\u907f\u514d\u9ad8\u7cd6\u3001\u9ad8\u8102\u80aa\u548c\u52a0\u5de5\u98df\u54c1\uff0c\u4ee5\u4fdd\u6301\u5065\u5eb7\u7684\u996e\u98df\u4e60\u60ef\u3002\n\n3. \u7761\u7720\u5145\u8db3\u3002\u7761\u7720\u5bf9\u4eba\u4f53\u5065\u5eb7\u81f3\u5173\u91cd\u8981\uff0c\u6210\u5e74\u4eba\u6bcf\u5929\u5e94\u4fdd\u8bc1 7-8 \u5c0f\u65f6\u7684\u7761\u7720\u3002\u826f\u597d\u7684\u7761\u7720\u6709\u52a9\u4e8e\u51cf\u8f7b\u538b\u529b\uff0c\u4fc3\u8fdb\u8eab\u4f53\u6062\u590d\uff0c\u5e76\u63d0\u9ad8\u6ce8\u610f\u529b\u548c\u8bb0\u5fc6\u529b\u3002"
},
"metadata": {},
"responses": [
{
"status": "submitted",
"user_id": "c658ddde-2d39-43ce-b478-633a1d19d2c7",
"values": {
"context-rating": {
"value": 5
},
"preference": {
"value": [
{
"rank": 2,
"value": "A"
},
{
"rank": 3,
"value": "B"
},
{
"rank": 1,
"value": "C"
}
]
},
"question-rating": {
"value": 4
},
"sentiment": {
"value": "\u4e2d\u7acb"
},
"suggestion": {
"value": "111"
},
"topics": {
"value": [
"\u903b\u8f91\u95ee\u9898",
"\u5730\u7406\u77e5\u8bc6"
]
}
}
}
],
"suggestions": [],
"vectors": {}
}
```

While the same record in HuggingFace `datasets` looks as follows:

```json
{
"context-rating": [
{
"status": "submitted",
"user_id": "c658ddde-2d39-43ce-b478-633a1d19d2c7",
"value": 5
}
],
"context-rating-suggestion": null,
"context-rating-suggestion-metadata": {
"agent": null,
"score": null,
"type": null
},
"external_id": null,
"input": "",
"instruction": "\u4fdd\u6301\u5065\u5eb7\u7684\u4e09\u4e2a\u63d0\u793a\u3002",
"metadata": "{}",
"output": "\u4ee5\u4e0b\u662f\u4fdd\u6301\u5065\u5eb7\u7684\u4e09\u4e2a\u63d0\u793a\uff1a\n\n1. \u4fdd\u6301\u8eab\u4f53\u6d3b\u52a8\u3002\u6bcf\u5929\u505a\u9002\u5f53\u7684\u8eab\u4f53\u8fd0\u52a8\uff0c\u5982\u6563\u6b65\u3001\u8dd1\u6b65\u6216\u6e38\u6cf3\uff0c\u80fd\u4fc3\u8fdb\u5fc3\u8840\u7ba1\u5065\u5eb7\uff0c\u589e\u5f3a\u808c\u8089\u529b\u91cf\uff0c\u5e76\u6709\u52a9\u4e8e\u51cf\u5c11\u4f53\u91cd\u3002\n\n2. \u5747\u8861\u996e\u98df\u3002\u6bcf\u5929\u98df\u7528\u65b0\u9c9c\u7684\u852c\u83dc\u3001\u6c34\u679c\u3001\u5168\u8c37\u7269\u548c\u8102\u80aa\u542b\u91cf\u4f4e\u7684\u86cb\u767d\u8d28\u98df\u7269\uff0c\u907f\u514d\u9ad8\u7cd6\u3001\u9ad8\u8102\u80aa\u548c\u52a0\u5de5\u98df\u54c1\uff0c\u4ee5\u4fdd\u6301\u5065\u5eb7\u7684\u996e\u98df\u4e60\u60ef\u3002\n\n3. \u7761\u7720\u5145\u8db3\u3002\u7761\u7720\u5bf9\u4eba\u4f53\u5065\u5eb7\u81f3\u5173\u91cd\u8981\uff0c\u6210\u5e74\u4eba\u6bcf\u5929\u5e94\u4fdd\u8bc1 7-8 \u5c0f\u65f6\u7684\u7761\u7720\u3002\u826f\u597d\u7684\u7761\u7720\u6709\u52a9\u4e8e\u51cf\u8f7b\u538b\u529b\uff0c\u4fc3\u8fdb\u8eab\u4f53\u6062\u590d\uff0c\u5e76\u63d0\u9ad8\u6ce8\u610f\u529b\u548c\u8bb0\u5fc6\u529b\u3002",
"preference": [
{
"status": "submitted",
"user_id": "c658ddde-2d39-43ce-b478-633a1d19d2c7",
"value": {
"rank": [
2,
3,
1
],
"value": [
"A",
"B",
"C"
]
}
}
],
"preference-suggestion": null,
"preference-suggestion-metadata": {
"agent": null,
"score": null,
"type": null
},
"question-rating": [
{
"status": "submitted",
"user_id": "c658ddde-2d39-43ce-b478-633a1d19d2c7",
"value": 4
}
],
"question-rating-suggestion": null,
"question-rating-suggestion-metadata": {
"agent": null,
"score": null,
"type": null
},
"sentiment": [
{
"status": "submitted",
"user_id": "c658ddde-2d39-43ce-b478-633a1d19d2c7",
"value": "\u4e2d\u7acb"
}
],
"sentiment-suggestion": null,
"sentiment-suggestion-metadata": {
"agent": null,
"score": null,
"type": null
},
"suggestion": [
{
"status": "submitted",
"user_id": "c658ddde-2d39-43ce-b478-633a1d19d2c7",
"value": "111"
}
],
"suggestion-suggestion": null,
"suggestion-suggestion-metadata": {
"agent": null,
"score": null,
"type": null
},
"topics": [
{
"status": "submitted",
"user_id": "c658ddde-2d39-43ce-b478-633a1d19d2c7",
"value": [
"\u903b\u8f91\u95ee\u9898",
"\u5730\u7406\u77e5\u8bc6"
]
}
],
"topics-suggestion": null,
"topics-suggestion-metadata": {
"agent": null,
"score": null,
"type": null
}
}
```

### Data Fields

Among the dataset fields, we differentiate between the following:

* **Fields:** These are the dataset records themselves, for the moment just text fields are supported. These are the ones that will be used to provide responses to the questions.

* **instruction** is of type `text`.
* **input** is of type `text`.
* **output** is of type `text`.

* **Questions:** These are the questions that will be asked to the annotators. They can be of different types, such as `RatingQuestion`, `TextQuestion`, `LabelQuestion`, `MultiLabelQuestion`, and `RankingQuestion`.

* (optional) **question-rating** is of type `rating` with the following allowed values [1, 2, 3, 4, 5, 6, 7, 8, 9].
* **context-rating** is of type `rating` with the following allowed values [1, 2, 3, 4, 5, 6, 7, 8, 9].
* **preference** is of type `ranking` with the following allowed values ['A', 'B', 'C'].
* **suggestion** is of type `text`.
* **topics** is of type `multi_label_selection` with the following allowed values ['数学问题', '逻辑问题', '地理知识', '文学知识', '居家知识', '行业知识'].
* **sentiment** is of type `label_selection` with the following allowed values ['积极', '中立', '消极'].

* **Suggestions:** As of Argilla 1.13.0, the suggestions have been included to provide the annotators with suggestions to ease or assist during the annotation process. Suggestions are linked to the existing questions, are always optional, and contain not just the suggestion itself, but also the metadata linked to it, if applicable.

* (optional) **question-rating-suggestion** is of type `rating` with the following allowed values [1, 2, 3, 4, 5, 6, 7, 8, 9].
* (optional) **context-rating-suggestion** is of type `rating` with the following allowed values [1, 2, 3, 4, 5, 6, 7, 8, 9].
* (optional) **preference-suggestion** is of type `ranking` with the following allowed values ['A', 'B', 'C'].
* (optional) **suggestion-suggestion** is of type `text`.
* (optional) **topics-suggestion** is of type `multi_label_selection` with the following allowed values ['数学问题', '逻辑问题', '地理知识', '文学知识', '居家知识', '行业知识'].
* (optional) **sentiment-suggestion** is of type `label_selection` with the following allowed values ['积极', '中立', '消极'].

Additionally, we also have two more fields that are optional and are the following:

* **metadata:** This is an optional field that can be used to provide additional information about the dataset record. This can be useful to provide additional context to the annotators, or to provide additional information about the dataset record itself. For example, you can use this to provide a link to the original source of the dataset record, or to provide additional information about the dataset record itself, such as the author, the date, or the source. The metadata is always optional, and can be potentially linked to the `metadata_properties` defined in the dataset configuration file in `argilla.yaml`.
* **external_id:** This is an optional field that can be used to provide an external ID for the dataset record. This can be useful if you want to link the dataset record to an external resource, such as a database or a file.

### Data Splits

The dataset contains a single split, which is `train`.

## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation guidelines

[More Information Needed]

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

[More Information Needed]

### Citation Information

[More Information Needed]

### Contributions

[More Information Needed]

原始信息汇总

数据集卡片 for demo

数据集描述

• 主页: Argilla
• 仓库: Argilla GitHub
• 论文:
• 排行榜:
• 联系人:

数据集摘要

该数据集包含：

• 符合 Argilla 数据集格式的配置文件 argilla.yaml。该配置文件将在使用 Argilla 的 FeedbackDataset.from_huggingface 方法时用于配置数据集。
• 兼容 HuggingFace datasets 格式的数据集记录。这些记录将在使用 FeedbackDataset.from_huggingface 时自动加载，也可以通过 datasets 库的 load_dataset 方法独立加载。
• 用于构建和整理数据集的标注指南（如果已在 Argilla 中定义）。

加载数据集

使用 Argilla 加载

安装 Argilla：

bash pip install argilla --upgrade

加载数据集：

python import argilla as rg

ds = rg.FeedbackDataset.from_huggingface("CodeTheory/demo")

使用 datasets 库加载

安装 datasets 库：

bash pip install datasets --upgrade

加载数据集：

python from datasets import load_dataset

ds = load_dataset("CodeTheory/demo")

支持的任务和排行榜

该数据集可以包含多个字段、问题和响应，因此可以用于不同的 NLP 任务，具体取决于配置。数据集结构在数据集结构部分中描述。

该数据集没有关联的排行榜。

语言

[更多信息需要]

数据集结构

数据字段

字段

• instruction：类型为 text。
• input：类型为 text。
• output：类型为 text。

问题

• (可选) question-rating：类型为 rating，允许值为 [1, 2, 3, 4, 5, 6, 7, 8, 9]。
• context-rating：类型为 rating，允许值为 [1, 2, 3, 4, 5, 6, 7, 8, 9]。
• preference：类型为 ranking，允许值为 [A, B, C]。
• suggestion：类型为 text。
• topics：类型为 multi_label_selection，允许值为 [数学问题, 逻辑问题, 地理知识, 文学知识, 居家知识, 行业知识]。
• sentiment：类型为 label_selection，允许值为 [积极, 中立, 消极]。

建议

• (可选) question-rating-suggestion：类型为 rating，允许值为 [1, 2, 3, 4, 5, 6, 7, 8, 9]。
• (可选) context-rating-suggestion：类型为 rating，允许值为 [1, 2, 3, 4, 5, 6, 7, 8, 9]。
• preference-suggestion：类型为 ranking，允许值为 [A, B, C]。
• suggestion-suggestion：类型为 text。
• topics-suggestion：类型为 multi_label_selection，允许值为 [数学问题, 逻辑问题, 地理知识, 文学知识, 居家知识, 行业知识]。
• sentiment-suggestion：类型为 label_selection，允许值为 [积极, 中立, 消极]。

其他字段

• metadata：可选字段，用于提供数据集记录的额外信息。
• external_id：可选字段，用于提供数据集记录的外部 ID。

数据分割

数据集包含一个分割，即 train。

数据集创建

整理理由

[更多信息需要]

源数据

初始数据收集和规范化

[更多信息需要]

源语言生产者

[更多信息需要]

标注

标注指南

[更多信息需要]

标注过程

[更多信息需要]

标注者

[更多信息需要]

个人和敏感信息

[更多信息需要]

使用数据的注意事项

数据集的社会影响

[更多信息需要]

讨论偏见

[更多信息需要]

其他已知限制

[更多信息需要]

附加信息

数据集策展人

[更多信息需要]

许可信息

[更多信息需要]

引用信息

[更多信息需要]

贡献

[更多信息需要]