webglm-qa

# WebGLM-QA

## Dataset Description
[WebGLM-QA](https://github.com/THUDM/WebGLM) is the dataset used to train the WebGLM generator module. It consists of 43,579 high-quality data samples for the train split, 1,000 for the validation split, and 400 for the test split. Refer to [our paper](https://arxiv.org/abs/2306.07906) for the data construction details.

## Dataset Structure
To load the dataset, you can try the following code.

```python
from datasets import load_dataset
load_dataset("THUDM/webglm-qa")
DatasetDict({
train: Dataset({
features: ['question', 'answer', 'references'],
num_rows: 43579
})
test: Dataset({
features: ['question', 'answer', 'references'],
num_rows: 400
})
validation: Dataset({
features: ['question', 'answer', 'references'],
num_rows: 1000
})
})
```

```python
next(iter(data["test"]))
{'question': 'Just got my (Canadian) mortgage renewal notice telling me I have to chose Subsequent Payment Terms.', 'answer': "When renewing a mortgage in Canada, your lender must notify you in advance of the renewal date with your options for renewal terms[1][2]. Your mortgage will typically automatically renew or become in default if you don't take action[3]. Depending on your lender, you may be able to renew your mortgage as early as 6 months prior to your current mortgage term expiring[2][3][5]. RBC Royal Bank mortgage customers can choose Subsequent Payment Terms and be protected from an increase in interest rates for the interest type and term they selected[4].", 'references': ['When faced with a mortgage renewal, this simply means that your current contracted mortgage term is approaching its expiration date. You see, the majority of mortgages in Toronto and in general mortgages in Ontario are contracted for a finite period of time that is referred to as the “mortgage term”. This period tends to range from as little as a few months to as long as 10 years in Canada.', 'You can either proactively reach out to your lender several months prior to your renewal date to find out, but if you don’t, your lender must notify you in advance of the renewal date what your options are. If your mortgage does happen to be with a federally regulated bank, then they are obligated to send you an official renewal statement with no less than 21 days remaining on your current mortgage term. Also, if your lender chooses not to renew your mortgage then they must notify you in advance and provide you with enough time to refinance your mortgage elsewhere or to pay it off.', 'When it comes to mortgage renewals, if you do not take action your mortgage will in many cases either renew automatically or become in default. When your mortgage term approaches the end, your mortgage lender will typically offer you renewal terms that you may choose to accept, negotiate, or decline. Provided you continue to make your monthly mortgage payments on time, lenders will rarely not extend to you an offer to renew your mortgage, although this can happen without cause depending on your mortgage commitment and contract.', "When you renew your RBC Royal Bank mortgage at maturity, you are protected from an increase in interest rates, for the interest type and term you selected, in the 30-day period prior to your regularly scheduled renewal date. And, if the interest rate changes before your actual mortgage renewal date, you'll automatically receive the lower rate for the term and type you chose.", 'When renewing your mortgage in Canada, some lenders may allow you to renew your mortgage as early as 6 months prior to your current mortgage term expiring.']}
```

## Data Fields

* ``question``: a question raised by a user or individual related to a certain topic.
* ``answer``: the generated response to the question.
* ``references``: a list of quotes or snippets from sources used to generate the answer given.

## Data Splits

We split the dataset into train, validation, and test.

## Citation Information

Refer to https://github.com/THUDM/WebGLM.

# WebGLM-QA
## 数据集说明
[WebGLM-QA](https://github.com/THUDM/WebGLM) 是用于训练WebGLM生成器模块的数据集。该数据集包含三类划分：训练集含43579条高质量数据样本，验证集含1000条，测试集含400条。有关数据集构建的详细细节，请参阅[我们的论文](https://arxiv.org/abs/2306.07906)。

## 数据集结构
若需加载该数据集，可尝试如下代码：
python
from datasets import load_dataset
load_dataset("THUDM/webglm-qa")
DatasetDict({
train: Dataset({
features: ['question', 'answer', 'references'],
num_rows: 43579
})
test: Dataset({
features: ['question', 'answer', 'references'],
num_rows: 400
})
validation: Dataset({
features: ['question', 'answer', 'references'],
num_rows: 1000
})
})

python
next(iter(data["test"]))
{'question': 'Just got my (Canadian) mortgage renewal notice telling me I have to chose Subsequent Payment Terms.', 'answer': "When renewing a mortgage in Canada, your lender must notify you in advance of the renewal date with your options for renewal terms[1][2]. Your mortgage will typically automatically renew or become in default if you don't take action[3]. Depending on your lender, you may be able to renew your mortgage as early as 6 months prior to your current mortgage term expiring[2][3][5]. RBC Royal Bank mortgage customers can choose Subsequent Payment Terms and be protected from an increase in interest rates for the interest type and term they selected[4].", 'references': ['When faced with a mortgage renewal, this simply means that your current contracted mortgage term is approaching its expiration date. You see, the majority of mortgages in Toronto and in general mortgages in Ontario are contracted for a finite period of time that is referred to as the "mortgage term". This period tends to range from as little as a few months to as long as 10 years in Canada.', 'You can either proactively reach out to your lender several months prior to your renewal date to find out, but if you don’t, your lender must notify you in advance of the renewal date what your options are. If your mortgage does happen to be with a federally regulated bank, then they are obligated to send you an official renewal statement with no less than 21 days remaining on your current mortgage term. Also, if your lender chooses not to renew your mortgage then they must notify you in advance and provide you with enough time to refinance your mortgage elsewhere or to pay it off.', 'When it comes to mortgage renewals, if you do not take action your mortgage will in many cases either renew automatically or become in default. When your mortgage term approaches the end, your mortgage lender will typically offer you renewal terms that you may choose to accept, negotiate, or decline. Provided you continue to make your monthly mortgage payments on time, lenders will rarely not extend to you an offer to renew your mortgage, although this can happen without cause depending on your mortgage commitment and contract.', "When you renew your RBC Royal Bank mortgage at maturity, you are protected from an increase in interest rates, for the interest type and term you selected, in the 30-day period prior to your regularly scheduled renewal date. And, if the interest rate changes before your actual mortgage renewal date, you'll automatically receive the lower rate for the term and type you chose.", 'When renewing your mortgage in Canada, some lenders may allow you to renew your mortgage as early as 6 months prior to your current mortgage term expiring.']}

## 数据字段
* `question`：用户或个体针对特定主题提出的问题。
* `answer`：针对该问题生成的应答内容。
* `references`：生成该应答时所引用的来源原文或片段列表。

## 数据集划分
我们将该数据集划分为训练集、验证集与测试集三个子集。

## 引用信息
引用详情请参阅 https://github.com/THUDM/WebGLM。